Project

General

Profile

action #112916

openQA Project - coordination #109846: [epic] Ensure all our database tables accomodate enough data, e.g. bigint for ids

postgresql.conf is invalid after recent salt changes size:M

Added by cdywan about 2 months ago. Updated about 1 month ago.

Status:
Resolved
Priority:
Normal
Assignee:
Target version:
Start date:
2022-06-22
Due date:
% Done:

0%

Estimated time:

Description

Observation

  • The current /srv/PSQL/data/postgresql.conf is invalid and contains lines and parts of lines containing "tMB".
  • Postgres was last restarted yesterday i.e. active (running) since Wed 2022-06-22 15:54:59 CEST; 19h ago acording to systemd meaning it was not restarted after the config changes
  • This seems to have been introduced by https://gitlab.suse.de/openqa/salt-states-openqa/-/commit/606eb48d1c54e25d6528c45dd230cf02d815f3d4
  • All pipelines succeeded
  • The config was last changed this morning, and there is an identical .bak file, and we don't know why: -rw------- 1 postgres postgres 29374 Jun 23 10:47 postgresql.conf -rw------- 1 root root 29374 Jun 23 09:48 postgresql.conf.bak

Suggestions


Related issues

Copied from openQA Infrastructure - coordination #112718: [alert][osd] openqa.suse.de is not reachable anymore, response times > 30s, multiple alerts over the weekendResolved2022-06-22

History

#1 Updated by cdywan about 2 months ago

  • Copied from coordination #112718: [alert][osd] openqa.suse.de is not reachable anymore, response times > 30s, multiple alerts over the weekend added

#2 Updated by tinita about 2 months ago

  • Subject changed from posttgresql.conf is invalid after recent salt changes size:M to postgresql.conf is invalid after recent salt changes size:M

#3 Updated by okurz about 2 months ago

  • Tracker changed from coordination to action
  • Status changed from New to In Progress

manually reverting the config file from https://gitlab.suse.de/openqa/salt-states-openqa/-/jobs/1026706#L1434 and restarted the postgresql service. The file was shortly overwritten again, likely by salt, but nevermind because I will fix that soon in the salt repo. But in the broken state I called in psql select pg_reload_conf(); which still shows "t" despite the config being invalid. This is in line with https://stackoverflow.com/questions/16333319/how-to-syntax-check-postgresql-config-files#comment101836284_55758799 which also states that this does not work. Also https://stackoverflow.com/a/16340569 states that there is no separate check config command. One needs to try to restart the server which refuses to load an invalid config.

https://github.com/os-autoinst/salt-states-openqa/blob/master/openqa/server.sls#L330 is within a block {%- if not grains.get('noservices', False) %} so we won't see those rules effective in CI jobs.

#4 Updated by okurz about 2 months ago

  • Status changed from In Progress to Feedback

Manually testing the changes on OSD in a separate file with
salt-call --local -l error --state-output=changes state.sls_id postgresql-work_mem2 openqa.postgresql test=True
which takes about 10s each run – why is salt so slow – but ok.

Trying out the pattern manually with grep or sed is fine but with salt the replacement ends up as tMB

https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/705

#5 Updated by okurz about 2 months ago

  • Due date set to 2022-07-07

#6 Updated by okurz about 1 month ago

  • Due date deleted (2022-07-07)
  • Status changed from Feedback to Resolved

We crosschecked and it's all good

Also available in: Atom PDF