Project

General

Profile

Actions
Copy link

action #167458

closed

openqa.oqa.prg2.suse.org SAN validity alert

Added by livdywan 4 months ago. Updated 4 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
nicksinger
Category:
Regressions/Crashes
Target version:
openQA Project (public) - Ready
Start date:
Due date:
% Done:

0%

Estimated time:
Tags:
alert, infra

Description

Observation

On openqa.oqa.prg2.suse.org - SAN validity it says:

https://openqa.nue.suse.com:443 validity left
6 days
https://openqa.oqa.prg2.suse.org:443 validity left
6 days
https://openqa.suse.de:443 validity left
6 days

Acceptance Criteria

  • AC1: A new certificate is known to be available and used
  • AC2: No alert is observed

Suggestions

  • Check why the certificate wasn't automatically renewed
  • See what was done in #134522

Related issues 2 (0 open — 2 closed)

Related to openQA Infrastructure (public) - action #165434: OSD SSL certificates not always refreshed within expected time, probably only after system reboots size:SResolvednicksinger2024-08-18

Actions
Copied from openQA Infrastructure (public) - action #134522: [alert] Certificate renewal on monitor.qa.suse.de might not be working causing alerts size:MResolvednicksinger2023-08-232023-09-07

Actions
Actions
Copy link
 #1

Updated by livdywan 4 months ago

  • Copied from action #134522: [alert] Certificate renewal on monitor.qa.suse.de might not be working causing alerts size:M added
Actions
Copy link
 #2

Updated by okurz 4 months ago

  • Category set to Regressions/Crashes
  • Priority changed from High to Urgent
Actions
Copy link
 #3

Updated by nicksinger 4 months ago

  • Related to action #165434: OSD SSL certificates not always refreshed within expected time, probably only after system reboots size:S added
Actions
Copy link
 #4

Updated by nicksinger 4 months ago

  • Status changed from New to In Progress

Reason was pretty much the same as described in https://progress.opensuse.org/issues/165434#note-2 . But back then I only changed the grain manually and not in salt so it got reset to apache2 again and we always just got lucky until now that something else restarted nginx (e.g. updates, reboots, etc).

Actions
Copy link
 #5

Updated by okurz 4 months ago

  • Assignee set to nicksinger
Actions
Copy link
 #6

Updated by tinita 4 months ago

The new cert files are there, it just may not be visible from the webserver because the webserver wasn't restarted since 2024-09-08.
We have the same issue on o3.

Actions
Copy link
 #7

Updated by nicksinger 4 months ago

  • Status changed from In Progress to Feedback
  • Priority changed from Urgent to Normal

https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/1273 created to hopefully fix it for good now. I tested it by changing the according line manually and run a highstate twice on OSD (first to apply the new grain, second to update the script based on that new grain). It updated /etc/dehydrated/postrun-hooks.d/reload-webserver.sh as expected and executing it updates the cert as expected:

workstation ~ ยป echo | openssl s_client -connect openqa.suse.de:443 | openssl x509 -noout -enddate
depth=3 C = DE, ST = Franconia, L = Nuremberg, O = SUSE Linux Products GmbH, OU = OPS Services, CN = SUSE Trust Root, emailAddress = rd-adm@suse.de
verify return:1
depth=2 C = DE, ST = Franconia, L = Nuremberg, O = SUSE Linux Products GmbH, OU = OPS Services, CN = SUSE CA Root, emailAddress = rd-adm@suse.de
verify return:1
depth=1 C = DE, ST = Franconia, L = Nuremberg, O = SUSE Linux Products GmbH, OU = OPS Services, CN = SUSE CA all 2023.1, emailAddress = infra@suse.de
verify return:1
depth=0 CN = openqa.oqa.prg2.suse.org
verify return:1
DONE
notAfter=Oct 26 00:38:28 2024 GMT
Actions
Copy link
 #8

Updated by nicksinger 4 months ago

  • Status changed from Feedback to Resolved
Actions
Copy link

Also available in: Atom PDF