action #165434
closed
OSD SSL certificates not always refreshed within expected time, probably only after system reboots size:S
Added by okurz 3 months ago.
Updated 3 months ago.
Category:
Regressions/Crashes
Description
Observation¶
See
or live on
https://stats.openqa-monitor.qa.suse.de/d/E9tyiQ17k/ssl-certificate-alerts?orgId=1&from=1712673350510&to=1723974839587&viewPanel=5
it is expected that the SSL certificates are refreshed as visible on the left-hand side so that the validity is always at least 3 weeks. Since around 2024-05-19 it seems that the certificates are not always refreshed as before anyway although we have never ran out of a valid certificate. Today, 2024-08-18, it seems that the certificate validity period decreased to just below the 6 days alerting threshold just before OSD rebooted which seemingly triggered a refresh.
Acceptance criteria¶
- AC1: OSD SSL certificates are ensured to refresh consistently well above the alerting threshold
Suggestions¶
Files
- Subject changed from OSD SSL certificates not always refreshed within expected time, probably only after system reboots to OSD SSL certificates not always refreshed within expected time, probably only after system reboots size:S
- Description updated (diff)
- Status changed from New to Workable
- Status changed from Workable to Resolved
- Assignee set to nicksinger
We have a hook-script in /etc/dehydrated/postrun-hooks.d/reload-webserver.sh
which is populated based on the grain webserver
(see https://gitlab.suse.de/openqa/salt-states-openqa/-/blob/master/certificates/dehydrated.sls?ref_type=heads#L37) but this grain was never changed to nginx and was still set to apache2. I manually changed /etc/salt/grains
to contain webserver: apache2
and called salt 'openqa.suse.de' saltutil.sync_grains
, verified with:
openqa:/etc/dehydrated # salt 'openqa.suse.de' grains.get webserver
openqa.suse.de:
nginx
and deployed with:
openqa:/etc/dehydrated # salt 'openqa.suse.de' state.sls_id /etc/dehydrated/postrun-hooks.d/reload-webserver.sh certificates.dehydrated
openqa.suse.de:
----------
ID: /etc/dehydrated/postrun-hooks.d/reload-webserver.sh
Function: file.managed
Result: True
Comment: File /etc/dehydrated/postrun-hooks.d/reload-webserver.sh updated
Started: 14:27:29.212692
Duration: 45.262 ms
Changes:
----------
diff:
---
+++
@@ -1,2 +1,2 @@
#!/bin/sh
-systemctl reload apache2
+systemctl reload nginx
Summary for openqa.suse.de
------------
Succeeded: 1 (changed=1)
Failed: 0
------------
Total states run: 1
Total run time: 45.262 ms
Unfortunately I reloaded nginx while testing so we will have to wait another 3 weeks before we see if it worked. But our alerts will let us know.
- Related to action #167458: openqa.oqa.prg2.suse.org SAN validity alert added
- Copied to action #169078: dashboard.qam.suse.de SSL certificate not deployed within expiry size:S added
Also available in: Atom
PDF