action #169078
Updated by mkittler about 1 month ago
## Observation
The certificate for dashboard.qam.suse.de expired on 10/30/2024.
The problem was basically resolved when I started to investigate it. Most likely dehydrated did not restart nginx after renewing the certificate 2 weeks ago like we've seen before.
```
Oct 17 00:11:15 qam2 dehydrated[28476]: # INFO: Using main config file /etc/dehydrat>
Oct 17 00:11:16 qam2 dehydrated[28476]: Processing qam2.suse.de with alternative nam>
Oct 17 00:11:16 qam2 dehydrated[28476]: + Checking domain name(s) of existing cert.>
Oct 17 00:11:16 qam2 dehydrated[28476]: + Checking expire date of existing cert...
Oct 17 00:11:16 qam2 dehydrated[28476]: + Valid till Oct 29 23:30:31 2024 GMT (Less>
Oct 17 00:11:16 qam2 dehydrated[28476]: + Signing domains...
Oct 17 00:11:16 qam2 dehydrated[28476]: + Generating private key...
Oct 17 00:11:17 qam2 dehydrated[28476]: + Generating signing request...
Oct 17 00:11:17 qam2 dehydrated[28476]: + Requesting new certificate order from CA.>
Oct 17 00:11:17 qam2 dehydrated[28476]: + Received 4 authorizations URLs from the CA
Oct 17 00:11:18 qam2 dehydrated[28476]: + Handling authorization for qam2.suse.de
Oct 17 00:11:18 qam2 dehydrated[28476]: + Handling authorization for qam2.qe.prg2.s>
Oct 17 00:11:18 qam2 dehydrated[28476]: + Handling authorization for qam.suse.de
Oct 17 00:11:18 qam2 dehydrated[28476]: + Handling authorization for dashboard.qam.>
Oct 17 00:11:18 qam2 dehydrated[28476]: + 4 pending challenge(s)
Oct 17 00:11:18 qam2 dehydrated[28476]: + Deploying challenge tokens...
Oct 17 00:11:18 qam2 dehydrated[28476]: + Responding to challenge for qam2.suse.de >
Oct 17 00:11:19 qam2 dehydrated[28476]: + Challenge is valid!
Oct 17 00:11:19 qam2 dehydrated[28476]: + Responding to challenge for qam2.qe.prg2.>
Oct 17 00:11:19 qam2 dehydrated[28476]: + Challenge is valid!
Oct 17 00:11:19 qam2 dehydrated[28476]: + Responding to challenge for qam.suse.de a>
Oct 17 00:11:19 qam2 dehydrated[28476]: + Challenge is valid!
Oct 17 00:11:19 qam2 dehydrated[28476]: + Responding to challenge for dashboard.qam>
Oct 17 00:11:19 qam2 dehydrated[28476]: + Challenge is valid!
Oct 17 00:11:19 qam2 dehydrated[28476]: + Cleaning challenge tokens...
Oct 17 00:11:19 qam2 dehydrated[28476]: + Requesting certificate...
Oct 17 00:11:19 qam2 dehydrated[28476]: + Checking certificate...
Oct 17 00:11:19 qam2 dehydrated[28476]: + Done!
Oct 17 00:11:19 qam2 dehydrated[28476]: + Creating fullchain.pem...
Oct 17 00:11:20 qam2 dehydrated[28476]: + Done!
```
## Acceptance criteria
* **AC1:** An updated certificate is used by dashboard.qam.suse.de before the old one expires
* **AC2:** NGINX is using the updated certificate, e.g. is reloaded as needed
## Suggestions
* This was not covered by any alerts but thanks to a user reporting it
* Apparently dashboard.qam.suse.de is not monitored on [SSL Certificate Alerts](https://stats.openqa-monitor.qa.suse.de/d/E9tyiQ17k/ssl-certificate-alerts) so far? Shouldn't it be?
* Investigate why nginx wasn't restarted
* https://gitlab.suse.de/opensuse/qem-dashboard/-/blob/master/.gitlab-ci.yml?ref_type=heads this seems old empty repo
* #165434 shows how it was solved for OSD