action #132461
closedQA - coordination #121720: [saga][epic] Migration to QE setup in PRG2+NUE3 while ensuring availability
QA - coordination #123800: [epic] Provide SUSE QE Tools services running in PRG2 aka. Prg CoLo
manage tls certificates on o3/ariel directly with dehydrated size:M
0%
Description
Motivation¶
(We got informed)[[https://app.slack.com/client/T02863RC2AC/C04MDKHQE20/thread/C04MDKHQE20-1688735468.778099]] that ariel/o3 will have no hydra/ha-proxy setup in the new location. Therefore we need to handle our tls certificates for nginx on our own in the future.
Acceptance criteria¶
- AC1: openqa.opensuse.org has a valid certificate requested by the webhost itself
- AC2: the process is fully automated and certificate renewal requires no human interaction
- AC3: Any generalizable config snippets are in github.com/os-autoinst/openQA/
Suggestions¶
- Install an Lets Encrypt compatible client on ariel (see https://wiki.archlinux.org/title/Transport_Layer_Security#ACME_clients for a list) - nsinger recommends (dehydrated)[[https://github.com/dehydrated-io/dehydrated]]
- Adjust nginx to serve the ACME challenges and reconfigure existing entries to use that new certificate
- Feel welcome to experiment on o3 as long as you monitor closely that everything still works as expected or is quickly reverted on problems
- Submit any generalizable config snippets into github.com/os-autoinst/openQA/, e.g. as commented nginx config templates
Updated by nicksinger 10 months ago
We could already set it up anyway (in parallel to the existing infra). Then we would be prepared for every possibility.
Updated by nicksinger 10 months ago
I was thinking about this and realized that we have to see how the current webserver proxy is setup. If it forwards requests to http://openqa.opensuse.org/.well-known/acme-challenge/ we can already create valid certificates. Otherwise we have no way to respond so Challenges from lets encrypt and won't be able to issue a certificate on our own.
Updated by nicksinger 10 months ago
- Status changed from New to In Progress
- Assignee set to nicksinger
Updated by nicksinger 10 months ago
Installed dehydrated and adjusted /etc/nginx/vhosts.d/openqa.conf
to include conf.d/dehydrated.inc
which basically just serves challenges contained in /var/lib/acme-challenge
from http://openqa.opensuse.org/.well-known/acme-challenge - it is the default location from dehydrated so no further configuration needed. After running:
root@ariel# dehydrated --display-terms
root@ariel# dehydrated --register --accept-terms
I received a valid certificate for o3:
ariel:/etc/dehydrated/certs/openqa.opensuse.org # tree .
.
├── cert-1688989838.csr
├── cert-1688989838.pem
├── cert.csr -> cert-1688989838.csr
├── cert.pem -> cert-1688989838.pem
├── chain-1688989838.pem
├── chain.pem -> chain-1688989838.pem
├── fullchain-1688989838.pem
├── fullchain.pem -> fullchain-1688989838.pem
├── privkey-1688989838.pem
└── privkey.pem -> privkey-1688989838.pem
Updated by nicksinger 10 months ago
I also added a crontab as root to refresh the certificate daily:
0 0 * * * /usr/bin/dehydrated --cron
also adjusted deploy_cert() in /etc/dehydrated/hook.sh
to reload nginx after a certificate got renewed
Updated by okurz 10 months ago
Nice! In https://gitlab.suse.de/openqa/salt-states-openqa/-/blob/master/certificates/dehydrated.sls#L52 we use a systemd timer which I assume is doing the same as the cron line you added. Any reason you selected cron over systemd here?
Updated by nicksinger 10 months ago
- Tags deleted (
infra) - Assignee deleted (
nicksinger) - Target version deleted (
Ready)
okurz wrote:
Nice! In https://gitlab.suse.de/openqa/salt-states-openqa/-/blob/master/certificates/dehydrated.sls#L52 we use a systemd timer which I assume is doing the same as the cron line you added. Any reason you selected cron over systemd here?
I had no particular reason besides being more comfortable with crontab :) But I can switch it to a systemd-timer too.
Updated by openqa_review 10 months ago
- Due date set to 2023-07-25
Setting due date based on mean cycle time of SUSE QE Tools
Updated by okurz 10 months ago
config template addition https://github.com/os-autoinst/openQA/pull/5243 merged. So, anything left?
Updated by nicksinger 10 months ago
- Status changed from In Progress to Resolved
nope, I think we're done here. o3 is now ready to run without hydra proxy