Project

General

Profile

Actions
Copy link

action #132461

closed

QA (public) - coordination #121720: [saga][epic] Migration to QE setup in PRG2+NUE3 while ensuring availability

QA (public) - coordination #123800: [epic] Provide SUSE QE Tools services running in PRG2 aka. Prg CoLo

manage tls certificates on o3/ariel directly with dehydrated size:M

Added by nicksinger over 1 year ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
nicksinger
Category:
-
Target version:
openQA Project (public) - Ready
Start date:
2023-07-07
Due date:
% Done:

0%

Estimated time:
Tags:
infra

Description

Motivation

(We got informed)[[https://app.slack.com/client/T02863RC2AC/C04MDKHQE20/thread/C04MDKHQE20-1688735468.778099]] that ariel/o3 will have no hydra/ha-proxy setup in the new location. Therefore we need to handle our tls certificates for nginx on our own in the future.

Acceptance criteria

  • AC1: openqa.opensuse.org has a valid certificate requested by the webhost itself
  • AC2: the process is fully automated and certificate renewal requires no human interaction
  • AC3: Any generalizable config snippets are in github.com/os-autoinst/openQA/

Suggestions

  • Install an Lets Encrypt compatible client on ariel (see https://wiki.archlinux.org/title/Transport_Layer_Security#ACME_clients for a list) - nsinger recommends (dehydrated)[[https://github.com/dehydrated-io/dehydrated]]
  • Adjust nginx to serve the ACME challenges and reconfigure existing entries to use that new certificate
  • Feel welcome to experiment on o3 as long as you monitor closely that everything still works as expected or is quickly reverted on problems
  • Submit any generalizable config snippets into github.com/os-autoinst/openQA/, e.g. as commented nginx config templates
Actions
Copy link
 #1

Updated by okurz over 1 year ago

  • Assignee set to okurz
  • Target version set to Ready
  • Parent task set to #123800

Thank you. lhaleplidis now stated that he plans to work with a proxy so the task is still up for discussion.

Actions
Copy link
 #2

Updated by nicksinger over 1 year ago

We could already set it up anyway (in parallel to the existing infra). Then we would be prepared for every possibility.

Actions
Copy link
 #3

Updated by okurz over 1 year ago

  • Assignee deleted (okurz)

Yes, I think we should do that

Actions
Copy link
 #4

Updated by nicksinger over 1 year ago

I was thinking about this and realized that we have to see how the current webserver proxy is setup. If it forwards requests to http://openqa.opensuse.org/.well-known/acme-challenge/ we can already create valid certificates. Otherwise we have no way to respond so Challenges from lets encrypt and won't be able to issue a certificate on our own.

Actions
Copy link
 #5

Updated by nicksinger over 1 year ago

  • Status changed from New to In Progress
  • Assignee set to nicksinger
Actions
Copy link
 #6

Updated by nicksinger over 1 year ago

Installed dehydrated and adjusted /etc/nginx/vhosts.d/openqa.conf to include conf.d/dehydrated.inc which basically just serves challenges contained in /var/lib/acme-challenge from http://openqa.opensuse.org/.well-known/acme-challenge - it is the default location from dehydrated so no further configuration needed. After running:

root@ariel# dehydrated --display-terms
root@ariel# dehydrated --register --accept-terms

I received a valid certificate for o3:

ariel:/etc/dehydrated/certs/openqa.opensuse.org # tree .
.
├── cert-1688989838.csr
├── cert-1688989838.pem
├── cert.csr -> cert-1688989838.csr
├── cert.pem -> cert-1688989838.pem
├── chain-1688989838.pem
├── chain.pem -> chain-1688989838.pem
├── fullchain-1688989838.pem
├── fullchain.pem -> fullchain-1688989838.pem
├── privkey-1688989838.pem
└── privkey.pem -> privkey-1688989838.pem
Actions
Copy link
 #7

Updated by nicksinger over 1 year ago

I also added a crontab as root to refresh the certificate daily:

0   0   *   *   *       /usr/bin/dehydrated --cron

also adjusted deploy_cert() in /etc/dehydrated/hook.sh to reload nginx after a certificate got renewed

Actions
Copy link
 #8

Updated by okurz over 1 year ago

Nice! In https://gitlab.suse.de/openqa/salt-states-openqa/-/blob/master/certificates/dehydrated.sls#L52 we use a systemd timer which I assume is doing the same as the cron line you added. Any reason you selected cron over systemd here?

Actions
Copy link
 #9

Updated by nicksinger over 1 year ago

  • Tags deleted (infra)
  • Assignee deleted (nicksinger)
  • Target version deleted (Ready)

okurz wrote:

Nice! In https://gitlab.suse.de/openqa/salt-states-openqa/-/blob/master/certificates/dehydrated.sls#L52 we use a systemd timer which I assume is doing the same as the cron line you added. Any reason you selected cron over systemd here?

I had no particular reason besides being more comfortable with crontab :) But I can switch it to a systemd-timer too.

Actions
Copy link
 #10

Updated by okurz over 1 year ago

  • Tags set to infra
  • Assignee set to nicksinger
  • Target version set to Ready
Actions
Copy link
 #11

Updated by openqa_review over 1 year ago

  • Due date set to 2023-07-25

Setting due date based on mean cycle time of SUSE QE Tools

Actions
Copy link
 #12

Updated by okurz over 1 year ago

  • Subject changed from manage tls certificates on o3/ariel directly with dehydrated to manage tls certificates on o3/ariel directly with dehydrated size:M
  • Description updated (diff)
Actions
Copy link
 #13

Updated by okurz over 1 year ago

config template addition https://github.com/os-autoinst/openQA/pull/5243 merged. So, anything left?

Actions
Copy link
 #14

Updated by nicksinger over 1 year ago

  • Status changed from In Progress to Resolved

nope, I think we're done here. o3 is now ready to run without hydra proxy

Actions
Copy link
 #15

Updated by okurz over 1 year ago

  • Due date deleted (2023-07-25)
Actions
Copy link

Also available in: Atom PDF