Added by okurz almost 4 years ago. Updated almost 4 years ago.
0%
Description
https://gitlab.suse.de/openqa/osd-deployment/-/jobs/240416 shows
$ eval "$GRAFANA_ALERTS" > current_alerts
ERROR: Job failed: command terminated with exit code 1
meaning that the curl-command itself failed.
https://infra.nue.suse.com/SelfService/Display.html?id=175585 is a related ticket by nicksinger asking for updated certificates and now stats.openqa-monitor.qa.suse.de is not properly reachable anymore, coincidence? Asked jdsn in chat https://chat.suse.de/channel/suse-it-ama?msg=aHmkC28BAoMR2J824
No response in RC nor in https://infra.nue.suse.com/SelfService/Display.html?id=175585 :(
It looks like the current problem with certificate was triggered by nsinger who did changes on the openqa-monitor host as described in #68785#note-8 but I could not find the information within the host itself directly, e.g. no entry in bash history of "root" nor "nsinger". Only that the command last tells me that he did login 2020-08-04 and we see changes:
root@openqa-monitor:/etc/ssl # ls -ltra
total 36
lrwxrwxrwx 1 root root 28 Aug 1 2019 certs -> /var/lib/ca-certificates/pem
lrwxrwxrwx 1 root root 38 Aug 1 2019 ca-bundle.pem -> /var/lib/ca-certificates/ca-bundle.pem
drwx------ 2 root root 4096 Jan 10 2020 private
-rw-r--r-- 1 root root 10771 Jan 10 2020 openssl.cnf
drwxr-xr-x 4 root root 4096 Jan 10 2020 .
drwxr-xr-x 105 root root 12288 Aug 4 03:02 ..
drwxr-x--- 2 root root 4096 Aug 4 19:53 stats.openqa-monitor.qa.suse.de
root@openqa-monitor:/etc/ssl # ls -ltra *
lrwxrwxrwx 1 root root 28 Aug 1 2019 certs -> /var/lib/ca-certificates/pem
lrwxrwxrwx 1 root root 38 Aug 1 2019 ca-bundle.pem -> /var/lib/ca-certificates/ca-bundle.pem
-rw-r--r-- 1 root root 10771 Jan 10 2020 openssl.cnf
private:
total 8
drwx------ 2 root root 4096 Jan 10 2020 .
drwxr-xr-x 4 root root 4096 Jan 10 2020 ..
stats.openqa-monitor.qa.suse.de:
total 48
-rw------- 1 root root 3276 Aug 28 2019 cert.key
-rw-r----- 1 root root 2569 Sep 10 2019 cert.crt
-rw-r--r-- 1 root root 4935 Sep 10 2019 suse_chain.crt
-rw-r--r-- 1 root root 7504 Sep 10 2019 fullchain.crt
drwxr-xr-x 4 root root 4096 Jan 10 2020 ..
-rw-r--r-- 1 root root 422 Aug 4 09:56 req.conf
-rw-r--r-- 1 root root 1890 Aug 4 09:56 cert.csr
drwxr-x--- 2 root root 4096 Aug 4 19:53 .
-rw-r--r-- 1 root root 7500 Aug 4 19:53 monitor.qa.suse.de.chained.crt
just additionally putting the files from https://infra.nue.suse.com/SelfService/Display.html?id=175585 does not seem to help.
maybe I can just revert changes in the directory /etc/ssl/stats.openqa-monitor.qa.suse.de but I do not know how. Asked in ticket for help additionally again.
Created workaround for deployment with https://gitlab.suse.de/openqa/osd-deployment/-/merge_requests/21 now and triggered https://gitlab.suse.de/openqa/osd-deployment/-/pipelines/73517
https://infra.nue.suse.com/SelfService/Display.html?id=175585 was resolved. We have received a new crt file. I copied the file over to openqa-monitor.qa:/etc/ssl/stats.openqa-monitor.qa.suse.de and on openqa-monitor.qa in /etc/ssl/stats.openqa-monitor.qa.suse.de I did:
cat monitor.qa.suse.de.crt SUSE_CA_suse.de.crt SUSE_CA_Root.crt > cert.crt
chromium says that https://stats.openqa-monitor.qa.suse.de/ is ok now. Also for verification nsinger suggested echo | openssl s_client -showcerts -connect 10.162.0.21:443 -servername stats.openqa-monitor.qa.suse.de which looks fine as well. Also fine: https://monitor.qa.suse.de/, https://stats.monitor.qa.suse.de/, https://openqa-monitor.qa.suse.de