QA » openQA Project » openQA Tests » Containers

FYI, you can connect to the postgresql database on k2 via

podman exec -ti -u postgres img-size-db psql --user sizemon

The openqa tests actually pushing data to this POSTGRES URL http://k2.qe.suse.de:8080/size are all yaml jeos and containers/[latest | bci/[15sp*|app|lang]]_images.yaml, but only for podman runtime on SLE-15-SP5 host, enabled by IMAGE_STORE_DATA=1.

The routines to push or check the data are in os-autoinst-distri-opensuse/lib/db_utils.pm: push_image_data_to_db and check_postgres_db.

A simple check of db-alive, can be done from any host (in VPN): curl -ILf http://k2.qe.suse.de:8080/size

Noted that, while working yesterday on it, running the check N times, I had always (regular) alternated ok then no reply from host, therefore suggested to also add a timeout, in loop, to improve check, for curl option -m 10 i.e.:
curl -ILf -m 10 http://k2.qe.suse.de:8080/size.

More investigation on this behavior would be useful, too.

See on this, Felix's PR https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/19404

Noted that:

while running the connection to k2 server on port 8080, like:
curl -I -m 10 http://k2.qe.suse.de:8080/size
regularly: first run reply is ok, second next no-reply;

on k2:/var/log/nginx/ we see corresponding the messages 1-2 in the logs:

a) img-mon.access

...
2024/05/31 12:32:53 [info] 1247#1247: *449 client 10.149.209.54 closed keepalive connection 
2024/05/31 12:33:06 [info] 1247#1247: *451 epoll_wait() reported that client prematurely closed connection, so upstream connection is closed too while sending request to upstream, client: 10.149.209.54, server: image-monitoring.suse.de, request: "HEAD /size HTTP/1.1", upstream: "http://[::1]:5444/size", host: "k2.qe.suse.de:8080"

b) img-mon.a.error:

...
10.149.209.54 - - [31/May/2024:13:01:46 +0000] "HEAD /size HTTP/1.1" 200 0 "-" "curl/8.7.1" 
10.149.209.54 - - [31/May/2024:13:01:59 +0000] "HEAD /size HTTP/1.1" 499 0 "-" "curl/8.7.1"
...

always alternating results.

Looking in the log (a) the port 5444, if we run the same connections on that port http://k2.qe.suse.de:5444/size, the result is always ok, but no log written in a) b).

Work someway delayed for other urgencies

I would nail the issue down to IPv6. Using IPv4 I could query the instance in 100/100 times. For IPv6, every second request fails.

phoenix@racetrack-7290:~> curl -6 -I http://k2.qe.suse.de:8080/size
HTTP/1.1 200 OK
Server: nginx/1.27.0
Date: Wed, 26 Jun 2024 12:49:11 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Content-Range: 0-99/*
Content-Location: /size

phoenix@racetrack-7290:~> curl -6 -I http://k2.qe.suse.de:8080/size
^C
phoenix@racetrack-7290:~> curl -6 -I http://k2.qe.suse.de:8080/size
HTTP/1.1 200 OK
Server: nginx/1.27.0
Date: Wed, 26 Jun 2024 12:49:35 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Content-Range: 0-99/*
Content-Location: /size

phoenix@racetrack-7290:~> curl -6 -I http://k2.qe.suse.de:8080/size
^C
phoenix@racetrack-7290:~> curl -6 -I http://k2.qe.suse.de:8080/size
HTTP/1.1 200 OK
Server: nginx/1.27.0
Date: Wed, 26 Jun 2024 12:50:30 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Content-Range: 0-99/*
Content-Location: /size

phoenix@racetrack-7290:~> curl -6 -I http://k2.qe.suse.de:8080/size
^C
phoenix@racetrack-7290:~> curl -6 -I http://k2.qe.suse.de:8080/size
HTTP/1.1 200 OK
Server: nginx/1.27.0
Date: Wed, 26 Jun 2024 12:51:28 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Content-Range: 0-99/*
Content-Location: /size

phoenix@racetrack-7290:~> curl -6 -I http://k2.qe.suse.de:8080/size
^C
phoenix@racetrack-7290:~> curl -6 -I http://k2.qe.suse.de:8080/size
HTTP/1.1 200 OK
Server: nginx/1.27.0
Date: Wed, 26 Jun 2024 12:51:30 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Content-Range: 0-99/*
Content-Location: /size

phoenix@racetrack-7290:~> curl -6 -I http://k2.qe.suse.de:8080/size
^C
phoenix@racetrack-7290:~> curl -6 -I http://k2.qe.suse.de:8080/size
HTTP/1.1 200 OK
Server: nginx/1.27.0
Date: Wed, 26 Jun 2024 12:51:33 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Content-Range: 0-99/*
Content-Location: /size

I now also see the issue with IPv4.