action #132692
closedSupport SUMA engineers to setup a SUMA containers test & release pipeline
0%
Description
SUMA engineers, in particular Maurizio Galli, contacted us because they need a pipeline to continuosly update the following SUMA containers:
proxy-helm
proxy-httpd-image
proxy-salt-broker-image
proxy-squid-image
proxy-ssh-image
proxy-tftpd-image
They already have setup the needed projects in IBS so that the container bot can interact with:
CR: https://build.suse.de/project/show/SUSE:SLE-15-SP4:Update:Products:Manager43:Update:CR
Test: https://build.suse.de/project/show/SUSE:SLE-15-SP4:Update:Products:Manager43:Update:CR:ToTest
Release: https://build.suse.de/project/show/SUSE:Containers:SUSE-Manager:4.3
Requirements:
- The tests can be run on 15-SP4 host
- Only x86_64
- 1 single job per container image (similar to Rancher container image)
- Use 15-SP4 Host
HDD_1=SLES-15-SP4-x86_64-containers.qcow2
What SUMA team will do:
- Provide a MR in the container release bot with a new yaml file suma.yaml to add these images
- Provide a new test module in os-autoinst-distri-opensuse with minimal coverage, similar to what we do with the Rancher conatiner
What QE-C can help with:
- Create a new sub job group in Containers parent group called "SUMA Containers" or just "SUMA".
- Populate that new group with a new yaml file (it could be hosted in
- Help the SUMA engineers to adapt main_containers.pm to schedule the test module that they will create.
- Make sure the IBS user we use in the bot has read/write access to the the new CR projects, and ask Autobuild team to provide access.
Updated by mauriziogalli about 1 year ago
MR for with the SUMA yaml files created: https://gitlab.suse.de/qac/container-release-bot/-/merge_requests/172
Next step is to provide the openQA tests for the images. openQA Test to be similar to Rancher's as we only need basic testing such as pull image from registry. https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/containers/slem_rancher.pm
For openQA test using the variable CONTAINER_IMAGE_TO_TEST, however note that helm works slightly different, for example we need to call it with oci:// (i.e helm pull oci://registry.suse.de/suse/sle-15-sp4/update/products/manager43/update/cr/totest/charts/suse/manager/4.3/proxy
)
Updated by mauriziogalli about 1 year ago
Initial draft for openQA basic test for SUMA container images: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/17437
Updated by ph03nix 12 months ago
MR for the container-release-bot: https://gitlab.suse.de/qac/container-release-bot/-/merge_requests/172
Updated by ph03nix 11 months ago
Had a short session with Maurizio, we agreed that I will do the VRs and share the results in the PR.
Maurizio shared the container location on registry.suse.de with me and I will now perform a bunch of VRs.
For my future self:
openqa-cli api --host http://duck-norris.qe.suse.de -X POST isos ARCH=x86_64 DISTRI=sle BUILD=petrosilius GROUP=0 VERSION=15-SP5 FLAVOR=BCI-Updates CONTAINERS_UNTESTED_IMAGES=1 CONTAINER_SUMA=image BCI_IMAGE_NAME=suma-proxy-squid CONTAINER_IMAGE_TO_TEST=registry.suse.de/suse/sle-15-sp4/update/products/manager43/update/cr/containers/suse/manager/4.3/proxy-squid CASEDIR=https://github.com/m4u9/os-autoinst-distri-opensuse.git#suma-containers2
Updated by ph03nix 11 months ago
proxy-helm needs helm
.
proxy-helm: registry.suse.de/suse/sle-15-sp4/update/products/manager43/update/cr/charts/suse/manager/4.3/proxy-helm
The other images need podman
.
proxy-httpd-image: registry.suse.de/suse/sle-15-sp4/update/products/manager43/update/cr/containers/suse/manager/4.3/proxy-httpd
proxy-salt-broker-image: registry.suse.de/suse/sle-15-sp4/update/products/manager43/update/cr/containers/suse/manager/4.3/proxy-salt-broker
proxy-squid-image: registry.suse.de/suse/sle-15-sp4/update/products/manager43/update/cr/containers/suse/manager/4.3/proxy-squid
proxy-ssh-image: registry.suse.de/suse/sle-15-sp4/update/products/manager43/update/cr/containers/suse/manager/4.3/proxy-ssh
proxy-tftpd-image: registry.suse.de/suse/sle-15-sp4/update/products/manager43/update/cr/containers/suse/manager/4.3/proxy-tftpd
Updated by ph03nix 11 months ago
#!/bin/bash -e
oqa_cli="openqa-cli"
IMAGES="proxy-httpd-image proxy-salt-broker-image proxy-squid-image proxy-ssh-image proxy-tftpd-image"
# suma-proxy-helm is special
$oqa_cli api --host http://duck-norris.qe.suse.de -X POST isos ARCH=x86_64 DISTRI=sle BUILD=suma-proxy-helm GROUP=0 VERSION=15-SP5 FLAVOR=BCI-Updates CONTAINERS_UNTESTED_IMAGES=1 CONTAINER_SUMA=image BCI_IMAGE_NAME=suma-proxy-helm CONTAINER_IMAGE_TO_TEST=registry.suse.de/suse/sle-15-sp4/update/products/manager43/update/cr/charts/suse/manager/4.3/proxy-helm CASEDIR=https://github.com/m4u9/os-autoinst-distri-opensuse.git#suma-containers2
for image in $IMAGES
do
$oqa_cli api --host http://duck-norris.qe.suse.de -X POST isos ARCH=x86_64 DISTRI=sle BUILD=suma-$image GROUP=0 VERSION=15-SP5 FLAVOR=BCI-Updates CONTAINERS_UNTESTED_IMAGES=1 CONTAINER_SUMA=image BCI_IMAGE_NAME=suma-$image CONTAINER_IMAGE_TO_TEST=registry.suse.de/suse/sle-15-sp4/update/products/manager43/update/cr/containers/suse/manager/4.3/$image CASEDIR=https://github.com/m4u9/os-autoinst-distri-opensuse.git#suma-containers2
done
Updated by ph03nix 11 months ago
VRs scheduled in https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/17437, waiting for reply from Maurizio
Updated by ph03nix 9 months ago
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/17437 should now be in a good state. VRs restarted, waiting for PR rebase and then we can move forward.
Updated by ph03nix 9 months ago
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/17437 is merged! Now I need to figure out how to schedule them in the crb.
Updated by ph03nix 9 months ago
https://gitlab.suse.de/qac/container-release-bot/-/merge_requests/172 is still needed. I asked Maurizio weather he wants to work on it or if we should take over. I'm happy with both ways.
Updated by ph03nix 9 months ago
https://gitlab.suse.de/qac/container-release-bot/-/merge_requests/172 is merged. Let's see how the first test runs are going to behave.
Updated by ph03nix 9 months ago
I don't see any scheduled jobs in https://openqa.suse.de/group_overview/516?limit_builds=50&limit_builds=10. Looking at the latest pipeline on the crb, I see the following:
DEBUG:urllib3.connectionpool:https://openqa.suse.de:443 "POST /api/v1/isos?DISTRI=sle&VERSION=15-SP4&ARCH=x86_64&FLAVOR=Container-Image-Updates&BUILD=15-SP4_37.28&_GROUP_ID=516&CONTAINER_IMAGE_BUILD=37.28&CONTAINERS_UNTESTED_IMAGES=1&PUBLIC_CLOUD_TOOLS_HDD=publiccloud_tools_0057.qcow2&RETRY=1&CONTAINER_IMAGE_TO_TEST=registry.suse.de%2Fsuse%2Fsle-15-sp4%2Fupdate%2Fproducts%2Fmanager43%2Fupdate%2Fcr%2Ftotest%2Fcontainers%2Fsuse%2Fmanager%2F4.3%2Fproxy-httpd%3Alatest&CONTAINER_SUMA=image HTTP/1.1" 404 131
SUMA-proxy-httpd-x86_64 - error: ('POST', 'https://openqa.suse.de/api/v1/isos?DISTRI=sle&VERSION=15-SP4&ARCH=x86_64&FLAVOR=Container-Image-Updates&BUILD=15-SP4_37.28&_GROUP_ID=516&CONTAINER_IMAGE_BUILD=37.28&CONTAINERS_UNTESTED_IMAGES=1&PUBLIC_CLOUD_TOOLS_HDD=publiccloud_tools_0057.qcow2&RETRY=1&CONTAINER_IMAGE_TO_TEST=registry.suse.de%2Fsuse%2Fsle-15-sp4%2Fupdate%2Fproducts%2Fmanager43%2Fupdate%2Fcr%2Ftotest%2Fcontainers%2Fsuse%2Fmanager%2F4.3%2Fproxy-httpd%3Alatest&CONTAINER_SUMA=image', 404)
ERROR:root:('POST', 'https://openqa.suse.de/api/v1/isos?DISTRI=sle&VERSION=15-SP4&ARCH=x86_64&FLAVOR=Container-Image-Updates&BUILD=15-SP4_37.28&_GROUP_ID=516&CONTAINER_IMAGE_BUILD=37.28&CONTAINERS_UNTESTED_IMAGES=1&PUBLIC_CLOUD_TOOLS_HDD=publiccloud_tools_0057.qcow2&RETRY=1&CONTAINER_IMAGE_TO_TEST=registry.suse.de%2Fsuse%2Fsle-15-sp4%2Fupdate%2Fproducts%2Fmanager43%2Fupdate%2Fcr%2Ftotest%2Fcontainers%2Fsuse%2Fmanager%2F4.3%2Fproxy-httpd%3Alatest&CONTAINER_SUMA=image', 404)
INFO:root:Processing SUMA-proxy-salt-broker (1 architectures)
INFO:root:Processing SUMA-proxy-salt-broker-x86_64
Now checking why this one in throwing a 404.
Updated by ph03nix 9 months ago
Not sure who https://openqa.suse.de/admin/job_templates/516 created and maintains. I asked Maurizio.
Updated by ph03nix 9 months ago
ph03nix wrote in #note-16:
https://gitlab.suse.de/qac/qac-openqa-yaml/-/merge_requests/1384 and https://gitlab.suse.de/qac/container-release-bot/-/merge_requests/192
merged, we will see tomorrow, if this has the expected impact.
Updated by ph03nix 9 months ago
Test runs are not scheduled:
- https://openqa.suse.de/tests/overview?distri=sle&version=15-SP4&build=15-SP4_37.28&groupid=516
- https://openqa.suse.de/tests/overview?distri=sle&version=15-SP4&build=15-SP4_36.24&groupid=516
- https://openqa.suse.de/tests/overview?distri=sle&version=15-SP4&build=15-SP4_27.27&groupid=516
- https://openqa.suse.de/tests/overview?distri=sle&version=15-SP4&build=15-SP4_27.24&groupid=516
Fix that the container name is not displayed: https://gitlab.suse.de/qac/container-release-bot/-/merge_requests/195
Updated by ph03nix 8 months ago
- Status changed from Feedback to Resolved
ph03nix wrote in #note-19:
I see pending test runs for
- SUMA-proxy-tftpd
- SUMA-proxy-ssh
- SUMA-proxy-squid
- SUMA-proxy-salt-broker
- SUMA-proxy-httpd
Gonna have a look and if they all pass then we can consider this job as done.
I see https://openqa.suse.de/tests/overview?groupid=516&version=15-SP4&build=15-SP4_40.17&distri=sle as a build or suma-proxy-httpd
, but not for other containers.
For the others: I checked https://gitlab.suse.de/qac/container-release-bot/-/jobs/2033126
INFO:root:SUMA-proxy-tftpd-x86_64: Build (CR|CR:ToTest|Release): 30.16 | 30.16 | 30.16
INFO:root:SUMA-proxy-ssh-x86_64: Build (CR|CR:ToTest|Release): 30.14 | 30.14 | 30.14
INFO:root:SUMA-proxy-squid-x86_64: Build (CR|CR:ToTest|Release): 39.15 | 39.15 | 39.15
INFO:root:SUMA-proxy-salt-broker-x86_64: Build (CR|CR:ToTest|Release): 30.18 | 30.18 | 30.18
They are all up-to-date. I think we can close the ticket as resolved.