Project

General

Profile

Actions
Copy link

action #132692

closed

Support SUMA engineers to setup a SUMA containers test & release pipeline

Added by jlausuch 12 months ago. Updated 7 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
ph03nix
Target version:
-
Start date:
2023-07-13
Due date:
% Done:

0%

Estimated time:
Tags:
suma

Description

SUMA engineers, in particular Maurizio Galli, contacted us because they need a pipeline to continuosly update the following SUMA containers:

  • proxy-helm
  • proxy-httpd-image
  • proxy-salt-broker-image
  • proxy-squid-image
  • proxy-ssh-image
  • proxy-tftpd-image

They already have setup the needed projects in IBS so that the container bot can interact with:
CR: https://build.suse.de/project/show/SUSE:SLE-15-SP4:Update:Products:Manager43:Update:CR
Test: https://build.suse.de/project/show/SUSE:SLE-15-SP4:Update:Products:Manager43:Update:CR:ToTest
Release: https://build.suse.de/project/show/SUSE:Containers:SUSE-Manager:4.3

Requirements:

  • The tests can be run on 15-SP4 host
  • Only x86_64
  • 1 single job per container image (similar to Rancher container image)
  • Use 15-SP4 Host HDD_1=SLES-15-SP4-x86_64-containers.qcow2

What SUMA team will do:

  • Provide a MR in the container release bot with a new yaml file suma.yaml to add these images
  • Provide a new test module in os-autoinst-distri-opensuse with minimal coverage, similar to what we do with the Rancher conatiner

What QE-C can help with:

  • Create a new sub job group in Containers parent group called "SUMA Containers" or just "SUMA".
  • Populate that new group with a new yaml file (it could be hosted in
  • Help the SUMA engineers to adapt main_containers.pm to schedule the test module that they will create.
  • Make sure the IBS user we use in the bot has read/write access to the the new CR projects, and ask Autobuild team to provide access.
Actions
Copy link
 #1

Updated by mauriziogalli 12 months ago

MR for with the SUMA yaml files created: https://gitlab.suse.de/qac/container-release-bot/-/merge_requests/172

Next step is to provide the openQA tests for the images. openQA Test to be similar to Rancher's as we only need basic testing such as pull image from registry. https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/containers/slem_rancher.pm

For openQA test using the variable CONTAINER_IMAGE_TO_TEST, however note that helm works slightly different, for example we need to call it with oci:// (i.e helm pull oci://registry.suse.de/suse/sle-15-sp4/update/products/manager43/update/cr/totest/charts/suse/manager/4.3/proxy)

Actions
Copy link
 #2

Updated by mauriziogalli 12 months ago

Initial draft for openQA basic test for SUMA container images: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/17437

Actions
Copy link
 #3

Updated by ph03nix 11 months ago

  • Status changed from New to In Progress
  • Assignee set to ph03nix
Actions
Copy link
 #5

Updated by ph03nix 10 months ago

Still waiting for Maurizio

Actions
Copy link
 #6

Updated by ph03nix 10 months ago

Had a short session with Maurizio, we agreed that I will do the VRs and share the results in the PR.

Maurizio shared the container location on registry.suse.de with me and I will now perform a bunch of VRs.

For my future self:

openqa-cli api --host http://duck-norris.qe.suse.de -X POST isos ARCH=x86_64 DISTRI=sle BUILD=petrosilius GROUP=0 VERSION=15-SP5 FLAVOR=BCI-Updates CONTAINERS_UNTESTED_IMAGES=1 CONTAINER_SUMA=image BCI_IMAGE_NAME=suma-proxy-squid CONTAINER_IMAGE_TO_TEST=registry.suse.de/suse/sle-15-sp4/update/products/manager43/update/cr/containers/suse/manager/4.3/proxy-squid CASEDIR=https://github.com/m4u9/os-autoinst-distri-opensuse.git#suma-containers2
Actions
Copy link
 #7

Updated by ph03nix 10 months ago

proxy-helm needs helm.

proxy-helm:                   registry.suse.de/suse/sle-15-sp4/update/products/manager43/update/cr/charts/suse/manager/4.3/proxy-helm

The other images need podman.

proxy-httpd-image:            registry.suse.de/suse/sle-15-sp4/update/products/manager43/update/cr/containers/suse/manager/4.3/proxy-httpd
proxy-salt-broker-image:      registry.suse.de/suse/sle-15-sp4/update/products/manager43/update/cr/containers/suse/manager/4.3/proxy-salt-broker
proxy-squid-image:            registry.suse.de/suse/sle-15-sp4/update/products/manager43/update/cr/containers/suse/manager/4.3/proxy-squid
proxy-ssh-image:              registry.suse.de/suse/sle-15-sp4/update/products/manager43/update/cr/containers/suse/manager/4.3/proxy-ssh
proxy-tftpd-image:            registry.suse.de/suse/sle-15-sp4/update/products/manager43/update/cr/containers/suse/manager/4.3/proxy-tftpd
Actions
Copy link
 #8

Updated by ph03nix 10 months ago 

#!/bin/bash -e


oqa_cli="openqa-cli"
IMAGES="proxy-httpd-image proxy-salt-broker-image proxy-squid-image proxy-ssh-image proxy-tftpd-image"



# suma-proxy-helm is special
$oqa_cli api --host http://duck-norris.qe.suse.de -X POST isos ARCH=x86_64 DISTRI=sle BUILD=suma-proxy-helm GROUP=0 VERSION=15-SP5 FLAVOR=BCI-Updates CONTAINERS_UNTESTED_IMAGES=1 CONTAINER_SUMA=image BCI_IMAGE_NAME=suma-proxy-helm CONTAINER_IMAGE_TO_TEST=registry.suse.de/suse/sle-15-sp4/update/products/manager43/update/cr/charts/suse/manager/4.3/proxy-helm CASEDIR=https://github.com/m4u9/os-autoinst-distri-opensuse.git#suma-containers2



for image in $IMAGES
do
    $oqa_cli api --host http://duck-norris.qe.suse.de -X POST isos ARCH=x86_64 DISTRI=sle BUILD=suma-$image GROUP=0 VERSION=15-SP5 FLAVOR=BCI-Updates CONTAINERS_UNTESTED_IMAGES=1 CONTAINER_SUMA=image BCI_IMAGE_NAME=suma-$image CONTAINER_IMAGE_TO_TEST=registry.suse.de/suse/sle-15-sp4/update/products/manager43/update/cr/containers/suse/manager/4.3/$image CASEDIR=https://github.com/m4u9/os-autoinst-distri-opensuse.git#suma-containers2
done
Actions
Copy link
 #9

Updated by ph03nix 10 months ago

VRs scheduled in https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/17437, waiting for reply from Maurizio

Actions
Copy link
 #10

Updated by ph03nix 9 months ago

https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/17437 should now be in a good state. VRs restarted, waiting for PR rebase and then we can move forward.

Actions
Copy link
 #11

Updated by ph03nix 8 months ago

https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/17437 is merged! Now I need to figure out how to schedule them in the crb.

Actions
Copy link
 #12

Updated by ph03nix 8 months ago

https://gitlab.suse.de/qac/container-release-bot/-/merge_requests/172 is still needed. I asked Maurizio weather he wants to work on it or if we should take over. I'm happy with both ways.

Actions
Copy link
 #13

Updated by ph03nix 8 months ago

https://gitlab.suse.de/qac/container-release-bot/-/merge_requests/172 is merged. Let's see how the first test runs are going to behave.

Actions
Copy link
 #14

Updated by ph03nix 8 months ago

I don't see any scheduled jobs in https://openqa.suse.de/group_overview/516?limit_builds=50&limit_builds=10. Looking at the latest pipeline on the crb, I see the following:

DEBUG:urllib3.connectionpool:https://openqa.suse.de:443 "POST /api/v1/isos?DISTRI=sle&VERSION=15-SP4&ARCH=x86_64&FLAVOR=Container-Image-Updates&BUILD=15-SP4_37.28&_GROUP_ID=516&CONTAINER_IMAGE_BUILD=37.28&CONTAINERS_UNTESTED_IMAGES=1&PUBLIC_CLOUD_TOOLS_HDD=publiccloud_tools_0057.qcow2&RETRY=1&CONTAINER_IMAGE_TO_TEST=registry.suse.de%2Fsuse%2Fsle-15-sp4%2Fupdate%2Fproducts%2Fmanager43%2Fupdate%2Fcr%2Ftotest%2Fcontainers%2Fsuse%2Fmanager%2F4.3%2Fproxy-httpd%3Alatest&CONTAINER_SUMA=image HTTP/1.1" 404 131
SUMA-proxy-httpd-x86_64 - error: ('POST', 'https://openqa.suse.de/api/v1/isos?DISTRI=sle&VERSION=15-SP4&ARCH=x86_64&FLAVOR=Container-Image-Updates&BUILD=15-SP4_37.28&_GROUP_ID=516&CONTAINER_IMAGE_BUILD=37.28&CONTAINERS_UNTESTED_IMAGES=1&PUBLIC_CLOUD_TOOLS_HDD=publiccloud_tools_0057.qcow2&RETRY=1&CONTAINER_IMAGE_TO_TEST=registry.suse.de%2Fsuse%2Fsle-15-sp4%2Fupdate%2Fproducts%2Fmanager43%2Fupdate%2Fcr%2Ftotest%2Fcontainers%2Fsuse%2Fmanager%2F4.3%2Fproxy-httpd%3Alatest&CONTAINER_SUMA=image', 404)
ERROR:root:('POST', 'https://openqa.suse.de/api/v1/isos?DISTRI=sle&VERSION=15-SP4&ARCH=x86_64&FLAVOR=Container-Image-Updates&BUILD=15-SP4_37.28&_GROUP_ID=516&CONTAINER_IMAGE_BUILD=37.28&CONTAINERS_UNTESTED_IMAGES=1&PUBLIC_CLOUD_TOOLS_HDD=publiccloud_tools_0057.qcow2&RETRY=1&CONTAINER_IMAGE_TO_TEST=registry.suse.de%2Fsuse%2Fsle-15-sp4%2Fupdate%2Fproducts%2Fmanager43%2Fupdate%2Fcr%2Ftotest%2Fcontainers%2Fsuse%2Fmanager%2F4.3%2Fproxy-httpd%3Alatest&CONTAINER_SUMA=image', 404)
INFO:root:Processing SUMA-proxy-salt-broker (1 architectures)
INFO:root:Processing SUMA-proxy-salt-broker-x86_64

Now checking why this one in throwing a 404.

Actions
Copy link
 #15

Updated by ph03nix 8 months ago

Not sure who https://openqa.suse.de/admin/job_templates/516 created and maintains. I asked Maurizio.

Actions
Copy link
 #17

Updated by ph03nix 8 months ago

Actions
Copy link
 #19

Updated by ph03nix 7 months ago

  • Status changed from In Progress to Feedback

I see pending test runs for

  • SUMA-proxy-tftpd
  • SUMA-proxy-ssh
  • SUMA-proxy-squid
  • SUMA-proxy-salt-broker
  • SUMA-proxy-httpd

Gonna have a look and if they all pass then we can consider this job as done.

Actions
Copy link
 #20

Updated by ph03nix 7 months ago

  • Status changed from Feedback to Resolved

ph03nix wrote in #note-19:

I see pending test runs for

  • SUMA-proxy-tftpd
  • SUMA-proxy-ssh
  • SUMA-proxy-squid
  • SUMA-proxy-salt-broker
  • SUMA-proxy-httpd

Gonna have a look and if they all pass then we can consider this job as done.

I see https://openqa.suse.de/tests/overview?groupid=516&version=15-SP4&build=15-SP4_40.17&distri=sle as a build or suma-proxy-httpd, but not for other containers.

For the others: I checked https://gitlab.suse.de/qac/container-release-bot/-/jobs/2033126

INFO:root:SUMA-proxy-tftpd-x86_64: Build (CR|CR:ToTest|Release): 30.16 | 30.16 | 30.16
INFO:root:SUMA-proxy-ssh-x86_64: Build (CR|CR:ToTest|Release): 30.14 | 30.14 | 30.14
INFO:root:SUMA-proxy-squid-x86_64: Build (CR|CR:ToTest|Release): 39.15 | 39.15 | 39.15
INFO:root:SUMA-proxy-salt-broker-x86_64: Build (CR|CR:ToTest|Release): 30.18 | 30.18 | 30.18

They are all up-to-date. I think we can close the ticket as resolved.

Actions
Copy link

Also available in: Atom PDF