QA (public) » openQA Project (public) » openQA Tests (public)

still important. I removed other tickets from the backlog to make room for this one.

Judging by the instructions on https://en.opensuse.org/openSUSE:Slowroll#Use the setup still requires disabling online repositories during the installation and manual changes to repositories. Do we really want to put effort into automating that (considering these manual steps hopefully won't be required for long anymore)?

I downloaded the DVD iso from https://download.opensuse.org/slowroll/iso/ into o3(fixed/). Most of the first basics test cases seems ok.

• mediacheck
• uefi

One test which failed was https://openqa.opensuse.org/tests/3849913#step/updates_packagekit_gpk/3 more due to the needling.
Note that the tests above are without the proper repos as they are described on https://en.opensuse.org/openSUSE:Slowroll

For this i guess, we need to update some tests modules or something.

at least mediacheck test now works after some needling.
For the rest i created https://github.com/os-autoinst/os-autoinst-needles-opensuse/pull/806 to fix the operations when the DISABLE_ONLINE_REPOS is used. Based on the Bernhard Slowroll needs the DVD in offline mode. And as such i enabled DISABLE_ONLINE_REPOS in Slowroll test variables.
Once this is merged, some modification will need to add the Slowroll repos

ybonatakis wrote in #note-14:

https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/18422 created which adds the Slowroll repos. https://aquarius.suse.cz/tests/20021#step/slowroll_repos/1
I am putting it on Feedback. I dont think the requirements are fulfilled. The jobgroup is added in Development Group and i believe this is not triggered automatically. I will talk with the QA-Core for followup steps

please see https://progress.opensuse.org/issues/135731#Suggestions which mention https://github.com/os-autoinst/openqa-trigger-from-obs/ for automatic triggering. Also https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/18422 adds Slowroll repos for the released product. I don't see how that makes sense. I suggest to compare against Tumbleweed Staging and development tests how they look like.

Also https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/18422 adds Slowroll repos for the released product

Please check the open questions. I see at least one q regarding the maintainer - a brief comment is enough, or a reminder in Slack if you were waiting on someone e.g. @szarate to get back to you

And the next step as discussed would be Extend https://github.com/os-autoinst/openqa-trigger-from-obs/ based on openSUSE:Factory / Tumbleweed

Discussed in the Unlock. Relevant notes:

• https://github.com/os-autoinst/openqa-trigger-from-obs has some documentation but it's omitting crucial parts like how it interacts with the ObsRsync plugin for openQA
• We need a new XML file, and a new job group (there is one but only for staging - remember we don't care about staging for now)
• https://github.com/os-autoinst/openqa-trigger-from-obs/commit/f16bc25a10d606afa0c070e5d0a9cabb556dbb88
• https://openqa.opensuse.org/tests/3905404
• Check with Bernhard or Fabian, they've added new XML configs most recently
  • Let us know what comes out of it so we can reduce the bus factor

Talked to bernhard and he told me that it makes more sense to test the images published in openSUSE:Slowroll:Staging.

As mentioned in the daily, using https://github.com/os-autoinst/openqa-trigger-from-obs/pull/239 is likely the best option now.

The pr looks good so far and seems to produce working scripts.

What I found out so far that needs to be done to get a project tested in openQA:

• add xml xml/obs/openSUSE:Slowroll:Staging-images.xml
• on ariel:
• cd /opt/openqa-trigger-from-obs
• git pull --rebase
• run make openSUSE:Slowroll:Staging
• cd openSUSE:Slowroll:Staging

• on this folder there have some scripts been generated for syncing images and tirggering jobs

• openQA will call /opt/openqa-trigger-from-obs/script/rsync.sh $project which will call those scripts when api route /obs_rsync/#project/runs is called

• that api route seems to be called by https://github.com/openSUSE/openSUSE-release-tools/blob/master/gocd/notify-obs_rsync.py

• notify-obs_rsync.py gets the list of projects to handle from https://openqa.opensuse.org/api/v1/obs_rsync (openqa-cli api --o3 obs_rsync)
  which will just look at the subdirs of /opt/openqa-trigger-from-obs that do NOT match ^(t|profiles|script|xml)$

So how do we find out if this works? Things look a little inconsistent:

• In the product log the most recent Slowroll I see is 23 days old
• The OBS Sync dashboard says published on 2024-02-07 11:01:14 +0000
• The latest Slowroll build openQA knows about is 29 days old
• Minion dashboard ran a obs_rsync_run job 2 hours ago with no errors
  • On OBS it says standard is building, images is published at the time of this writing (this is not a permalink)

Also: Do we care about build errors on OBS?

https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/18622

Bernhard wrote today:

https://download.opensuse.org/repositories/openSUSE:/Slowroll:/Staging/images/ got a new openSUSE-Tumbleweed-Minimal-VM.x86_64-1.0.0-kvm-and-xen-Build4.25.qcow2 but openqa doesn't triggert on it

Let's see if this helps:
https://github.com/openSUSE/openSUSE-release-tools/pull/3060

https://github.com/openSUSE/openSUSE-release-tools/pull/3060 merged since long and as far as I read the minutes from openSUSE release meetings Slowroll is fine with just staging openQA tests so far.

@bmwiedemann https://en.opensuse.org/openSUSE:Slowroll still states "Untested packages go into https://build.opensuse.org/project/show/openSUSE:Slowroll:Staging first and are tested by openQA (TBD)". What needs to be done so that we can state that Slowroll is sufficiently tested by openQA?

There are some workflows that still need re-design. It is not only related to openQA.

Currently we only test one JeOS image in openQA, while the documented way of installation is to use the old Tumbleweed DVD iso snapshot in an offline install and then change the repos.
That install process still needs to be improved. https://build.opensuse.org/request/show/1162881 can help a bit there.

We could also test upgrades from Leap 15.5 to Slowroll.

The JeOS qcow2 is built in the Staging context with package versions that might have been rebuilt after being released into openSUSE:Slowroll. So there is some risk that what we test is not what will be released.

ok, all those suggestions sound sensible to me but guess you don't need exactly the QE Tools team to do that then, right? So should we declare this ticket with the original goal to have "some" production tests as "Resolved"?