Project

General

Profile

action #72202

[qe-core] setup openSUSE Leap 15.3 openQA

Added by msmeissn 10 months ago. Updated 6 days ago.

Status:
In Progress
Priority:
Urgent
Assignee:
Category:
New test
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

(this old ticket started as a ticket for Jump, later modified to be about Leap 15.3)

We need to setup openSUSE Leap 15.3 openqa.

It will have 3 update repos it consumes:

http://download.opensuse.org/update/leap/15.3/sle/
http://download.opensuse.org/repositories/openSUSE:/Backports:/SLE-15-SP3:/Update/standard/
http://download.opensuse.org/repositories/openSUSE:/Maintenance:/Test:/Leap_15.3/

the full test repo is at:

the incidents are below openSUSE Maintenance as usual, example incident openSUSE:Maintenance:14199

Current Build ISOs are below http://download.opensuse.org/distribution/leap/15.3/

[Acceptance Criteria]

  • Write also the documentation of the process preferably in confluence, see notes in comment #note-13

* Clarify why are we testing what on leap :)

This would need at least PR to https://github.com/openSUSE/openSUSE-release-tools/tree/master/data


Related issues

Related to openSUSE Leap 15.0 - action #25032: enable openQA maintenance testingResolved2018-04-012018-04-19

Related to openSUSE Leap 15.1 - action #39449: enable openQA maintenance testingResolved2019-03-112019-04-05

Related to openQA Tests - action #91827: [qe-core] Write tests and documentation for oqamaint botWorkable

Precedes openSUSE Leap 15.3 - action #76702: [qe-core][u] disable iso download in openQARejected2021-02-022021-02-02

History

#1 Updated by msmeissn 10 months ago

https://progress.opensuse.org/issues/70957 tracker issue in jump space

#3 Updated by tjyrinki_suse 10 months ago

  • Status changed from New to Workable
  • Priority changed from Normal to High
  • Start date deleted (2020-10-02)

#4 Updated by hrommel1 9 months ago

  • Due date set to 2020-10-30

Please evaluate until the deadline if there are any challenges with setting up.

Planned release is in November.

#5 Updated by geor 9 months ago

  • Priority changed from High to Urgent

#6 Updated by geor 9 months ago

  • Description updated (diff)

#7 Updated by geor 9 months ago

  • Subject changed from setup openSUSE Jump 15.2 openqa to [sprint] setup openSUSE Jump 15.2 openqa

#8 Updated by punkioudi 9 months ago

  • Assignee set to punkioudi

#9 Updated by punkioudi 9 months ago

Hi msmeissn,

Right now there are these runs :

So what more is it expected from the openqa tests in relation to these repos? Is it sufficient for the system to be up-to-date, by using the aforementioned repositories and then do the testing?
thanks!

#10 Updated by msmeissn 9 months ago

These are the groups for product development before release.

What we also need is the openqa setup for the maintenance phase.

e.g. for Leap 15.2: https://openqa.opensuse.org/group_overview/69 and https://openqa.opensuse.org/group_overview/70

#11 Updated by punkioudi 9 months ago

Thank you msmeissn for the answers!

After some research though I did today, an implementation of the openQA bot is also required which is handled by osukup (who has time-off this week). Consequently, this ticket can be handled on the beginning of next week.

tjyrinki_suse , it is an urgent+sprint item, but I have to postpone it for next week, will it be ok? :)

#12 Updated by tjyrinki_suse 9 months ago

Sure ok punkioudi, it's urgent but we need Ondrej indeed.

#13 Updated by szarate 9 months ago

So what I get from asking around:

#14 Updated by szarate 9 months ago

  • Due date changed from 2020-10-30 to 2021-02-01
  • Priority changed from Urgent to Low

So after double checking with Marcus, this ticket is not urgent anymore as the Jump 15.2.1 received a no-go.

I'd like to announce that we have a final no-Go for Leap 15.2.1 after talking to stakeholders or engineers working on blocking issues.

https://lists.opensuse.org/opensuse-project/2020-11/msg00001.html

#15 Updated by msmeissn 9 months ago

  • Subject changed from [sprint] setup openSUSE Jump 15.2 openqa to [sprint] setup openSUSE Jump 15.3 openqa

change to be the 15.3 setup ticket.

#16 Updated by punkioudi 9 months ago

  • Subject changed from [sprint] setup openSUSE Jump 15.3 openqa to setup openSUSE Jump 15.3 openqa

I remove also the sprint tag for now. Thanks szarate and msmeissn for the update!

#17 Updated by punkioudi 9 months ago

  • Assignee deleted (punkioudi)

#18 Updated by szarate 9 months ago

  • Description updated (diff)

#19 Updated by szarate 9 months ago

  • Related to action #25032: enable openQA maintenance testing added

#20 Updated by szarate 9 months ago

  • Related to action #39449: enable openQA maintenance testing added

#21 Updated by szarate 9 months ago

See linked tickets for future reference

#22 Updated by tjyrinki_suse 9 months ago

  • Subject changed from setup openSUSE Jump 15.3 openqa to [qe-core] setup openSUSE Jump 15.3 openqa

#23 Updated by tjyrinki_suse 8 months ago

  • Project changed from QAM to openQA Tests
  • Priority changed from Low to Normal

#24 Updated by tjyrinki_suse 8 months ago

  • Category set to New test

#25 Updated by msmeissn 6 months ago

  • Subject changed from [qe-core] setup openSUSE Jump 15.3 openqa to [qe-core] setup Leap Jump 15.3 openqa

Name is openSUSE Leap 15.3.

#26 Updated by msmeissn 6 months ago

  • Subject changed from [qe-core] setup Leap Jump 15.3 openqa to [qe-core] setup openSUSE Leap 15.3 openqa

we need openqa setup, and it will be a bit different.

we can have incidents now for 2 different locations

  • openSUSE Backports SLE-15-SP3
    test incident: openSUSE:Maintenance:15678

  • openSUSE Leap 15.3

test incident: openSUSE:Maintenance:15681

both targets will go into Leap 15.3.

#27 Updated by tjyrinki_suse 6 months ago

  • Priority changed from Normal to Urgent

This needs bumping in priority and finding someone who can configure O3 accordingly.

#28 Updated by tjyrinki_suse 6 months ago

I've pinged osukup about this.

#29 Updated by szarate 6 months ago

  • Precedes action #76702: [qe-core][u] disable iso download in openQA added

#31 Updated by szarate 5 months ago

  • Description updated (diff)

#32 Updated by szarate 5 months ago

  • Description updated (diff)

Santiago to clarify if it's jump or leap

#33 Updated by szarate 5 months ago

  • Description updated (diff)

#34 Updated by msmeissn 5 months ago

it is called leap now. jump was only for 15.2.1, but 15.3 will be called leap, also in the directory paths.

#35 Updated by tjyrinki_suse 5 months ago

  • Description updated (diff)

#36 Updated by tjyrinki_suse 5 months ago

  • Description updated (diff)

#37 Updated by tjyrinki_suse 5 months ago

  • Due date deleted (2021-02-01)
  • Target version set to Ready

#38 Updated by tjyrinki_suse 5 months ago

  • Target version changed from Ready to QE-Core: Ready

#39 Updated by tjyrinki_suse 5 months ago

  • Target version changed from QE-Core: Ready to Ready

#40 Updated by szarate 5 months ago

  • Target version changed from Ready to QE-Core: Ready

#41 Updated by tjyrinki_suse 5 months ago

  • Description updated (diff)
  • Target version changed from QE-Core: Ready to future

#42 Updated by szarate 4 months ago

  • Assignee set to szarate
  • Target version changed from future to QE-Core: Ready

I'm picking this up while in training.

For now: https://github.com/openSUSE/openSUSE-release-tools/pull/2537 has been created based on the information that we have within the ticket.

#43 Updated by tjyrinki_suse 4 months ago

  • Status changed from Workable to In Progress

#44 Updated by szarate 4 months ago

  • How to test this?
  • Who consumes/what the json files modified in the PR
  • How do we check that the results are correct? (i.E that maintenance updates will be picked up)
  • What does the syncing of the jobgroups between Maintenance and OBS once the leap product is released

#45 Updated by msmeissn 4 months ago

someone needs to setup 15.3 job groups in openqa.opensuse.org. (antonios?)

There will be code changes needed in opensuse-release-tools and os-autoinst-distri-opensuse, as we now have 2 different looking incidents.

what is not handled yet, are incidents like this:

openSUSE:Maintenance:15953 hello

As we now have 2 sources for updates, openSUSE:Leap:15.3:Update AND openSUSE:Backports:SLE-15-SP3:Update to test for Leap 15.3.

openSUSE:Maintenance:15681 is the other possible case, with rpmlint-mini

BOTH cases must be handled.

#46 Updated by apappas 3 months ago

The code changes in the repos have been done, and the mediums and JobGrous have been set. Temporarily the beta iso is being used.

I expected that they woulkd have been triggered. I'll investigate why they haven't.

#47 Updated by apappas 3 months ago

The runs are now posted for "regular" updates but the bot continues to ignore the backports. I just asked the UM if it's possible to send a dummy update to the sle repo so that can be tested.

#48 Updated by msmeissn 3 months ago

openSUSE:Maintenance:15953 - recommended (low): hello
locked: 4
release: 879946, review, 39 days ago

maintenance_release: openSUSE:Maintenance:15953/hello.openSUSE_Backports_SLE-15-SP3_Update -> openSUSE:Backports:SLE-15-SP3:Update/hello.15953
maintenance_release: openSUSE:Maintenance:15953/patchinfo -> openSUSE:Backports:SLE-15-SP3:Update/patchinfo.15953

It was missing a qam-openqa review, I added one just now.

#49 Updated by apappas 3 months ago

  • Related to action #91827: [qe-core] Write tests and documentation for oqamaint bot added

#50 Updated by szarate 3 months ago

  • Assignee changed from szarate to apappas

I think for now, Anton has done more than I have :)

#51 Updated by okurz 2 months ago

  • Subject changed from [qe-core] setup openSUSE Leap 15.3 openqa to [qe-core] setup openSUSE Leap 15.3 openQA

Had a meeting with hrommel, vpelcak, msmeissn, mawerner about two aspects

  1. "repository setup in openQA" which is mainly covered in this ticket. Updates to openSUSE Leap 15.3 can come from 3 different projects:
    • SLE 15 SP3 shared packages
    • PackageHub 'Backports'
    • packages specific to openSUSE Leap 15.3

It is expected that QE-Core takes care of that. In best case this can be fixed with just configuration changes for the openSUSE Maintenance openQA bot, in worst case this might need more changes https://github.com/openSUSE/openSUSE-release-tools/blob/master/openqa-maintenance.py . msmeissn has reminded of the release date of Leap 15.3 in just a couple of days until this issue has been fixed

  1. To avoid iterations after release of SLE 15 SP3 updates caused by failures that show only on openSUSE Leap 15.3, we should incorporate the review of openQA results of openSUSE Leap 15.3 into the release decision of SLE 15 SP3 updates. For this I suggested to extend the existing SLE maintenance tests on openqa.suse.de with according Leap test scenarios. Of course, a good start is to just add one scenario in either aggregate or incident tests and slowly extend from there. hrommel1 as discussed plans to create a specific ticket for that. Responsibility area for the initial setup is QE-Core as well.

#52 Updated by apappas 2 months ago

The configuration for the packages specific to Leap 15.3 has been done and has been going on for some time. The problem is doing the rest of them. Complicating that, is, that to our knowledge is qe-core, the only documentation we have is the code, which means we just can't copy paste the config like we did before.

#53 Updated by tjyrinki_suse 2 months ago

okurz wrote:

It is expected that QE-Core takes care of that. In best case this can be fixed with just configuration changes for the openSUSE Maintenance openQA bot, in worst case this might need more changes https://github.com/openSUSE/openSUSE-release-tools/blob/master/openqa-maintenance.py . msmeissn has reminded of the release date of Leap 15.3 in just a couple of days until this issue has been fixed

Ok.

  1. To avoid iterations after release of SLE 15 SP3 updates caused by failures that show only on openSUSE Leap 15.3, we should incorporate the review of openQA results of openSUSE Leap 15.3 into the release decision of SLE 15 SP3 updates. For this I suggested to extend the existing SLE maintenance tests on openqa.suse.de with according Leap test scenarios. Of course, a good start is to just add one scenario in either aggregate or incident tests and slowly extend from there. hrommel1 as discussed plans to create a specific ticket for that. Responsibility area for the initial setup is QE-Core as well.

If we don't look at O3 results (I think it would be possible to watch a subset of those results for the decision) but want to instead add Leap tests to OSD, then yes makes sense. Respective squads would be adding their tests, butfirst any media/product adding of Leap to OSD would be on QE Core.

#54 Updated by okurz 2 months ago

tjyrinki_suse wrote:

  1. To avoid iterations after release of SLE 15 SP3 updates caused by failures that show only on openSUSE Leap 15.3, we should incorporate the review of openQA results of openSUSE Leap 15.3 into the release decision of SLE 15 SP3 updates. For this I suggested to extend the existing SLE maintenance tests on openqa.suse.de with according Leap test scenarios. Of course, a good start is to just add one scenario in either aggregate or incident tests and slowly extend from there. hrommel1 as discussed plans to create a specific ticket for that. Responsibility area for the initial setup is QE-Core as well.

If we don't look at O3 results (I think it would be possible to watch a subset of those results for the decision) but want to instead add Leap tests to OSD, then yes makes sense. Respective squads would be adding their tests, butfirst any media/product adding of Leap to OSD would be on QE Core.

The main and only reason to use openqa.suse.de is because the unreleased SLE maintenance updates are not available on openqa.opensuse.org. This should not be the default for new tests that run on Leap as Leap is still a community project which should be developed and tested in the open where possible.

#55 Updated by apappas about 2 months ago

  • Status changed from In Progress to Feedback

The Backports part of the ticket has been completeted

What remains are the Leap SLE updates. According to Marcus, there is no Build Service project that publishes these updates, but there is a script that collects SLE15GA->SLE15SP# into the Leap SLE repo. He suggests we test by adding the SLE15SP3 testrepos to a Leap 15.3 installation.

I personally need feedback as to what the test scenarios for those updates will be.

#56 Updated by tjyrinki_suse about 2 months ago

  • Parent task set to #92563

For SLE updates on Leap, I'm suggesting new job groups in OSD that test the same things as these job groups on O3, but by using the prospective SLE incident updates:

I'd suggest starting with cryptlvm/cryptlvm@uefi-2G, gnome/gnome@uefi, textmode@64bit and install_with_updates_gnome - and if those go fine, add KDE as well. x86_64 only! Requires creating also Leap products etc on OSD.

Heiko agreed this sounds like a good initial plan.

#57 Updated by okurz about 2 months ago

how about starting with a single scenario as proof of concept? Especially as there would be no help from any openSUSE community to review and as people are already suffering from the review workload the additional tests should be really limited to only show that the SLE update side of things is not completely broken while keeping the main test load still on o3, right?

#58 Updated by tjyrinki_suse about 2 months ago

  • Status changed from Feedback to In Progress

I think the above should be enough to get forward with the OSD SLE-on-Leap updates.

#59 Updated by maritawerner 14 days ago

Today there was a meeting with Marcus Meissner, Antonios, Santi, Anna and myself with this topics:

  • Anton has finished the set-up of the channels for 15.3, all good
  • For 15.4 Lubos will create a progress ticket at Beta1 to inform QE that openQa needs to be set-up for SP4
  • The review of the Leap maintenance job groups is still under discussion, Santi will follow-up with an email, the idea is that the QE Core team will take over the review -> more info will come
  • Anna & QE Core team will maintain the tests for Leap and increase the test coverage step by step

#60 Updated by okurz 6 days ago

waiting for apappas and osukup regarding scheduling tests within OSD. Would it better if we can not wait for 1:1 talks but better coordinate on team level. Maybe I can offer some help?

Also available in: Atom PDF