action #72202
closed[qe-core] setup openSUSE Leap 15.3 openQA
0%
Description
(this old ticket started as a ticket for Jump, later modified to be about Leap 15.3)
We need to setup openSUSE Leap 15.3 openqa.
It will have 3 update repos it consumes:
http://download.opensuse.org/update/leap/15.3/sle/
http://download.opensuse.org/repositories/openSUSE:/Backports:/SLE-15-SP3:/Update/standard/
http://download.opensuse.org/repositories/openSUSE:/Maintenance:/Test:/Leap_15.3/
the full test repo is at:
the incidents are below openSUSE Maintenance as usual, example incident openSUSE:Maintenance:14199
Current Build ISOs are below http://download.opensuse.org/distribution/leap/15.3/
[Acceptance Criteria]
- Write also the documentation of the process preferably in confluence, see notes in comment #note-13
* Clarify why are we testing what on leap :)¶
This would need at least PR to https://github.com/openSUSE/openSUSE-release-tools/tree/master/data
Updated by msmeissn about 4 years ago
https://progress.opensuse.org/issues/70957 tracker issue in jump space
Updated by msmeissn about 4 years ago
http://download.opensuse.org/repositories/openSUSE:/Maintenance:/Test:/Jump_15.2/ is the merged testrepo
Updated by tjyrinki_suse about 4 years ago
- Status changed from New to Workable
- Priority changed from Normal to High
- Start date deleted (
2020-10-02)
Updated by hrommel1 about 4 years ago
- Due date set to 2020-10-30
Please evaluate until the deadline if there are any challenges with setting up.
Planned release is in November.
Updated by geor about 4 years ago
- Subject changed from setup openSUSE Jump 15.2 openqa to [sprint] setup openSUSE Jump 15.2 openqa
Updated by punkioudi about 4 years ago
Hi @msmeissn,
Right now there are these runs :
So what more is it expected from the openqa tests in relation to these repos? Is it sufficient for the system to be up-to-date, by using the aforementioned repositories and then do the testing?
thanks!
Updated by msmeissn about 4 years ago
These are the groups for product development before release.
What we also need is the openqa setup for the maintenance phase.
e.g. for Leap 15.2: https://openqa.opensuse.org/group_overview/69 and https://openqa.opensuse.org/group_overview/70
Updated by punkioudi about 4 years ago
Thank you @msmeissn for the answers!
After some research though I did today, an implementation of the openQA bot is also required which is handled by @osukup (who has time-off this week). Consequently, this ticket can be handled on the beginning of next week.
@tjyrinki_suse , it is an urgent+sprint item, but I have to postpone it for next week, will it be ok? :)
Updated by tjyrinki_suse about 4 years ago
Sure ok @punkioudi, it's urgent but we need Ondrej indeed.
Updated by szarate about 4 years ago
So what I get from asking around:
- Iso has to be synced (https://github.com/os-autoinst/openqa-trigger-from-obs/pull/89), but rather we need the fixed asset (that would be living in
/var/lib/openqa/share/factory/hdd/fixed
- Medium type setup: (flavor=DVD-Incidents) in o3 for jump (Leap 15.2 mediums can be used as a template)
- It's not clear to me, But likely there's a go: Jump has to be added here: https://github.com/openSUSE/openSUSE-release-tools/pull/2383
Updated by szarate about 4 years ago
- Due date changed from 2020-10-30 to 2021-02-01
- Priority changed from Urgent to Low
So after double checking with Marcus, this ticket is not urgent anymore as the Jump 15.2.1 received a no-go.
I'd like to announce that we have a final no-Go for Leap 15.2.1 after talking to stakeholders or engineers working on blocking issues.
https://lists.opensuse.org/opensuse-project/2020-11/msg00001.html
Updated by msmeissn about 4 years ago
- Subject changed from [sprint] setup openSUSE Jump 15.2 openqa to [sprint] setup openSUSE Jump 15.3 openqa
change to be the 15.3 setup ticket.
Updated by szarate about 4 years ago
- Related to action #25032: enable openQA maintenance testing added
Updated by szarate about 4 years ago
- Related to action #39449: enable openQA maintenance testing added
Updated by tjyrinki_suse about 4 years ago
- Subject changed from setup openSUSE Jump 15.3 openqa to [qe-core] setup openSUSE Jump 15.3 openqa
Updated by tjyrinki_suse about 4 years ago
- Project changed from 119 to openQA Tests (public)
- Priority changed from Low to Normal
Updated by msmeissn almost 4 years ago
- Subject changed from [qe-core] setup openSUSE Jump 15.3 openqa to [qe-core] setup Leap Jump 15.3 openqa
Name is openSUSE Leap 15.3.
Updated by msmeissn almost 4 years ago
- Subject changed from [qe-core] setup Leap Jump 15.3 openqa to [qe-core] setup openSUSE Leap 15.3 openqa
we need openqa setup, and it will be a bit different.
we can have incidents now for 2 different locations
openSUSE Backports SLE-15-SP3
test incident: openSUSE:Maintenance:15678openSUSE Leap 15.3
test incident: openSUSE:Maintenance:15681
both targets will go into Leap 15.3.
Updated by tjyrinki_suse almost 4 years ago
- Priority changed from Normal to Urgent
This needs bumping in priority and finding someone who can configure O3 accordingly.
Updated by szarate almost 4 years ago
- Precedes action #76702: [qe-core][u] disable iso download in openQA added
Updated by osukup almost 4 years ago
Updated by szarate almost 4 years ago
- Description updated (diff)
Santiago to clarify if it's jump or leap
Updated by msmeissn almost 4 years ago
it is called leap now. jump was only for 15.2.1, but 15.3 will be called leap, also in the directory paths.
Updated by tjyrinki_suse almost 4 years ago
- Due date deleted (
2021-02-01) - Target version set to Ready
Updated by tjyrinki_suse almost 4 years ago
- Target version changed from Ready to QE-Core: Ready
Updated by tjyrinki_suse almost 4 years ago
- Target version changed from QE-Core: Ready to Ready
Updated by szarate almost 4 years ago
- Target version changed from Ready to QE-Core: Ready
Updated by tjyrinki_suse almost 4 years ago
- Description updated (diff)
- Target version changed from QE-Core: Ready to future
Updated by szarate over 3 years ago
- Assignee set to szarate
- Target version changed from future to QE-Core: Ready
I'm picking this up while in training.
For now: https://github.com/openSUSE/openSUSE-release-tools/pull/2537 has been created based on the information that we have within the ticket.
Updated by tjyrinki_suse over 3 years ago
- Status changed from Workable to In Progress
Updated by szarate over 3 years ago
- How to test this?
- Who consumes/what the json files modified in the PR
- How do we check that the results are correct? (i.E that maintenance updates will be picked up)
- What does the syncing of the jobgroups between Maintenance and OBS once the leap product is released
Updated by msmeissn over 3 years ago
someone needs to setup 15.3 job groups in openqa.opensuse.org. (antonios?)
There will be code changes needed in opensuse-release-tools and os-autoinst-distri-opensuse, as we now have 2 different looking incidents.
what is not handled yet, are incidents like this:
openSUSE:Maintenance:15953 hello
As we now have 2 sources for updates, openSUSE:Leap:15.3:Update AND openSUSE:Backports:SLE-15-SP3:Update to test for Leap 15.3.
openSUSE:Maintenance:15681 is the other possible case, with rpmlint-mini
BOTH cases must be handled.
Updated by apappas over 3 years ago
The code changes in the repos have been done, and the mediums and JobGrous have been set. Temporarily the beta iso is being used.
I expected that they woulkd have been triggered. I'll investigate why they haven't.
Updated by apappas over 3 years ago
The runs are now posted for "regular" updates but the bot continues to ignore the backports. I just asked the UM if it's possible to send a dummy update to the sle repo so that can be tested.
Updated by msmeissn over 3 years ago
openSUSE:Maintenance:15953 - recommended (low): hello
locked: 4
release: 879946, review, 39 days ago
maintenance_release: openSUSE:Maintenance:15953/hello.openSUSE_Backports_SLE-15-SP3_Update -> openSUSE:Backports:SLE-15-SP3:Update/hello.15953
maintenance_release: openSUSE:Maintenance:15953/patchinfo -> openSUSE:Backports:SLE-15-SP3:Update/patchinfo.15953
It was missing a qam-openqa review, I added one just now.
Updated by apappas over 3 years ago
- Related to action #91827: [qe-core] Write tests and documentation for oqamaint bot added
Updated by szarate over 3 years ago
- Assignee changed from szarate to apappas
I think for now, Anton has done more than I have :)
Updated by okurz over 3 years ago
- Subject changed from [qe-core] setup openSUSE Leap 15.3 openqa to [qe-core] setup openSUSE Leap 15.3 openQA
Had a meeting with hrommel, vpelcak, msmeissn, mawerner about two aspects
- "repository setup in openQA" which is mainly covered in this ticket. Updates to openSUSE Leap 15.3 can come from 3 different projects:
- SLE 15 SP3 shared packages
- PackageHub 'Backports'
- packages specific to openSUSE Leap 15.3
It is expected that QE-Core takes care of that. In best case this can be fixed with just configuration changes for the openSUSE Maintenance openQA bot, in worst case this might need more changes https://github.com/openSUSE/openSUSE-release-tools/blob/master/openqa-maintenance.py . msmeissn has reminded of the release date of Leap 15.3 in just a couple of days until this issue has been fixed
- To avoid iterations after release of SLE 15 SP3 updates caused by failures that show only on openSUSE Leap 15.3, we should incorporate the review of openQA results of openSUSE Leap 15.3 into the release decision of SLE 15 SP3 updates. For this I suggested to extend the existing SLE maintenance tests on openqa.suse.de with according Leap test scenarios. Of course, a good start is to just add one scenario in either aggregate or incident tests and slowly extend from there. @hrommel1 as discussed plans to create a specific ticket for that. Responsibility area for the initial setup is QE-Core as well.
Updated by apappas over 3 years ago
The configuration for the packages specific to Leap 15.3 has been done and has been going on for some time. The problem is doing the rest of them. Complicating that, is, that to our knowledge is qe-core, the only documentation we have is the code, which means we just can't copy paste the config like we did before.
Updated by tjyrinki_suse over 3 years ago
okurz wrote:
It is expected that QE-Core takes care of that. In best case this can be fixed with just configuration changes for the openSUSE Maintenance openQA bot, in worst case this might need more changes https://github.com/openSUSE/openSUSE-release-tools/blob/master/openqa-maintenance.py . msmeissn has reminded of the release date of Leap 15.3 in just a couple of days until this issue has been fixed
Ok.
- To avoid iterations after release of SLE 15 SP3 updates caused by failures that show only on openSUSE Leap 15.3, we should incorporate the review of openQA results of openSUSE Leap 15.3 into the release decision of SLE 15 SP3 updates. For this I suggested to extend the existing SLE maintenance tests on openqa.suse.de with according Leap test scenarios. Of course, a good start is to just add one scenario in either aggregate or incident tests and slowly extend from there. @hrommel1 as discussed plans to create a specific ticket for that. Responsibility area for the initial setup is QE-Core as well.
If we don't look at O3 results (I think it would be possible to watch a subset of those results for the decision) but want to instead add Leap tests to OSD, then yes makes sense. Respective squads would be adding their tests, butfirst any media/product adding of Leap to OSD would be on QE Core.
Updated by okurz over 3 years ago
tjyrinki_suse wrote:
- To avoid iterations after release of SLE 15 SP3 updates caused by failures that show only on openSUSE Leap 15.3, we should incorporate the review of openQA results of openSUSE Leap 15.3 into the release decision of SLE 15 SP3 updates. For this I suggested to extend the existing SLE maintenance tests on openqa.suse.de with according Leap test scenarios. Of course, a good start is to just add one scenario in either aggregate or incident tests and slowly extend from there. @hrommel1 as discussed plans to create a specific ticket for that. Responsibility area for the initial setup is QE-Core as well.
If we don't look at O3 results (I think it would be possible to watch a subset of those results for the decision) but want to instead add Leap tests to OSD, then yes makes sense. Respective squads would be adding their tests, butfirst any media/product adding of Leap to OSD would be on QE Core.
The main and only reason to use openqa.suse.de is because the unreleased SLE maintenance updates are not available on openqa.opensuse.org. This should not be the default for new tests that run on Leap as Leap is still a community project which should be developed and tested in the open where possible.
Updated by apappas over 3 years ago
- Status changed from In Progress to Feedback
The Backports part of the ticket has been completeted
What remains are the Leap SLE updates. According to Marcus, there is no Build Service project that publishes these updates, but there is a script that collects SLE15GA->SLE15SP# into the Leap SLE repo. He suggests we test by adding the SLE15SP3 testrepos to a Leap 15.3 installation.
I personally need feedback as to what the test scenarios for those updates will be.
Updated by tjyrinki_suse over 3 years ago
- Parent task set to #92563
For SLE updates on Leap, I'm suggesting new job groups in OSD that test the same things as these job groups on O3, but by using the prospective SLE incident updates:
- openSUSE Leap 15.3 Updates => Maintenance: Test Repo -> Maintenance: Leap 15.3 Updates
- openSUSE Leap 15.3 Incidents => Maintenance: Single Incidents -> Maintenance: Leap 15.3 Incidents
I'd suggest starting with cryptlvm/cryptlvm@uefi-2G, gnome/gnome@uefi, textmode@64bit and install_with_updates_gnome - and if those go fine, add KDE as well. x86_64 only! Requires creating also Leap products etc on OSD.
Heiko agreed this sounds like a good initial plan.
Updated by okurz over 3 years ago
how about starting with a single scenario as proof of concept? Especially as there would be no help from any openSUSE community to review and as people are already suffering from the review workload the additional tests should be really limited to only show that the SLE update side of things is not completely broken while keeping the main test load still on o3, right?
Updated by tjyrinki_suse over 3 years ago
- Status changed from Feedback to In Progress
I think the above should be enough to get forward with the OSD SLE-on-Leap updates.
Updated by maritawerner over 3 years ago
Today there was a meeting with Marcus Meissner, Antonios, Santi, Anna and myself with this topics:
- Anton has finished the set-up of the channels for 15.3, all good
- For 15.4 Lubos will create a progress ticket at Beta1 to inform QE that openQa needs to be set-up for SP4
- The review of the Leap maintenance job groups is still under discussion, Santi will follow-up with an email, the idea is that the QE Core team will take over the review -> more info will come
- Anna & QE Core team will maintain the tests for Leap and increase the test coverage step by step
Updated by okurz over 3 years ago
waiting for apappas and osukup regarding scheduling tests within OSD. Would it better if we can not wait for 1:1 talks but better coordinate on team level. Maybe I can offer some help?
Updated by okurz over 3 years ago
There was no update and we are in the dark here. We should have apappas provide the actual requirements.
Updated by apappas over 3 years ago
- Status changed from In Progress to Blocked
Updated by okurz over 3 years ago
- Status changed from Blocked to Feedback
Please stick to https://progress.opensuse.org/projects/openqav3/wiki/Wiki#ticket-workflow and use "Blocked" only when there is some external dependency (e.g. bugs, not implemented features). There should be a link to another ticket, bug, trello card, etc. where it can be seen what the ticket is blocked by. You can use "Feedback" though: "Further work on the ticket is blocked by open points or is awaiting for the feedback to proceed. Sometimes also used to ask Assignee about progress on inactivity."
Updated by maritawerner over 3 years ago
Thanks for the update, Oli. But could you or somebody else please specify the "help" that Anton is waiting for? Is it the tools team, is it Ondrey Sukup or who else? On Wednesday in the call I had the impression that we are waiting for Anton. He has said that he can not solve the problem so we are going in cycles here.
Updated by apappas over 3 years ago
I am waiting for Ondrej or people that have knowledge of the openqa maintenance bot to help me edit the ΜR(1) so that it can post the jobs.
Updated by okurz over 3 years ago
I commented in the MR: https://gitlab.suse.de/qa-maintenance/metadata/-/merge_requests/503/diffs#note_335709
Updated by szarate over 3 years ago
apappas wrote:
I am waiting for Ondrej or people that have knowledge of the openqa maintenance bot to help me edit the ΜR(1) so that it can post the jobs.
Ondrej can you help to move this forward?
Updated by okurz over 3 years ago
We discussed this in the weekly SUSE QE Sync 2021-08-04. Responsibility for qa-maintenance/openQABot itself is for sure within the scope of SUSE QE Tools. But enabling "openSUSE repo tests as part of SLE updates" is certainly a new domain and not something we have experience with. Just don't expect too much :)
Updated by apappas over 3 years ago
I am not aware of the current limits of the bot, but I want to make it clear that what is needed is, to take incidents from the SUSE:Updates:openSUSE-SLE:15.3
internal build service project and post the corresponding jobs for an already set up OSD JobGroup. AFAICT this is already what it does for the SLE projects.
Updated by osukup over 3 years ago
apappas wrote:
I am not aware of the current limits of the bot, but I want to make it clear that what is needed is, to take incidents from the
SUSE:Updates:openSUSE-SLE:15.3
internal build service project and post the corresponding jobs for an already set up OSD JobGroup. AFAICT this is already what it does for the SLE projects.
OK, any example of incident with populated openSUSE-SLE:15.3 ?
- we now doing switch to shedule based on qem-dashboard with different format of config files .. can be found in qa-maintenance/metadata/bot-ng
Updated by msmeissn over 3 years ago
SUSE:Maintenance:20230:245273
e.g. is a dracut update with openSUSE-SLE-15.3 in it.
https://smelt.suse.de/overview/ -> Testing -> search for openSUSE in the products.
Updated by osukup over 3 years ago
msmeissn wrote:
SUSE:Maintenance:20230:245273
e.g. is a dracut update with openSUSE-SLE-15.3 in it.https://smelt.suse.de/overview/ -> Testing -> search for openSUSE in the products.
thx
openSUSE-SLE has a different repo structure in IBS so it needs add few workaround into bot:(
Updated by osukup over 3 years ago
jobs are scheduled in osd, but there is a missing Leap 15.3 ISO and the second question is if Leap needles are deployed for OSD jobs
Updated by okurz over 3 years ago
Discussed in weekly QE sync 2021-09-01: This ticket was mentioned again by multiple stakeholders and should be worked on as soon as apappas is available again. @apappas if you need support from members of SUSE QE Tools please reach out to us. So far I see the prerequisites in place. Of course test scheduling details need to be ruled out.
Updated by apappas about 3 years ago
- Status changed from Feedback to In Progress
I have rescheduled the tests that were disabled, while I was away. It probably won't take much to make them fail less spectacularly. I will pull the teams in for tests where qe-core is not responsible, eg. cryptlvm
Updated by apappas about 3 years ago
- Status changed from In Progress to Feedback
IMHO the group is setup, the scenarios need some love by the various teams (needles etc.) and the schedule needs defining but I think that's a topic for another ticket. I am asking for feedback to resolve this ticket and make new ones.
Updated by okurz about 3 years ago
@apappas seems like you enabled broken Leap tests in production job groups so we have scenarios that "never worked" affecting releases of updates. Please do not do something like this. For example there is https://openqa.suse.de/tests/7142084#next_previous
Can you revert this ASAP and/or move tests to the "development" job group to first stabilize before moving into the production job groups?
EDIT: Wait, they are in the development job group like https://openqa.suse.de/tests/overview?distri=opensuse&version=15.3&build=%3A21237%3Asamba&groupid=393 . I think I accused you wrongly, sorry.
Updated by okurz about 3 years ago
apappas wrote:
IMHO the group is setup, the scenarios need some love by the various teams (needles etc.) and the schedule needs defining but I think that's a topic for another ticket. I am asking for feedback to resolve this ticket and make new ones.
We discussed this in weekly QE sync 2021-09-22 and clarified: @apappas yes, please create new tickets (first!) assigned to the according squads as further subtasks of #92563 and then (afterwards!) resolve this ticket.
Updated by apappas about 3 years ago
- Category deleted (
New test) - Status changed from Feedback to Resolved
- Assignee deleted (
apappas) - Priority changed from Urgent to Low
Updated by szarate about 3 years ago
Email mentioned at https://progress.opensuse.org/issues/72202#note-59 has been finally sent: https://mailman.suse.de/mailman/private/qa-team/2021-September/005466.html TL;DR: go here and check: https://confluence.suse.com/display/qasle/Tests+Maintained+by+QE+Core#TestsMaintainedbyQECore-openqa.opensuse.org
Updated by pvorel about 3 years ago
I wonder if it's desired to have also kernel testing for Leap 15.3. Just out of curiosity, because kernel QE team is that busy, that even Tumbleweed is not reviewed and tagged regularly (I try to watch it, but SLE and other projects are the priority).
Updated by okurz about 3 years ago
pvorel wrote:
I wonder if it's desired to have also kernel testing for Leap 15.3. Just out of curiosity, because kernel QE team is that busy, that even Tumbleweed is not reviewed and tagged regularly (I try to watch it, but SLE and other projects are the priority).
I consider the risk of breaking Leap with SLE kernel updates low considering that with Leap 15.3 after the "Jump" project the very same SLE binaries are used in Leap. Implicitly of course the kernel is tested against Leap with the existing Leap (for SLE maintenance updates) tests. As you said you are too busy so better focus on Tumbleweed tests with the more recent kernel versions before considering extending Leap kernel specific tests.
Updated by msmeissn about 3 years ago
the kernel opensuse leap 15.3 uses is tested by the SLE kernel testing... so I think it does not need it seperately for openSUSE at this time.
Updated by hrommel1 over 2 years ago
- Status changed from Resolved to In Progress
- Priority changed from Low to Normal
I have reopened the ticket since the openQA job group I was pointed to for results (https://openqa.suse.de/group_overview/393) is empty.
Can somebody share a bit of insight where to look for test results of unreleased SLE updates in the context of openSUSE 15.3 testing (on OSD) ?
Updated by zluo over 2 years ago
- Status changed from In Progress to Workable
ticket 'In progress' without Assigne is not good. Set it as workable.
Updated by szarate over 2 years ago
- Due date set to 2022-04-08
- Status changed from Workable to Resolved
- Assignee set to szarate
I'm setting to feedback atm, setting due date to get a reminder to find the info... last I recall, there was some issue with the maintenance bot
Updated by szarate over 2 years ago
Is the status of this ticket current? any updates? can we resolve the ticket?
Updated by msmeissn over 2 years ago
So the opensuse.openqa.org side this was done and is running in production.
For the IBS side we talked about it, but without a link to the openqa group where it happens I do not consider it done.
I looked into "Core Maintenance" runs, it seems not to be in.
Updated by szarate over 2 years ago
- Related to coordination #108797: [qe-core] Leap setup 15.4 openqa for updates added
Updated by szarate over 2 years ago
- Status changed from Feedback to Resolved
hrommel1 wrote:
I have reopened the ticket since the openQA job group I was pointed to for results (https://openqa.suse.de/group_overview/393) is empty.
Can somebody share a bit of insight where to look for test results of unreleased SLE updates in the context of openSUSE 15.3 testing (on OSD) ?
Info from a meeting on May 20th: Ondrej had unscheduled the tests because they were failing some months ago and he had informed the (then) PO, there was no follow up and the team was not informed
The tests are running now: https://openqa.suse.de/group_overview/393