action #99072
closed[qe-core] Make Leap 15.3 on osd green
Added by apappas over 2 years ago. Updated 8 months ago.
0%
Description
Motivation¶
Leap 15.3 was added to osd, but no run has ever finished green. At time of creation the tests were stuck at the installer, however after qe-yast makes it work, we will have to intervene, with the ultimate goal of moving them away from the parent group.
Acceptance criteria¶
- AC1: A run is fully green
- AC2: Move the tests to production groups
Suggestions¶
- Create a YAML schedule
- Add all tests from o3 that are applicable
Updated by apappas over 2 years ago
- Status changed from New to Feedback
- Assignee set to apappas
The tests are now green.
As mentioned in Slack, the only thing remaining is to figure out who is doing the reviewing and define it's scope.
Updated by okurz over 2 years ago
Topic was discussed in weekly QE sync 2021-09-29. Current state of tests with the existing two scenarios looks promising. IMHO no more scenarios should be added right now but only after people grew to really trust these tests over the next couple of months.
Looking at https://openqa.suse.de/group_overview/393 I saw not all tests green. There had been one occurence of https://progress.opensuse.org/issues/77728 which I reopened and retriggered the incomplete test. Such problems will be automatically handled as soon as the tests are not in the "development" job group anymore.
Problems I saw currently in the tests that look like should be looked into by QE-Core:
- https://openqa.suse.de/tests/7242423#step/updates_packagekit_gpk/2
- https://openqa.suse.de/tests/7224318#step/updates_packagekit_gpk/2
- https://openqa.suse.de/tests/7224319#step/shutdown/2
- https://openqa.suse.de/tests/7257861#step/chromium/20
I suggest to diligently investigate and fix these problems before putting the tests into production where such failures would prevent auto-approval of maintenance updates.
Updated by apappas over 2 years ago
- Status changed from Feedback to Resolved
The chromium failure was caused by openqa failing to make type chrome://version and typing only chromium.
The failures have stopped happening during the last week, and they seem more like stall related to openqa load.
Furthermore these failures are happening on mature tests for a released product and so I believe the should be treated like failures in production. Not failures in development.
As a result I moved the tests to production and I am resolving the ticket.
Updated by okurz over 2 years ago
- Status changed from Resolved to Feedback
hi, great to see an improvement in stability. Just to be sure we are not introducing additional load on openQA reviewers: What helped to reduce the openQA load so that tests would not fail anymore in shutdown, chromium or updates_packagekit_gpk ? I am not sure I understand your statement "I believe the should be treated like failures in production. Not failures in development.". Does that mean there are according tickets for that? If yes, please reference them.
Updated by szarate over 2 years ago
- Status changed from Feedback to Resolved
I believe that the failure anton is mentioning is https://progress.opensuse.org/issues/43889 #43889. This ticket is done.
Updated by okurz over 1 year ago
- Status changed from Resolved to Feedback
- Priority changed from Urgent to High
Seems like AC2 was never fulfilled. a quick SQL query on OSD reveals:
select id,group_id from jobs where flavor = 'Leap-DVD-Incidents' and group_id != 393 and group_id != 432 limit 10;
id | group_id
----+----------
(0 rows)
so 0 jobs for Leap maintenance incidents excluding the two Leap 15.3 and Leap 15.4 development job groups which are ignored. So tests were added and stabilized but never moved out of Development job groups. Jobs within the development job group are ignored by qem-bot so tests should move out of development job groups after stabilization.
Updated by szarate over 1 year ago
We need to make sure that we switch to using autoyast the same we have already for leap in o3, and slowly add the tests here too
Updated by slo-gin over 1 year ago
This ticket was set to High priority but was not updated within the SLO period. Please consider picking up this ticket or just set the ticket to the next lower priority.
Updated by szarate 8 months ago
- Status changed from Feedback to Rejected
okurz wrote in #note-14:
Then please ensure that there are according tasks for 15.4+15.5+15.6. You know that this is important as Marcus Meissner brought it up frustratedly the past days.
Yeah, no need to reopen the ticket though.