action #122296
openFix openqa-trigger-bisect-jobs to actually remove incidents from bisection tests again
0%
Description
Motivation¶
Originally brought up in #97118#note-10 by okurz and mgrifalconi.
https://github.com/os-autoinst/scripts/blob/master/openqa-trigger-bisect-jobs originally removed individual incidents from a list to conduct tests excluding specific incidents to be able to bisect with/without individual incidents. Seems like something was changed in the test architecture to install all updates in the image generation jobs meaning that the repos and packages from the repos are not removed by just changing the test variable content within bisection jobs. The test code needs to be changed to account for that. Maybe as simple as removing the according zypper repo and calling zypper dup or zypper rm or something.
Impact¶
All results of "openqa-trigger-bisect-jobs" are invalid as the mentioned incidents are not actually excluded. Without correct results also "openqa-trigger-bisect-jobs" can not be used for any automatic decisions which would help in the daily work of openQA reviewers concerning SLE maintenance update testing.
Acceptance criteria¶
- AC1: openQA jobs triggered by https://github.com/os-autoinst/scripts/blob/master/openqa-trigger-bisect-jobs ensure that the updates from incidents that are not to be included are not present in the SUT
Suggestions¶
- Take a look at any jobs from https://openqa.suse.de/tests?match=bisect. E.g. https://openqa.suse.de/tests/10217887 is a job "sle-15-SP4-Server-DVD-Updates-aarch64-Build20221220-20221221-qam-minimal+base:investigate:bisect_without_27235@aarch64-virtio" so judging from the name of the job it should not include incident "27235". https://openqa.suse.de/tests/10217887#settings shows that while 27235 is mentioned in the variable "BASE_TEST_ISSUES" it is not included in the test variable "BASE_TEST_REPOS". But https://openqa.suse.de/tests/10217887/logfile?filename=serial_terminal.txt mentions a zypper repo with incident 27235
Updated by okurz almost 2 years ago
- Related to action #97118: enhance bot automatic approval: check multiple days added
Updated by slo-gin almost 2 years ago
This ticket was set to High priority but was not updated within the SLO period. Please consider picking up this ticket or just set the ticket to the next lower priority.
Updated by okurz almost 2 years ago
- Blocks action #124379: [qe-core] geekotest bisect is broken for child test runs added
Updated by okurz almost 2 years ago
- Blocks action #124370: [qe-core] geekotest bisect is not removing the issue from all test repositories added
Updated by szarate almost 2 years ago
- Project changed from openQA Tests (public) to openQA Project (public)
- Subject changed from [qe-core] Fix openqa-trigger-bisect-jobs to actually remove incidents from bisection tests again to Fix openqa-trigger-bisect-jobs to actually remove incidents from bisection tests again
- Category changed from Bugs in existing tests to Regressions/Crashes
- Assignee set to okurz
QE-Core didn´t implement the feature, while we might be one of the consumers, atm to the best of my knowledge we don't have know-how on this, and since it's an important thing for reviewers, tools team is better equipped to fix the problem. We can take over when working on: https://progress.opensuse.org/issues/106930#note-21
Updated by szarate almost 2 years ago
- Category deleted (
Regressions/Crashes)
See my comment on #124379, If you plan to reassign the ticket again, better reject it, and bring the topic to the Wednesday Sync meeting.
Updated by okurz almost 2 years ago
- Tags set to scripts, tests, bisect, maintenance
- Category set to Regressions/Crashes
- Assignee deleted (
okurz) - Priority changed from High to Normal
- Target version set to future
szarate wrote:
See my comment on #124379, If you plan to reassign the ticket again, better reject it, and bring the topic to the Wednesday Sync meeting.
uh, I don't see a comment by you on #124379. Which one do you mean?
The reason why I do not consider this ticket for the tools team is that the approach of openqa-trigger-bisect-jobs worked just fine until os-autoinst-distri-opensuse together with schedules have been restructured. I doubt it's an efficient approach if I ask people within the SUSE QE Tools team to follow changes done by QE Core and integrate fixes without introducing regressions. Also the changes would need to be done within os-autoinst-distri-opensuse.
Updated by szarate almost 2 years ago
- Tags deleted (
scripts, tests, bisect, maintenance) - Assignee set to okurz
- Priority changed from Normal to High
- Target version deleted (
future)
okurz wrote:
szarate wrote:
See my comment on #124379, If you plan to reassign the ticket again, better reject it, and bring the topic to the Wednesday Sync meeting.
uh, I don't see a comment by you on #124379. Which one do you mean?
My bad, copy pasted in the wrong place :) I meant the comment on #122296#note-6
Updated by szarate almost 2 years ago
- Tags set to scripts, tests, bisect, maintenance
- Priority changed from High to Normal
- Target version set to future
Sometimes, I don't love redmine.
Updated by okurz over 1 year ago
- Related to action #131204: openqa-trigger-bisect-jobs is called on passed jobs size:M added
Updated by szarate 11 months ago
- Blocks deleted (action #124379: [qe-core] geekotest bisect is broken for child test runs)