action #38621
closedopenQA Project - coordination #39719: [saga][epic] Detection of "known failures" for stable tests, easy test results review and easy tracking of known issues
[functional][y] test fails in welcome - "Module is not signed with expected PKCS#7 message" (bsc#1093659) - Use serial exception catching feature from openQA to make sure the jobs reference the bug, e.g. as label
0%
Description
Observation¶
openQA test in scenario sle-12-SP4-Server-DVD-Staging:B-x86_64-ext4@uefi-staging fails in
welcome
on https://bugzilla.suse.com/show_bug.cgi?id=1093659
Acceptance criteria¶
- AC1: All jobs hitting bsc#1093659 will not need any work from manual reviewers
Suggestions¶
- Work on #36448 first
- Abort the test as soon as this soft-fail has been hit (or find a way how a job label can be created)
- Update documentation about feature detecting serial error
Further details¶
Always latest result in this scenario: latest
The "automatic label" in the acceptance criteria could be either (1) just soft-failing jobs showing up with the bug reference on http://s.qa.suse.de/test-status or (2) an automatic comment in the openQA webui. Most likely (1) would be easier but means we have to abort the whole test run as soon as we found the issue and recorded the soft failure to prevent any subsequent test module failing the module as "fail" overrides "softfail".
Updated by okurz almost 6 years ago
- Copied from action #36448: [functional][y] Fix serial exception catching feature from openQA to be able to use for job labels added
Updated by okurz almost 6 years ago
- Copied from deleted (action #36448: [functional][y] Fix serial exception catching feature from openQA to be able to use for job labels)
Updated by okurz almost 6 years ago
- Related to action #36448: [functional][y] Fix serial exception catching feature from openQA to be able to use for job labels added
Updated by okurz almost 6 years ago
- Related to deleted (action #36448: [functional][y] Fix serial exception catching feature from openQA to be able to use for job labels)
Updated by okurz almost 6 years ago
- Blocked by action #36448: [functional][y] Fix serial exception catching feature from openQA to be able to use for job labels added
Updated by riafarov over 5 years ago
- Status changed from Workable to In Progress
Updated by riafarov over 5 years ago
- Due date changed from 2018-07-31 to 2018-08-14
Updated by riafarov over 5 years ago
- Status changed from In Progress to Feedback
Updated by okurz over 5 years ago
- Related to action #39059: [sle][functional][y] detect "openSUSE sucks bug" about btrfs balance and record_soft_fail (was: yast2_gui tests modules as application could not start up) added
Updated by riafarov over 5 years ago
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/5573 - changes to the tests code
Updated by riafarov over 5 years ago
I guess it's about bsc#1093659 bsc#1093659 is verified fixed, so no further action is required.
Updated by riafarov over 5 years ago
- Due date changed from 2018-08-14 to 2018-08-28
Updated by riafarov over 5 years ago
Missing part: if test fails before (e.g. cause of assert_screen failure or wait_serial), then test won't have serial parsed, and hence carry over won't work. Checking for a solution to that.
Updated by okurz over 5 years ago
- Related to coordination #39719: [saga][epic] Detection of "known failures" for stable tests, easy test results review and easy tracking of known issues added
Updated by okurz over 5 years ago
- Related to deleted (coordination #39719: [saga][epic] Detection of "known failures" for stable tests, easy test results review and easy tracking of known issues)
Updated by riafarov over 5 years ago
- Status changed from Feedback to In Progress
Updated by riafarov over 5 years ago
https://github.com/os-autoinst/os-autoinst/pull/1014 changes in os-autoinst to detect serial failures even if test failed before
Updated by riafarov over 5 years ago
- Status changed from In Progress to Feedback
Not to rush with solution and get implementation which matches bigger vision, I will stop with changes in os-autoinst to make original idea work better, and then continue after having discussion with at least all stakeholders.
Updated by riafarov over 5 years ago
- Status changed from Feedback to Resolved
We've discussed with @szarate and he also has concerns about automated comment, so I will resolve this ticket, as carryover works now even test module failed before. On top bug is resolved.
We can continue with bigger vision in epic kindly created by @okurz: https://progress.opensuse.org/issues/39719