Actions
action #151310
closedcoordination #112862: [saga][epic] Future ideas for easy multi-machine handling: MM-tests as first-class citizens
coordination #111929: [epic] Stable multi-machine tests covering multiple physical workers
[regression] significant increase of parallel_failed+failed since 2023-11-21 size:M
Description
Motivation¶
As visible on https://monitor.qa.suse.de/d/nRDab3Jiz/openqa-jobs-test?orgId=1&from=1700508932604&to=1700724085546&viewPanel=24
since about 2023-11-21 there is again a significant increase of multi-machine tests which should be investigated, mitigated, fixed and prevented.
Acceptance criteria¶
- AC1: failed+parallel_failed on https://monitor.qa.suse.de/d/nRDab3Jiz/openqa-jobs-test?orgId=1&viewPanel=24 is significantly below 20% again
Suggestions¶
- Start to look into the issue early as waiting longer makes everything harder for us :)
- Lookup common failure sources and find out if it's actually not test or product regressions.
- Ask common stakeholders and/or test reviewers if they know something
- Review recent infrastructure changes which might be possibly related
- Mitigate, fix and prevent the issues you find
- Consider using the scientific method https://progress.opensuse.org/projects/openqav3/wiki/#Further-decision-steps-working-on-test-issues
- Use SQL queries to find out what failures are most common
- Consider using this opportunity to document one or two examples of how we commonly do that
Updated by livdywan about 1 year ago
- Subject changed from [regression] significant increase of parallel_failed+failed since 2023-11-21 to [regression] significant increase of parallel_failed+failed since 2023-11-21 size:M
- Description updated (diff)
- Status changed from New to Workable
Updated by okurz about 1 year ago
- Related to action #151612: [kernel][tools] test fails in suseconnect_scc - SUT times out trying to reach https://scc.suse.com added
Updated by okurz about 1 year ago
- Related to action #152389: significant increase in MM-test failure ratio 2023-12-11: test fails in multipath_iscsi and other multi-machine scenarios due to MTU size auto_review:"ping with packet size 1350 failed, problems with MTU" size:M added
Actions