coordination #39719: [saga][epic] Detection of "known failures" for stable tests, easy test results review and easy tracking of known issues
tests fail with auto_review:"(?s)version is 4\.6\.1606298538\.191b5988.*Can.*t locate object method.*code.*via package":retry
the latest merged PR for #80264 caused a regression as seen on o3: https://openqa.opensuse.org/tests/1484502#step/yast2_nfs_server/60 shows
# Test died: Can't locate object method "code" via package "Mojo::Transaction::HTTP" at /usr/lib/os-autoinst/lockapi.pm line 110.
I rolled back the deployment on o3:
for i in aarch64 openqaworker1 openqaworker4 openqaworker7 power8 imagetester rebel; do echo $i && ssh root@$i "transactional-update rollback last && reboot"; done
Also I checked
snapper diff 797..798
to see what were the real changes in before. This shows for example the diff in os-autoinst version:
4.6.1605852905.39d54910..4.6.1606298538.191b5988 so we can list
$ git log1 --no-merges 39d54910..191b5988
49fd0a4e mmapi: Enable error logging in all `get_` functions 61cc3516 Add test for mmapi 3655e0c1 Fix "Use of uninitialized value in numeric eq" on connection errors in mmapi
which were the expected changes.
- Status changed from In Progress to Resolved
Using the above version-specific regex I could identify relevant failed jobs and handle them automatically:
$ ./openqa-monitor-investigation-candidates https://openqa.opensuse.org/tests/1484464 textmode https://openqa.opensuse.org/tests/1484497 RAID0_gpt https://openqa.opensuse.org/tests/1484526 textmode https://openqa.opensuse.org/tests/1484502 textmode https://openqa.opensuse.org/tests/1484527 kde https://openqa.opensuse.org/tests/1484883 lxde $ ./openqa-monitor-investigation-candidates | ./openqa-label-known-issues …