QA » openQA Project » openQA Tests

We have serial detection now, so it would be one line change, so what's about moving it to Milestone 20/21

Yes, I would like to see this in action but QSF-u needs to work on other priorities first and the re-planning effort recently is high so I would like to keep it in future. Of course, in the idealistic case that we would be done with all tasks in M20/21 sooner we can still pick it up :)

+1 as discussed in the sprint planning meeting today

Well, the PR you mentioned just adds the feature to be able to add detections but we have to add the actual check patterns. Please see the example https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/4704 mentioned in the description as well.

This PR:
https://github.com/os-autoinst/os-autoinst/pull/977
contains the documentation how to use the serial detection feature:
$testapi::distri->set_expected_serial_failures(soft=>{"AWESOME SOFT MSG 1"=>[qr/gcc version/], "AWESOME SOFT MSG 2"=>[qr/insmod error: 1/]}, hard=>{"AWESOME HARD MSG"=>[qr/No iBFT detected/]});

We should add an extra module for checking the serial log for all known failures

We have https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/caasp/journal_check.pm
to check for serial failures, so I will now try to generalize that code and execute it in the post_fail_hook and in best case before the ending of every test_suite

Following elements should be considered:

• A hash with bugreferences and a pattern to recognize them
• A function that checks the journal and records the bugs, called in the general post_fail_hook
• set_expected_failures to recognize stopper bugs on worker level

jorauch wrote:

We have https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/caasp/journal_check.pm
to check for serial failures

don't confuse "checking the journal from within the SUT" to "check for errors reported on the serial terminal checked by the worker", two different things even though the journal and the serial terminal might include the same or related messages.

Agreed with okurz to:

• put the creation of the 'bug table' in a separate file
• keep the detection on worker
• Check in boot_to_desktop for a welcome message to verify how the detection works

The latest implementation of the needed feature:
https://github.com/os-autoinst/os-autoinst/pull/998

Created WIP PR:
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6398

Finally working:
http://pinky.arch.suse.de/tests/1685#step/boot_to_desktop/4

PR merged

broke a lot of tests, fix applied, retriggered like 1k jobs, should be fine now.

Since we have no more outtages by this we should be able to close the ticket?

Sure, if both ACs are covered and we have a verification run on production

We can wait until we see this by luck or we have some really new and cool bug that goes through enough builds so we can use the new mechanics to detect it in production.
But imho that's a waste of time because that would be completely random and we cannot even estimate when it will happen.
I think a normal verification run must be sufficient here

I agree with Jojo here... However, there's always the trick of turning off the host's thread or cpu while the machine is running? if such thing is possible :P