QA » openQA Project » openQA Tests

New round of re-needling here : https://gitlab.suse.de/openqa/os-autoinst-needles-sles/-/merge_requests/1514

For future references, we need to avoid these false negatives: https://openqa.suse.de/tests/5642237#step/select_patterns/13

Here is another case:

• now you see me: https://openqa.suse.de/tests/5645754#step/select_patterns/76 (X Window System unselected: good)
• now you don't: https://openqa.suse.de/tests/5645754#step/select_patterns/80 (X Window System selected back: bad)
• So, we get IceWM instead of textmode: https://openqa.suse.de/tests/5645754#step/first_boot/3

And assert_screen times out

Hm, the tricky thing here is how to overcome these false negatives, especially on this test that for different cases we have different needles etc etc. Probably we could consider opening a ticket for re-writing the module or something ^{^}

punkioudi wrote:

Hm, the tricky thing here is how to overcome these false negatives, especially on this test that for different cases we have different needles etc etc. Probably we could consider opening a ticket for re-writing the module or something ^{^}

I expected this ticket to be that ticket for fixing (re-writing) sub select_specific_patterns_by_iteration.

You will love this one :)

Command not found: https://openqa.suse.de/tests/5651879#step/validate_pcm_aws/2
Because the qcow2 image doesn't contain AWS pattern: https://openqa.suse.de/tests/5651878#step/select_patterns/11

https://github.com/os-autoinst/os-autoinst/pull/1633 should make needling more reliable - but I can't say if this will be accepted and when it can be deployed.

We would appreciate more testing before throwing that onto production so for example QE-Core and QE-Y can try out some specific test scenarios and at best already provide new necessary needles before we accept that change

Reported https://bugzilla.suse.com/show_bug.cgi?id=1183519 about the bad contrast specific to recent SLE branding changes.

I think select_patterns issue impacts also create_hdd_minimal_base+sdk. Job didn't fail, but for some reason many patterns were selected https://openqa.suse.de/tests/5670487#step/installation_overview/2.

pcervinka wrote:

I think select_patterns issue impacts also create_hdd_minimal_base+sdk. Job didn't fail, but for some reason many patterns were selected https://openqa.suse.de/tests/5670487#step/installation_overview/2.

Yes, sadly.

FYI: I would be surprised if there is any job that doesn't have problems with select_patterns (false negatives included)

SLindoMansilla wrote:

FYI: I would be surprised if there is any job that doesn't have problems with select_patterns (false negatives included)

Luckily staging jobs do work: https://openqa.suse.de/tests/5659239 But this is coincidence, as we somehow do not select gnome patterns and still boot into text mode.
I've made Release Managers aware of this issue when it was detected there, so they are aware that test doesn't do what it's supposed to.

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: create_hdd_minimal_base+sdk_withhome
https://openqa.suse.de/tests/5702807

To prevent further reminder comments one of the following options should be followed:

1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
2. The openQA job group is moved to "Released"
3. The label in the openQA scenario is removed

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: extra_tests_kernel
https://openqa.suse.de/tests/7261529

To prevent further reminder comments one of the following options should be followed:

1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

This ticket was set to Normal priority but was not updated within the SLO period. Please consider picking up this ticket or just set the ticket to the next lower priority.

This ticket was set to Normal priority but was not updated within the SLO period. Please consider picking up this ticket or just set the ticket to the next lower priority.