Project

General

Profile

action #89314

[qe-core] test fails in select_patterns - sub select_specific_patterns_by_iteration needs to be fixed

Added by SLindoMansilla 7 months ago. Updated 17 days ago.

Status:
Blocked
Priority:
Normal
Assignee:
-
Category:
Bugs in existing tests
Target version:
-
Start date:
Due date:
% Done:

50%

Estimated time:
Difficulty:

Description

Observation

lib/y2_installbase.pm::select_specific_patterns_by_iteration needs to be fixed

Reproducible


Related issues

Related to openQA Tests - action #89632: [opensuse][qe-core] test fails in select_packages in package-dependency test suiteFeedback

History

#1 Updated by tjyrinki_suse 7 months ago

  • Priority changed from Normal to Urgent
  • Start date deleted (2021-03-02)

#2 Updated by punkioudi 7 months ago

  • Assignee set to punkioudi

#3 Updated by tjyrinki_suse 7 months ago

  • Status changed from New to Workable
  • Priority changed from Urgent to High
  • Target version set to Ready

#4 Updated by tjyrinki_suse 7 months ago

  • Target version changed from Ready to QE-Core: Ready

#5 Updated by punkioudi 6 months ago

  • % Done changed from 0 to 50

PR : https://gitlab.suse.de/openqa/os-autoinst-needles-sles/-/merge_requests/1512

For these cases, no need of modifying the select_specific_patterns_by_iteration function. It is a matter of re-needling.

#6 Updated by SLindoMansilla 6 months ago

  • Has duplicate action #89632: [opensuse][qe-core] test fails in select_packages in package-dependency test suite added

#7 Updated by SLindoMansilla 6 months ago

  • Has duplicate deleted (action #89632: [opensuse][qe-core] test fails in select_packages in package-dependency test suite)

#8 Updated by SLindoMansilla 6 months ago

  • Related to action #89632: [opensuse][qe-core] test fails in select_packages in package-dependency test suite added

#10 Updated by SLindoMansilla 6 months ago

For future references, we need to avoid these false negatives: https://openqa.suse.de/tests/5642237#step/select_patterns/13

#11 Updated by SLindoMansilla 6 months ago

Here is another case:

And assert_screen times out

#12 Updated by punkioudi 6 months ago

Hm, the tricky thing here is how to overcome these false negatives, especially on this test that for different cases we have different needles etc etc. Probably we could consider opening a ticket for re-writing the module or something ^

#13 Updated by SLindoMansilla 6 months ago

punkioudi wrote:

Hm, the tricky thing here is how to overcome these false negatives, especially on this test that for different cases we have different needles etc etc. Probably we could consider opening a ticket for re-writing the module or something ^

I expected this ticket to be that ticket for fixing (re-writing) sub select_specific_patterns_by_iteration.

#14 Updated by SLindoMansilla 6 months ago

You will love this one :)

Command not found: https://openqa.suse.de/tests/5651879#step/validate_pcm_aws/2
Because the qcow2 image doesn't contain AWS pattern: https://openqa.suse.de/tests/5651878#step/select_patterns/11

#15 Updated by punkioudi 6 months ago

  • Assignee deleted (punkioudi)

Unassigning myself, as there is already another team working on the topic ^

#16 Updated by coolo 6 months ago

https://github.com/os-autoinst/os-autoinst/pull/1633 should make needling more reliable - but I can't say if this will be accepted and when it can be deployed.

#17 Updated by okurz 6 months ago

We would appreciate more testing before throwing that onto production so for example QE-Core and QE-Y can try out some specific test scenarios and at best already provide new necessary needles before we accept that change

#18 Updated by okurz 6 months ago

Reported https://bugzilla.suse.com/show_bug.cgi?id=1183519 about the bad contrast specific to recent SLE branding changes.

#19 Updated by szarate 6 months ago

  • Status changed from Workable to Blocked
  • Target version deleted (QE-Core: Ready)

#20 Updated by pcervinka 6 months ago

I think select_patterns issue impacts also create_hdd_minimal_base+sdk. Job didn't fail, but for some reason many patterns were selected https://openqa.suse.de/tests/5670487#step/installation_overview/2.

#21 Updated by SLindoMansilla 6 months ago

pcervinka wrote:

I think select_patterns issue impacts also create_hdd_minimal_base+sdk. Job didn't fail, but for some reason many patterns were selected https://openqa.suse.de/tests/5670487#step/installation_overview/2.

Yes, sadly.

#22 Updated by SLindoMansilla 6 months ago

FYI: I would be surprised if there is any job that doesn't have problems with select_patterns (false negatives included)

#23 Updated by riafarov 6 months ago

SLindoMansilla wrote:

FYI: I would be surprised if there is any job that doesn't have problems with select_patterns (false negatives included)

Luckily staging jobs do work: https://openqa.suse.de/tests/5659239 But this is coincidence, as we somehow do not select gnome patterns and still boot into text mode.
I've made Release Managers aware of this issue when it was detected there, so they are aware that test doesn't do what it's supposed to.

#24 Updated by okurz 6 months ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: create_hdd_minimal_base+sdk_withhome
https://openqa.suse.de/tests/5702807

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released"
  3. The label in the openQA scenario is removed

#25 Updated by okurz 17 days ago

  • Priority changed from High to Normal

This ticket was set to "High" priority but was not updated within 120 days which is 4 times the period of the SLO for "High" tickets (30 days) as described on https://progress.opensuse.org/projects/openqatests/wiki/Wiki#SLOs-service-level-objectives . The ticket will be set to the next lower priority of "Normal".

Also available in: Atom PDF