Project

General

Profile

action #89314

[qe-core] test fails in select_patterns - sub select_specific_patterns_by_iteration needs to be fixed

Added by SLindoMansilla 11 months ago. Updated 4 months ago.

Status:
Blocked
Priority:
Normal
Assignee:
-
Category:
Bugs in existing tests
Target version:
-
Start date:
Due date:
% Done:

50%

Estimated time:
Difficulty:

Description

Observation

lib/y2_installbase.pm::select_specific_patterns_by_iteration needs to be fixed

Reproducible


Related issues

Related to openQA Tests - action #89632: [opensuse][qe-core] test fails in select_packages in package-dependency test suiteFeedback

History

#1 Updated by tjyrinki_suse 11 months ago

  • Priority changed from Normal to Urgent
  • Start date deleted (2021-03-02)

#2 Updated by punkioudi 11 months ago

  • Assignee set to punkioudi

#3 Updated by tjyrinki_suse 11 months ago

  • Status changed from New to Workable
  • Priority changed from Urgent to High
  • Target version set to Ready

#4 Updated by tjyrinki_suse 11 months ago

  • Target version changed from Ready to QE-Core: Ready

#5 Updated by punkioudi 11 months ago

  • % Done changed from 0 to 50

PR : https://gitlab.suse.de/openqa/os-autoinst-needles-sles/-/merge_requests/1512

For these cases, no need of modifying the select_specific_patterns_by_iteration function. It is a matter of re-needling.

#6 Updated by SLindoMansilla 11 months ago

  • Has duplicate action #89632: [opensuse][qe-core] test fails in select_packages in package-dependency test suite added

#7 Updated by SLindoMansilla 11 months ago

  • Has duplicate deleted (action #89632: [opensuse][qe-core] test fails in select_packages in package-dependency test suite)

#8 Updated by SLindoMansilla 11 months ago

  • Related to action #89632: [opensuse][qe-core] test fails in select_packages in package-dependency test suite added

#10 Updated by SLindoMansilla 11 months ago

For future references, we need to avoid these false negatives: https://openqa.suse.de/tests/5642237#step/select_patterns/13

#11 Updated by SLindoMansilla 11 months ago

Here is another case:

And assert_screen times out

#12 Updated by punkioudi 11 months ago

Hm, the tricky thing here is how to overcome these false negatives, especially on this test that for different cases we have different needles etc etc. Probably we could consider opening a ticket for re-writing the module or something ^

#13 Updated by SLindoMansilla 11 months ago

punkioudi wrote:

Hm, the tricky thing here is how to overcome these false negatives, especially on this test that for different cases we have different needles etc etc. Probably we could consider opening a ticket for re-writing the module or something ^

I expected this ticket to be that ticket for fixing (re-writing) sub select_specific_patterns_by_iteration.

#14 Updated by SLindoMansilla 11 months ago

You will love this one :)

Command not found: https://openqa.suse.de/tests/5651879#step/validate_pcm_aws/2
Because the qcow2 image doesn't contain AWS pattern: https://openqa.suse.de/tests/5651878#step/select_patterns/11

#15 Updated by punkioudi 11 months ago

  • Assignee deleted (punkioudi)

Unassigning myself, as there is already another team working on the topic ^

#16 Updated by coolo 11 months ago

https://github.com/os-autoinst/os-autoinst/pull/1633 should make needling more reliable - but I can't say if this will be accepted and when it can be deployed.

#17 Updated by okurz 11 months ago

We would appreciate more testing before throwing that onto production so for example QE-Core and QE-Y can try out some specific test scenarios and at best already provide new necessary needles before we accept that change

#18 Updated by okurz 11 months ago

Reported https://bugzilla.suse.com/show_bug.cgi?id=1183519 about the bad contrast specific to recent SLE branding changes.

#19 Updated by szarate 11 months ago

  • Status changed from Workable to Blocked
  • Target version deleted (QE-Core: Ready)

#20 Updated by pcervinka 11 months ago

I think select_patterns issue impacts also create_hdd_minimal_base+sdk. Job didn't fail, but for some reason many patterns were selected https://openqa.suse.de/tests/5670487#step/installation_overview/2.

#21 Updated by SLindoMansilla 11 months ago

pcervinka wrote:

I think select_patterns issue impacts also create_hdd_minimal_base+sdk. Job didn't fail, but for some reason many patterns were selected https://openqa.suse.de/tests/5670487#step/installation_overview/2.

Yes, sadly.

#22 Updated by SLindoMansilla 11 months ago

FYI: I would be surprised if there is any job that doesn't have problems with select_patterns (false negatives included)

#23 Updated by riafarov 11 months ago

SLindoMansilla wrote:

FYI: I would be surprised if there is any job that doesn't have problems with select_patterns (false negatives included)

Luckily staging jobs do work: https://openqa.suse.de/tests/5659239 But this is coincidence, as we somehow do not select gnome patterns and still boot into text mode.
I've made Release Managers aware of this issue when it was detected there, so they are aware that test doesn't do what it's supposed to.

#24 Updated by okurz 10 months ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: create_hdd_minimal_base+sdk_withhome
https://openqa.suse.de/tests/5702807

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released"
  3. The label in the openQA scenario is removed

#25 Updated by okurz 5 months ago

  • Priority changed from High to Normal

This ticket was set to "High" priority but was not updated within 120 days which is 4 times the period of the SLO for "High" tickets (30 days) as described on https://progress.opensuse.org/projects/openqatests/wiki/Wiki#SLOs-service-level-objectives . The ticket will be set to the next lower priority of "Normal".

#26 Updated by openqa_review 4 months ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: extra_tests_kernel
https://openqa.suse.de/tests/7261529

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
  3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Also available in: Atom PDF