Project

General

Profile

action #105358

[qe-yast] [yast2_lan] test fails in yast2_lan_restart

Added by hurhaj 4 months ago. Updated 3 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Bugs in existing tests
Target version:
-
Start date:
2022-01-24
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Observation

openQA test in scenario sle-15-SP1-Server-DVD-Updates-x86_64-mau-extratests-desktop@64bit fails in
yast2_lan_restart

Test suite description

Testsuite maintained at https://gitlab.suse.de/qa-maintenance/qam-openqa-yml. Run desktop tests against aggregated test repo

Reproducible

Fails since (at least) Build 20220122-1

Expected result

Last good: 20220121-1 (or more recent)

Further details

Always latest result in this scenario: latest


Related issues

Related to openQA Tests - action #88742: [qe-core][functional] test fails in yast2_lan_restart - Network reload was expectedRejected2021-02-18

History

#1 Updated by hurhaj 4 months ago

started with the introduction of wicked update S:M:22316:262793, so it might be also a regression

#2 Updated by oorlov 4 months ago

I investigated the issue and it seems like the test module was false positive before the last build. So, the steps where the test module failed are the following:

  1. Open "Network Settings";
  2. Press "Edit" button;
  3. Select "Hardware" Tab, then "General" Tab;
  4. Press "Next";
  5. Press "Ok".

The test expected that the network will be reloaded which is incorrect, as NO changes were made in the config. I even opened a bug some time ago for this issue for SLE15-SP3: https://bugzilla.suse.com/show_bug.cgi?id=1185775

Also, please see the comment about similar issue that happened almost a year ago in Functional Job Group: https://progress.opensuse.org/issues/88742#note-2

So, I see several possible solutions here:

  • adjust the yast2_lan_restart to check that the network is not reloaded once Network Settings were opened, navigated through tabs and closed without changes; OR
  • remove the test module from your Job Group and use more recent yast2_lan_restart_bridge/vlan/bond/

UPD: I remembered that SLE15-SP1 does not have libyui-rest-api. So the first solution can be applied as the second one requires libyui-rest-api to be available.

#3 Updated by okurz 4 months ago

  • Related to action #88742: [qe-core][functional] test fails in yast2_lan_restart - Network reload was expected added

#4 Updated by okurz 4 months ago

hurhaj wrote:

started with the introduction of wicked update S:M:22316:262793, so it might be also a regression

https://openqa.suse.de/tests/8018878#comment-473091 is making it pretty clear to me that this is a product regression as in: The product changed and hence the test fails. Of course it can be an intended change. That remains to be determined.

@Juraj Hura when reviewing the test results, have you found https://openqa.suse.de/tests/8018878#comment-473091 and do you read it the same as me as: "Product regression, not test regression"?

#5 Updated by hurhaj 4 months ago

okurz wrote:

@Juraj Hura when reviewing the test results, have you found https://openqa.suse.de/tests/8018878#comment-473091 and do you read it the same as me as: "Product regression, not test regression"?

I did find it but TBH that is not at all how I read it. After all, you had to paraphrase it to ask me, right?

What I'm reading is only that retry failed too and I have no idea how to interpret "likely not a sporadic test".

#6 Updated by okurz 4 months ago

Agreed, the "likely not sporadic" was a first step and I'd like to have that extended soon to more messages and clear conclusions

#7 Updated by hurhaj 4 months ago

okurz wrote:

Agreed, the "likely not sporadic" was a first step and I'd like to have that extended soon to more messages and clear conclusions

That would be of great help.

#8 Updated by martinsmac 3 months ago

The test https://openqa.suse.de/tests/8085031#step/yast2_lan_restart/81 failed all time.
Need some atention to not block updates in Openqa.

#9 Updated by apappas 3 months ago

  • Subject changed from [qem] [yast2_lan] test fails in yast2_lan_restart to [qe-yast] [yast2_lan] test fails in yast2_lan_restart

martinsmac wrote:

The test https://openqa.suse.de/tests/8085031#step/yast2_lan_restart/81 failed all time.
Need some atention to not block updates in Openqa.

This is a product bug. The update that introduced the regression should be rescinded for the test to stop complaining.

As this is breaking on a yast test I am moving this to qe-yast, should they want to introduce a workaround.

#10 Updated by oorlov 3 months ago

  • Status changed from New to Closed
  • Assignee set to oorlov

Also available in: Atom PDF