QA » openQA Project » openQA Tests

@msmeissn for easier ticket creation you can use the openQA built-in reporting feature, see https://wiki.microfocus.net/index.php?title=RD-OPS_QA/openQA_review#Workflow_for_SLES.2C_SLED_and_HA_as_an_example for an example with screenshot

After further looking I am pretty the problem is gnome taking too long to show up:

https://openqa.opensuse.org/tests/848696#step/updates_packagekit_gpk/3
https://openqa.opensuse.org/tests/849010#step/updates_packagekit_gpk/3

One time the screen is completely black, one time we have at least the bar on the top
I am not totally sure whether this is a test-performance or a product issue

Setting back to "Urgent" as you haven't actually picked it up.

I'll give it a look through the day.

It was consistently failing on Feb 7, but seems not after?.

I fail to see the urgency of this ticket at this point, as for what I could check, the tests simply pass on my worker (They weren't matching the needle for the first ones, the rest are chopping through the work without problems, but is not showing the symptoms described in the ticket itself, after a needle update, life finds it's way again...).

Anywho, changing priority as last builds on o3 have been passing without problems. Waiting for the rest of jobs in my instance to finish...

So, last job is passing: https://openqa.opensuse.org/tests/latest?version=15.0&arch=x86_64&test=gnome&machine=64bit-2G&distri=opensuse&flavor=DVD-Updates

The rest of jobs on my instance started to fail due to missing repos. In any case, looks like gnome is just slow to show up (Peformance problems? also worker seems busy with 4+ seconds)... after looking at: https://openqa.opensuse.org/tests/854596/file/autoinst-log.txt

Suggestions would be:

• - Change the assert_screen to increase the timeout to 60 seconds for the second loop, and add a soft failure already there
• - Actually wait, since this wait_still_screen call looks like a noop for me, sleeping for 5 seconds.

Regarless of any of the suggestions: Log where's the counter. Eases up bug investigation.

szarate wrote:

It was consistently failing on Feb 7, but seems not after?.

It still fails some times as we can see now, e.g. https://openqa.opensuse.org/tests/854596#step/updates_packagekit_gpk/3 from 13h ago so as in the subject: "sporadic" as in "not reproducibly all the time"

I fail to see the urgency of this ticket at this point
[…]
Anywho, changing priority as last builds on o3 have been passing without problems. Waiting for the rest of jobs in my instance to finish...

Thank you for looking into this. And I agree that you removed the urgency by confirming that the jobs do not fail in 100% of the cases and that we have a valid workaround (retrigger failed tests). I have noted that in the description now.

In any case, looks like gnome is just slow to show up (Peformance problems? also worker seems busy with 4+ seconds)

And this is why I bumped the prio up a little bit again to "High" as it could be either a product regression or a recent test regression and bisecting is easier when both product and test code changes are more recent. I have put suggestions in the ticket description to bisect both test and/or product.

Suggestions would be:

• - Change the assert_screen to increase the timeout to 60 seconds for the second loop, and add a soft failure already there
• - Actually wait, since this wait_still_screen call looks like a noop for me, sleeping for 5 seconds.

Regarless of any of the suggestions: Log where's the counter. Eases up bug investigation.

Yes, that can help however I see other possibilities as well which I have suggested in the updated description.

So as you asked, updated and back to "Workable" with urgency removed, hence "Urgent"->"High"

I don't think wait_still_screen(5) is a stable approach either. Hence my suggestion "Increase stability of test module or scenario, e.g. by reordering test module schedule"

updated my pr to better mark calls to ensure unlocked desktop. That's all for the time being for me.

Updated the description with suggestions, @okurz set back to workable if agreed.

Unfortunately I doubt any approach similar to https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6825 would help here as we start from a graphical session and that is the problem. I blocked this ticket by #48110 now as I think that one has priority which could involve many more unstable test modules. Only then we should revisit here.

Move to new QSF-u PO after I moved to the "tools"-team. I mainly checked the subject line so in individual instances you might not agree to take it over completely into QSF-u. Feel free to discuss with me or reassign to me or someone else in this case. Thanks.

to be groomed - still happening

The reported problem doesn't happen anymore. The new failure is related to the serial device: https://openqa.opensuse.org/tests/1014115#step/updates_packagekit_gpk/48

Leap 15.0 is now EOL, but this still happens sporadically for 15.1 updates_packagekit_gpk and it looks the similar kind of problems like originally, not any different https://openqa.opensuse.org/tests/1283905

This used to be in QSF-u backlog for a long time and the milestones are still being updated accordingly, should it be still there as the problem remains as originally described?

Not reproducible as described in this ticket.

The last occurrence of a failing updates_packagekit_gpk is caused by missing keys (notice missing pair single quote) and clicks (needle expects xterm to be closed): https://openqa.opensuse.org/tests/1014115#step/updates_packagekit_gpk/51