action #123496: [tools][openQA-in-openQA][sporadic] test fails in worker because of GNOME authentication dialog size:M - openQA Tests (public) - openSUSE Project Management Tool

Actions

Copy link

action #123496

closed

[tools][openQA-in-openQA][sporadic] test fails in worker because of GNOME authentication dialog size:M

Added by jbaier_cz almost 2 years ago. Updated almost 2 years ago.

Status:

Resolved

Priority:

High

Assignee:

osukup

Category:

Bugs in existing tests

Target version:

openQA Project (public) - Ready

Start date:

2023-01-23

Due date:

2023-03-17

% Done:

Estimated time:

Difficulty:

Description

[tools][openQA-in-openQA][sporadic] test fails in worker

Observation¶

openQA test in scenario openqa-Tumbleweed-dev-x86_64-openqa_install+publish@64bit-2G fails in
worker

Apparently, gnome call for an extra authentication during screen unlocking. Some related communication also on https://matrix.to/#/!ilXMcHXPOjTZeauZcg:libera.chat/$EXhHPDm1tcODaHNPKhWlatwmdmS5ujTR8LrZDuPnDnQ

Test suite description¶

Maintainer: okurz@suse.de Test for installation of openQA itself. To be used with "openqa" distri. Publishes an qcow2 image including the openQA installation ready to run as an appliance.

Reproducible¶

Fails since (at least) Build :TW.15948 (current job).

https://openqa.opensuse.org/tests/latest?arch=x86_64&distri=openqa&flavor=dev&machine=64bit-2G&test=openqa_install%2Bpublish&version=Tumbleweed#next_previous looks very stable over multiple hundreds of jobs.

Expected result¶

Last good: :TW.15947 (or more recent)

Further details¶

Always latest result in this scenario: latest

Suggestions¶

Update needles?
Look for prior art (e.g. in the opensuse distribution), and handle the login accordingly

Related issues 3 (0 open — 3 closed)

Actions

Copy link

Updated by jbaier_cz almost 2 years ago

Subject changed from [tools][openQA-in-openQA] test fails in worker to [tools][openQA-in-openQA][sporadic] test fails in worker

Actions

Copy link

Updated by okurz almost 2 years ago

Related to action #122440: [sporadic] openQA Assetpack download can fail on initial download size:M added

Actions

Copy link

Updated by jbaier_cz almost 2 years ago

Related to action #123106: [tools][openQA-in-openQA][sporadic] test fails in docker build size:M added

Actions

Copy link

Updated by okurz almost 2 years ago

Description updated (diff)
Status changed from New to Rejected
Assignee set to okurz

We checked the issue shortly during the SUSE QE Tools estimation call 2023-01-26 and we have found the test scenario to be very stable, not reproducing the issue for multiple hundreds of jobs. We assume it might be a race condition in product behaviour which we do not need to care about right now assuming it will not happen again. If it does then with the additional data we can look into it in more detail.

Actions

Copy link

Updated by mkittler almost 2 years ago

Status changed from Rejected to New

It unfortunately happened again: https://openqa.opensuse.org/tests/3107691
This is not the only occurrence. There were 8 occurrences within the last 24 hours.

Actions

Copy link

Updated by livdywan almost 2 years ago

Status changed from New to In Progress
Assignee changed from okurz to livdywan

mkittler wrote:

It unfortunately happened again: https://openqa.opensuse.org/tests/3107691
This is not the only occurrence. There were 8 occurrences within the last 24 hours.

Confirmed. And it happens here:

retry -s 30 -- sh -c "zypper -n --gpg-auto-import-keys ref && zypper --no-cd -n in openQA-worker"

So I suppose we need to retry harder here as well, like we've done in #123106.

Actions

Copy link

Updated by okurz almost 2 years ago

What I have seen is not a gnome screen asking for authentication but s problem when calling zypper and something about package versions

Actions

Copy link

Updated by jbaier_cz almost 2 years ago

cdywan wrote:

mkittler wrote:

It unfortunately happened again: https://openqa.opensuse.org/tests/3107691
This is not the only occurrence. There were 8 occurrences within the last 24 hours.

Confirmed. And it happens here:
retry -s 30 -- sh -c "zypper -n --gpg-auto-import-keys ref && zypper --no-cd -n in openQA-worker"
So I suppose we need to retry harder here as well, like we've done in #123106.

Are you suggesting that https://openqa.opensuse.org/tests/3108013 and https://openqa.opensuse.org/tests/3107691 are the same errors? This ticket was originally created to address https://openqa.opensuse.org/tests/3107691#step/worker/4

Actions

Copy link

Updated by mkittler almost 2 years ago

I'm not suggesting they're the same error. The test I've been looking at was https://openqa.opensuse.org/tests/3107691 which is about the GNOME auth dialog. The other 7 tests might not all be about this issue because I've just checked a few of them (and assumed for the rest they'd be the same as the failing module is the same).

Actions

Copy link

#10

Updated by jbaier_cz almost 2 years ago

mkittler wrote:

I'm not suggesting they're the same error. The test I've been looking at was https://openqa.opensuse.org/tests/3107691 which is about the GNOME auth dialog. The other 7 tests might not all be about this issue because I've just checked a few of them (and assumed for the rest they'd be the same as the failing module is the same).

Cris created a comment in the openQA job linking this issue to the other failure, so the question was directed more to @cdywan rather than you. The GNOME auth dialog is in the worker module, the other one (zypper and repository problem) is in openqa_worker. On the other hand, both of them are sporadic and irritating.

Actions

Copy link

#11

Updated by openqa_review almost 2 years ago

Due date set to 2023-02-24

Setting due date based on mean cycle time of SUSE QE Tools

Actions

Copy link

#12

Updated by livdywan almost 2 years ago

Status changed from In Progress to Workable

jbaier_cz wrote:

mkittler wrote:

I'm not suggesting they're the same error. The test I've been looking at was https://openqa.opensuse.org/tests/3107691 which is about the GNOME auth dialog. The other 7 tests might not all be about this issue because I've just checked a few of them (and assumed for the rest they'd be the same as the failing module is the same).

Cris created a comment in the openQA job linking this issue to the other failure, so the question was directed more to @cdywan rather than you. The GNOME auth dialog is in the worker module, the other one (zypper and repository problem) is in openqa_worker. On the other hand, both of them are sporadic and irritating.

Sorry, I didn't realize there was another error message. The other jobs I was checking had the zypper issue, and just now checking the original one here again I see it. The other link doesn't point to a specific job.

Maybe we need another ticket? Or otherwise I would unassign here for now. At least I'm not clear what the cause of the authentication issue is for now.

So feel free to grab it if you do have a better idea, otherwise I'll investigate it further on the next opportunity.

Actions

Copy link

#13

Updated by jbaier_cz almost 2 years ago

We should spit those two issues, however no idea to any of them.

The zypper issue could be solved by retrying if those attempts are long enough (in this particular case, the time between first bad and first good -- ie. OBS repository was finally consistent -- was 3 hours).

To the GNOME auth, as Oliver already wrote, it could be a product bug / race condition. Now we have the second occurrence during one month. Probably still not worth our time to investigate more.

Actions

Copy link

#14

Updated by jbaier_cz almost 2 years ago

Related to action #124316: [tools][openQA-in-openQA] test fails in openqa_worker auto_review:"zypper -n --gpg-auto-import-keys ref && zypper --no-cd -n in openQA-worker.*failed" size:M added

Actions

Copy link

#15

Updated by livdywan almost 2 years ago

Subject changed from [tools][openQA-in-openQA][sporadic] test fails in worker to [tools][openQA-in-openQA][sporadic] test fails in worker because of GNOME authentication dialog
Due date deleted (~~2023-02-24~~)
Status changed from Workable to New
Assignee deleted (~~livdywan~~)

Actions

Copy link

#16

Updated by livdywan almost 2 years ago

Subject changed from [tools][openQA-in-openQA][sporadic] test fails in worker because of GNOME authentication dialog to [tools][openQA-in-openQA][sporadic] test fails in worker because of GNOME authentication dialog size:M
Description updated (diff)
Status changed from New to Workable

Actions

Copy link

#17

Updated by osukup almost 2 years ago

Assignee set to osukup

It looks like simple packagekit issue - > after installation openqa-worker was released new snapshot / update in repos used by test and then packagekit asked for root password via gnome auth dialog, which broke test. --> I think there is simple workaround -> disable packagekit on SUT.

Actions

Copy link

#18