Project

General

Profile

Actions
Copy link

action #123496

closed

[tools][openQA-in-openQA][sporadic] test fails in worker because of GNOME authentication dialog size:M

Added by jbaier_cz over 1 year ago. Updated about 1 year ago.

Status:
Resolved
Priority:
High
Assignee:
osukup
Category:
Bugs in existing tests
Target version:
openQA Project - Ready
Start date:
2023-01-23
Due date:
2023-03-17
% Done:

0%

Estimated time:
Difficulty:

Description

[tools][openQA-in-openQA][sporadic] test fails in worker

Observation

openQA test in scenario openqa-Tumbleweed-dev-x86_64-openqa_install+publish@64bit-2G fails in
worker

Apparently, gnome call for an extra authentication during screen unlocking. Some related communication also on https://matrix.to/#/!ilXMcHXPOjTZeauZcg:libera.chat/$EXhHPDm1tcODaHNPKhWlatwmdmS5ujTR8LrZDuPnDnQ

Test suite description

Maintainer: okurz@suse.de Test for installation of openQA itself. To be used with "openqa" distri. Publishes an qcow2 image including the openQA installation ready to run as an appliance.

Reproducible

Fails since (at least) Build :TW.15948 (current job).

https://openqa.opensuse.org/tests/latest?arch=x86_64&distri=openqa&flavor=dev&machine=64bit-2G&test=openqa_install%2Bpublish&version=Tumbleweed#next_previous looks very stable over multiple hundreds of jobs.

Expected result

Last good: :TW.15947 (or more recent)

Further details

Always latest result in this scenario: latest

Suggestions

  • Update needles?
  • Look for prior art (e.g. in the opensuse distribution), and handle the login accordingly

Related issues 3 (0 open — 3 closed)

Related to openQA Project - action #122440: [sporadic] openQA Assetpack download can fail on initial download size:MResolvedkraih2022-12-252023-02-24

Actions
Related to openQA Tests - action #123106: [tools][openQA-in-openQA][sporadic] test fails in docker build size:MResolvedjbaier_cz2023-01-13

Actions
Related to openQA Project - action #124316: [tools][openQA-in-openQA] test fails in openqa_worker auto_review:"zypper -n --gpg-auto-import-keys ref && zypper --no-cd -n in openQA-worker.*failed" size:MResolvedjbaier_cz2023-02-112023-02-28

Actions
Actions
Copy link
 #1

Updated by jbaier_cz over 1 year ago

  • Subject changed from [tools][openQA-in-openQA] test fails in worker to [tools][openQA-in-openQA][sporadic] test fails in worker
Actions
Copy link
 #2

Updated by okurz over 1 year ago

  • Related to action #122440: [sporadic] openQA Assetpack download can fail on initial download size:M added
Actions
Copy link
 #3

Updated by jbaier_cz over 1 year ago

  • Related to action #123106: [tools][openQA-in-openQA][sporadic] test fails in docker build size:M added
Actions
Copy link
 #4

Updated by okurz over 1 year ago

  • Description updated (diff)
  • Status changed from New to Rejected
  • Assignee set to okurz

We checked the issue shortly during the SUSE QE Tools estimation call 2023-01-26 and we have found the test scenario to be very stable, not reproducing the issue for multiple hundreds of jobs. We assume it might be a race condition in product behaviour which we do not need to care about right now assuming it will not happen again. If it does then with the additional data we can look into it in more detail.

Actions
Copy link
 #5

Updated by mkittler about 1 year ago

  • Status changed from Rejected to New

It unfortunately happened again: https://openqa.opensuse.org/tests/3107691
This is not the only occurrence. There were 8 occurrences within the last 24 hours.

Actions
Copy link
 #6

Updated by livdywan about 1 year ago

  • Status changed from New to In Progress
  • Assignee changed from okurz to livdywan

mkittler wrote:

It unfortunately happened again: https://openqa.opensuse.org/tests/3107691
This is not the only occurrence. There were 8 occurrences within the last 24 hours.

Confirmed. And it happens here:

retry -s 30 -- sh -c "zypper -n --gpg-auto-import-keys ref && zypper --no-cd -n in openQA-worker"

So I suppose we need to retry harder here as well, like we've done in #123106.

Actions
Copy link
 #7

Updated by okurz about 1 year ago

What I have seen is not a gnome screen asking for authentication but s problem when calling zypper and something about package versions

Actions
Copy link
 #8

Updated by jbaier_cz about 1 year ago

cdywan wrote:

mkittler wrote:

It unfortunately happened again: https://openqa.opensuse.org/tests/3107691
This is not the only occurrence. There were 8 occurrences within the last 24 hours.

Confirmed. And it happens here:

retry -s 30 -- sh -c "zypper -n --gpg-auto-import-keys ref && zypper --no-cd -n in openQA-worker"

So I suppose we need to retry harder here as well, like we've done in #123106.

Are you suggesting that https://openqa.opensuse.org/tests/3108013 and https://openqa.opensuse.org/tests/3107691 are the same errors? This ticket was originally created to address https://openqa.opensuse.org/tests/3107691#step/worker/4

Actions
Copy link
 #9

Updated by mkittler about 1 year ago

I'm not suggesting they're the same error. The test I've been looking at was https://openqa.opensuse.org/tests/3107691 which is about the GNOME auth dialog. The other 7 tests might not all be about this issue because I've just checked a few of them (and assumed for the rest they'd be the same as the failing module is the same).

Actions
Copy link
 #10

Updated by jbaier_cz about 1 year ago

mkittler wrote:

I'm not suggesting they're the same error. The test I've been looking at was https://openqa.opensuse.org/tests/3107691 which is about the GNOME auth dialog. The other 7 tests might not all be about this issue because I've just checked a few of them (and assumed for the rest they'd be the same as the failing module is the same).

Cris created a comment in the openQA job linking this issue to the other failure, so the question was directed more to @cdywan rather than you. The GNOME auth dialog is in the worker module, the other one (zypper and repository problem) is in openqa_worker. On the other hand, both of them are sporadic and irritating.

Actions
Copy link
 #11

Updated by openqa_review about 1 year ago

  • Due date set to 2023-02-24

Setting due date based on mean cycle time of SUSE QE Tools

Actions
Copy link
 #12

Updated by livdywan about 1 year ago

  • Status changed from In Progress to Workable

jbaier_cz wrote:

mkittler wrote:

I'm not suggesting they're the same error. The test I've been looking at was https://openqa.opensuse.org/tests/3107691 which is about the GNOME auth dialog. The other 7 tests might not all be about this issue because I've just checked a few of them (and assumed for the rest they'd be the same as the failing module is the same).

Cris created a comment in the openQA job linking this issue to the other failure, so the question was directed more to @cdywan rather than you. The GNOME auth dialog is in the worker module, the other one (zypper and repository problem) is in openqa_worker. On the other hand, both of them are sporadic and irritating.

Sorry, I didn't realize there was another error message. The other jobs I was checking had the zypper issue, and just now checking the original one here again I see it. The other link doesn't point to a specific job.

Maybe we need another ticket? Or otherwise I would unassign here for now. At least I'm not clear what the cause of the authentication issue is for now.

So feel free to grab it if you do have a better idea, otherwise I'll investigate it further on the next opportunity.

Actions
Copy link
 #13

Updated by jbaier_cz about 1 year ago

We should spit those two issues, however no idea to any of them.

The zypper issue could be solved by retrying if those attempts are long enough (in this particular case, the time between first bad and first good -- ie. OBS repository was finally consistent -- was 3 hours).

To the GNOME auth, as Oliver already wrote, it could be a product bug / race condition. Now we have the second occurrence during one month. Probably still not worth our time to investigate more.

Actions
Copy link
 #14

Updated by jbaier_cz about 1 year ago

  • Related to action #124316: [tools][openQA-in-openQA] test fails in openqa_worker auto_review:"zypper -n --gpg-auto-import-keys ref && zypper --no-cd -n in openQA-worker.*failed" size:M added
Actions
Copy link
 #15

Updated by livdywan about 1 year ago

  • Subject changed from [tools][openQA-in-openQA][sporadic] test fails in worker to [tools][openQA-in-openQA][sporadic] test fails in worker because of GNOME authentication dialog
  • Due date deleted (2023-02-24)
  • Status changed from Workable to New
  • Assignee deleted (livdywan)
Actions
Copy link
 #16

Updated by livdywan about 1 year ago

  • Subject changed from [tools][openQA-in-openQA][sporadic] test fails in worker because of GNOME authentication dialog to [tools][openQA-in-openQA][sporadic] test fails in worker because of GNOME authentication dialog size:M
  • Description updated (diff)
  • Status changed from New to Workable
Actions
Copy link
 #17

Updated by osukup about 1 year ago

  • Assignee set to osukup

It looks like simple packagekit issue - > after installation openqa-worker was released new snapshot / update in repos used by test and then packagekit asked for root password via gnome auth dialog, which broke test. --> I think there is simple workaround -> disable packagekit on SUT.

Actions
Copy link
 #18

Updated by osukup about 1 year ago

  • Status changed from Workable to In Progress
Actions
Copy link
 #20

Updated by osukup about 1 year ago

Ah, is possible there is also another processes / services in openSUSE's GNOME that can request authorization for update --> I must look into this..

Actions
Copy link
 #21

Updated by osukup about 1 year ago

on forums found gsettings set org.gnome.software download-updates false ... now question , under root user or it must be specially for bernhard ?

Actions
Copy link
 #22

Updated by openqa_review about 1 year ago

  • Due date set to 2023-03-10

Setting due date based on mean cycle time of SUSE QE Tools

Actions
Copy link
 #23

Updated by osukup about 1 year ago

  • Status changed from In Progress to Resolved

merged

Actions
Copy link
 #24

Updated by okurz about 1 year ago

  • Status changed from Resolved to Feedback
Actions
Copy link
 #25

Updated by osukup about 1 year ago

  • Status changed from Feedback to In Progress

so .. there is another trigger :(

Actions
Copy link
 #26

Updated by okurz about 1 year ago

  • Due date changed from 2023-03-10 to 2023-03-17
Actions
Copy link
 #27

Updated by okurz about 1 year ago

Actions
Copy link
 #28

Updated by osukup about 1 year ago

  • Status changed from In Progress to Feedback

merged , we will see :D

Actions
Copy link
 #29

Updated by livdywan about 1 year ago

  • Status changed from Feedback to Resolved

Always latest result in this scenario: latest

This looks all green, so I guess we're good

Actions
Copy link

Also available in: Atom PDF