Project

General

Profile

Actions

action #123496

closed

[tools][openQA-in-openQA][sporadic] test fails in worker because of GNOME authentication dialog size:M

Added by jbaier_cz almost 2 years ago. Updated over 1 year ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Bugs in existing tests
Start date:
2023-01-23
Due date:
2023-03-17
% Done:

0%

Estimated time:
Difficulty:

Description

[tools][openQA-in-openQA][sporadic] test fails in worker

Observation

openQA test in scenario openqa-Tumbleweed-dev-x86_64-openqa_install+publish@64bit-2G fails in
worker

Apparently, gnome call for an extra authentication during screen unlocking. Some related communication also on https://matrix.to/#/!ilXMcHXPOjTZeauZcg:libera.chat/$EXhHPDm1tcODaHNPKhWlatwmdmS5ujTR8LrZDuPnDnQ

Test suite description

Maintainer: okurz@suse.de Test for installation of openQA itself. To be used with "openqa" distri. Publishes an qcow2 image including the openQA installation ready to run as an appliance.

Reproducible

Fails since (at least) Build :TW.15948 (current job).

https://openqa.opensuse.org/tests/latest?arch=x86_64&distri=openqa&flavor=dev&machine=64bit-2G&test=openqa_install%2Bpublish&version=Tumbleweed#next_previous looks very stable over multiple hundreds of jobs.

Expected result

Last good: :TW.15947 (or more recent)

Further details

Always latest result in this scenario: latest

Suggestions

  • Update needles?
  • Look for prior art (e.g. in the opensuse distribution), and handle the login accordingly

Related issues 3 (0 open3 closed)

Related to openQA Project (public) - action #122440: [sporadic] openQA Assetpack download can fail on initial download size:MResolvedkraih2022-12-252023-02-24

Actions
Related to openQA Tests (public) - action #123106: [tools][openQA-in-openQA][sporadic] test fails in docker build size:MResolvedjbaier_cz2023-01-13

Actions
Related to openQA Project (public) - action #124316: [tools][openQA-in-openQA] test fails in openqa_worker auto_review:"zypper -n --gpg-auto-import-keys ref && zypper --no-cd -n in openQA-worker.*failed" size:MResolvedjbaier_cz2023-02-112023-02-28

Actions
Actions #1

Updated by jbaier_cz almost 2 years ago

  • Subject changed from [tools][openQA-in-openQA] test fails in worker to [tools][openQA-in-openQA][sporadic] test fails in worker
Actions #2

Updated by okurz almost 2 years ago

  • Related to action #122440: [sporadic] openQA Assetpack download can fail on initial download size:M added
Actions #3

Updated by jbaier_cz almost 2 years ago

  • Related to action #123106: [tools][openQA-in-openQA][sporadic] test fails in docker build size:M added
Actions #4

Updated by okurz almost 2 years ago

  • Description updated (diff)
  • Status changed from New to Rejected
  • Assignee set to okurz

We checked the issue shortly during the SUSE QE Tools estimation call 2023-01-26 and we have found the test scenario to be very stable, not reproducing the issue for multiple hundreds of jobs. We assume it might be a race condition in product behaviour which we do not need to care about right now assuming it will not happen again. If it does then with the additional data we can look into it in more detail.

Actions #5

Updated by mkittler almost 2 years ago

  • Status changed from Rejected to New

It unfortunately happened again: https://openqa.opensuse.org/tests/3107691
This is not the only occurrence. There were 8 occurrences within the last 24 hours.

Actions #6

Updated by livdywan almost 2 years ago

  • Status changed from New to In Progress
  • Assignee changed from okurz to livdywan

mkittler wrote:

It unfortunately happened again: https://openqa.opensuse.org/tests/3107691
This is not the only occurrence. There were 8 occurrences within the last 24 hours.

Confirmed. And it happens here:

retry -s 30 -- sh -c "zypper -n --gpg-auto-import-keys ref && zypper --no-cd -n in openQA-worker"

So I suppose we need to retry harder here as well, like we've done in #123106.

Actions #7

Updated by okurz almost 2 years ago

What I have seen is not a gnome screen asking for authentication but s problem when calling zypper and something about package versions

Actions #8

Updated by jbaier_cz almost 2 years ago

cdywan wrote:

mkittler wrote:

It unfortunately happened again: https://openqa.opensuse.org/tests/3107691
This is not the only occurrence. There were 8 occurrences within the last 24 hours.

Confirmed. And it happens here:

retry -s 30 -- sh -c "zypper -n --gpg-auto-import-keys ref && zypper --no-cd -n in openQA-worker"

So I suppose we need to retry harder here as well, like we've done in #123106.

Are you suggesting that https://openqa.opensuse.org/tests/3108013 and https://openqa.opensuse.org/tests/3107691 are the same errors? This ticket was originally created to address https://openqa.opensuse.org/tests/3107691#step/worker/4

Actions #9

Updated by mkittler almost 2 years ago

I'm not suggesting they're the same error. The test I've been looking at was https://openqa.opensuse.org/tests/3107691 which is about the GNOME auth dialog. The other 7 tests might not all be about this issue because I've just checked a few of them (and assumed for the rest they'd be the same as the failing module is the same).

Actions #10

Updated by jbaier_cz almost 2 years ago

mkittler wrote:

I'm not suggesting they're the same error. The test I've been looking at was https://openqa.opensuse.org/tests/3107691 which is about the GNOME auth dialog. The other 7 tests might not all be about this issue because I've just checked a few of them (and assumed for the rest they'd be the same as the failing module is the same).

Cris created a comment in the openQA job linking this issue to the other failure, so the question was directed more to @cdywan rather than you. The GNOME auth dialog is in the worker module, the other one (zypper and repository problem) is in openqa_worker. On the other hand, both of them are sporadic and irritating.

Actions #11

Updated by openqa_review almost 2 years ago

  • Due date set to 2023-02-24

Setting due date based on mean cycle time of SUSE QE Tools

Actions #12

Updated by livdywan almost 2 years ago

  • Status changed from In Progress to Workable

jbaier_cz wrote:

mkittler wrote:

I'm not suggesting they're the same error. The test I've been looking at was https://openqa.opensuse.org/tests/3107691 which is about the GNOME auth dialog. The other 7 tests might not all be about this issue because I've just checked a few of them (and assumed for the rest they'd be the same as the failing module is the same).

Cris created a comment in the openQA job linking this issue to the other failure, so the question was directed more to @cdywan rather than you. The GNOME auth dialog is in the worker module, the other one (zypper and repository problem) is in openqa_worker. On the other hand, both of them are sporadic and irritating.

Sorry, I didn't realize there was another error message. The other jobs I was checking had the zypper issue, and just now checking the original one here again I see it. The other link doesn't point to a specific job.

Maybe we need another ticket? Or otherwise I would unassign here for now. At least I'm not clear what the cause of the authentication issue is for now.

So feel free to grab it if you do have a better idea, otherwise I'll investigate it further on the next opportunity.

Actions #13

Updated by jbaier_cz almost 2 years ago

We should spit those two issues, however no idea to any of them.

The zypper issue could be solved by retrying if those attempts are long enough (in this particular case, the time between first bad and first good -- ie. OBS repository was finally consistent -- was 3 hours).

To the GNOME auth, as Oliver already wrote, it could be a product bug / race condition. Now we have the second occurrence during one month. Probably still not worth our time to investigate more.

Actions #14

Updated by jbaier_cz almost 2 years ago

  • Related to action #124316: [tools][openQA-in-openQA] test fails in openqa_worker auto_review:"zypper -n --gpg-auto-import-keys ref && zypper --no-cd -n in openQA-worker.*failed" size:M added
Actions #15

Updated by livdywan almost 2 years ago

  • Subject changed from [tools][openQA-in-openQA][sporadic] test fails in worker to [tools][openQA-in-openQA][sporadic] test fails in worker because of GNOME authentication dialog
  • Due date deleted (2023-02-24)
  • Status changed from Workable to New
  • Assignee deleted (livdywan)
Actions #16

Updated by livdywan almost 2 years ago

  • Subject changed from [tools][openQA-in-openQA][sporadic] test fails in worker because of GNOME authentication dialog to [tools][openQA-in-openQA][sporadic] test fails in worker because of GNOME authentication dialog size:M
  • Description updated (diff)
  • Status changed from New to Workable
Actions #17

Updated by osukup almost 2 years ago

  • Assignee set to osukup

It looks like simple packagekit issue - > after installation openqa-worker was released new snapshot / update in repos used by test and then packagekit asked for root password via gnome auth dialog, which broke test. --> I think there is simple workaround -> disable packagekit on SUT.

Actions #18

Updated by osukup almost 2 years ago

  • Status changed from Workable to In Progress
Actions #20

Updated by osukup almost 2 years ago

Ah, is possible there is also another processes / services in openSUSE's GNOME that can request authorization for update --> I must look into this..

Actions #21

Updated by osukup almost 2 years ago

on forums found gsettings set org.gnome.software download-updates false ... now question , under root user or it must be specially for bernhard ?

Actions #22

Updated by openqa_review almost 2 years ago

  • Due date set to 2023-03-10

Setting due date based on mean cycle time of SUSE QE Tools

Actions #23

Updated by osukup almost 2 years ago

  • Status changed from In Progress to Resolved

merged

Actions #24

Updated by okurz almost 2 years ago

  • Status changed from Resolved to Feedback
Actions #25

Updated by osukup almost 2 years ago

  • Status changed from Feedback to In Progress

so .. there is another trigger :(

Actions #26

Updated by okurz almost 2 years ago

  • Due date changed from 2023-03-10 to 2023-03-17
Actions #27

Updated by okurz almost 2 years ago

Actions #28

Updated by osukup almost 2 years ago

  • Status changed from In Progress to Feedback

merged , we will see :D

Actions #29

Updated by livdywan over 1 year ago

  • Status changed from Feedback to Resolved

Always latest result in this scenario: latest

This looks all green, so I guess we're good

Actions

Also available in: Atom PDF