Project

General

Profile

action #19926

[qam] test fails in firefox - new tutorial window covering the matching area

Added by vsvecova about 5 years ago. Updated almost 5 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Bugs in existing tests
Target version:
-
Start date:
2017-09-20
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Difficulty:

Description

Observation

openQA test in scenario sle-12-SP2-Server-DVD-Updates-x86_64-qam-gnome@64bit fails in
firefox

Firefox now has a new mini-tutorial about tracking protection upon first launch.
The instruction bubble is positioned close to the tool bar and efficiently covers the matching areas.

Reproducible

Fails since (at least) Build 20170620-3

Expected result

Last good: 20170620-1 (or more recent)

Further details

Always latest result in this scenario: latest


Subtasks

action #25444: [opensuse]firefox_java: define needles for 'firefox_clean'Closedvsvecova


Related issues

Is duplicate of openQA Tests - action #19934: [qam] test fails in firefox_smoke - changes in the firefox UI, needles need updatingClosed2017-06-20

History

#1 Updated by vsvecova about 5 years ago

  • Is duplicate of action #19934: [qam] test fails in firefox_smoke - changes in the firefox UI, needles need updating added

#2 Updated by vsvecova about 5 years ago

  • Status changed from New to Closed

#3 Updated by vsvecova about 5 years ago

  • Status changed from Closed to In Progress

#4 Updated by vsvecova about 5 years ago

  • Assignee set to vsvecova

#6 Updated by RBrownSUSE about 5 years ago

  • Priority changed from Normal to Immediate

This test bug is now blocking all testing of SLE 12 SP3 RC2

Is there an ETA on a solution?

#7 Updated by dzedro about 5 years ago

  • Assignee changed from vsvecova to dzedro

#9 Updated by dzedro about 5 years ago

  • Assignee changed from dzedro to vsvecova

#11 Updated by vsvecova about 5 years ago

Ah, this is the first time I see both tracking protection and reader view in one run.
In the previous test runs it has been always either one or the other.

I'll whip up a loop then, to account for all the permutations of various pop ups.
I'll submit a new PR asap.

#13 Updated by vsvecova about 5 years ago

Ok, so as I see in https://openqa.suse.de/tests/1031692#step/firefox/6, the test is still failing because of another corner case.

The failure is due to the fact that the test asserts the reader view window, which however is immediately replaced with tracking protection window, while the test still obsesses over the reader view match and wants to close it.

I'll tweak this.

#14 Updated by okurz about 5 years ago

some recent examples of failures can also be found looking on https://w3.nue.suse.com/~okurz/openqa_suse_de_status.html for the reference to this ticket with poo#19926 , e.g. https://openqa.suse.de/tests/1033308#step/firefox/6

#15 Updated by vsvecova about 5 years ago

It appears to me that these failures are mostly due to isolated cases where the reader view window is immediately followed by the track protection tutorial.
Therefore I added a wait_still_screen into the code, right before the assert checking for the pop ups.

PR: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3202

I was not able to reproduce this particular case on my machine though, as these pesky pop ups don't seem to follow any particular pattern to be triggered, and just appear at random.
Passed on my machine: http://dreamyhamster.suse.cz/tests/342#step/firefox/4

Another thing I did, which is more or less cosmetic, is that I removed the checks from the start_firefox subroutine and instead added two more subroutines: one to check for the default browser pop up, and one for the reader view and tracking info. The reason to split the default browser out and away from the other pop ups is that the default browser check seems to always happen first, regardless of the other pop ups, therefore it did not make sense to keep it inside the loop and waste resources checking for it three times.

Also, another reason is that I am working on implementing the default browser check and pop up check subroutines into the x11regressiontest library, so they can be reused in other tests too. I currently have the first working version ready and testing it on QAM test modules. Once this is out and ready, they can be swapped in the x11/firefox.pm module too.

#16 Updated by okurz about 5 years ago

Hi, I think the "immediate" situation was resolved. We should be honest to ourselves and update an "immediate" ticket at least on a daily basis. Please reduce the priority accordingly to where you see persisting problems. It seems like the gnome-live test on o3 at least still has related problems, please take a look into https://openqa.opensuse.org/tests/453702#step/firefox/8

#17 Updated by okurz almost 5 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: qam-regression-firefox
http://openqa.suse.de/tests/1124666

#18 Updated by okurz almost 5 years ago

Hi, could you see my last comment regarding the priority of the ticket?

#19 Updated by vsvecova almost 5 years ago

  • Priority changed from Immediate to High

Hi Oliver, thanks for the nudge - indeed I didn't notice your last comment. I didn't mean to neglect updating the ticket but it does seem that it slipped between the cracks.

I saw your helpful comments on my PR (https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3327) and I've reworked the test code accordingly.
Due to issues with my machine, I have not been able to run the verification tests, however this seems resolved now and I will amend my commit ASAP.

Same goes for the needles (https://gitlab.suse.de/openqa/os-autoinst-needles-sles/merge_requests/431).

I'll get back to the opensuse issue too (unfortunately I lost some work I had done on that while I was troubleshooting my machine).
I'll update the ticket again as I continue committing my work.

#20 Updated by okurz almost 5 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: qam-regression-firefox
https://openqa.suse.de/tests/1156542

#21 Updated by vsvecova almost 5 years ago

  • Status changed from In Progress to Resolved

The PR is merged and the underlying issue has been tackled.

There are currently a few other firefox issues, which however have a different root cause and are being addressed within separate tickets (poo#25690, poo#25856 etc.)

Recent runs:
SP3: https://openqa.suse.de/tests/1213759
SP2: https://openqa.suse.de/tests/1213458
O3: https://openqa.opensuse.org/tests/484911#step/firefox/6

Marking this one as resolved.
In case I overlooked other related issues, I suggest opening a new ticket.

Also available in: Atom PDF