action #19926
closed[qam] test fails in firefox - new tutorial window covering the matching area
Added by vsvecova about 7 years ago. Updated almost 7 years ago.
100%
Description
Observation¶
openQA test in scenario sle-12-SP2-Server-DVD-Updates-x86_64-qam-gnome@64bit fails in
firefox
Firefox now has a new mini-tutorial about tracking protection upon first launch.
The instruction bubble is positioned close to the tool bar and efficiently covers the matching areas.
Reproducible¶
Fails since (at least) Build 20170620-3
Expected result¶
Last good: 20170620-1 (or more recent)
Further details¶
Always latest result in this scenario: latest
Updated by vsvecova about 7 years ago
- Is duplicate of action #19934: [qam] test fails in firefox_smoke - changes in the firefox UI, needles need updating added
Updated by SLindoMansilla about 7 years ago
A lot of failures about this firefox change on Build0453 (RC1C)
Updated by RBrownSUSE about 7 years ago
- Priority changed from Normal to Immediate
This test bug is now blocking all testing of SLE 12 SP3 RC2
Is there an ETA on a solution?
Updated by vsvecova about 7 years ago
- % Done changed from 0 to 60
x11/firefox.pm
Test code PR: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3175
Needles MR: https://gitlab.suse.de/openqa/os-autoinst-needles-sles/merge_requests/401
Test now passing on my local instance:
SP1: http://dreamyhamster.suse.cz/tests/272#step/firefox/4
SP2: http://dreamyhamster.suse.cz/tests/273#step/firefox/4
Updated by SLindoMansilla about 7 years ago
The previous PR hasn't fix all tests failures:
Please take a look at this list: https://openqa.suse.de/tests/overview?arch=&failed_modules=firefox&distri=sle&version=12-SP3&build=0453&groupid=55#
Updated by vsvecova about 7 years ago
Ah, this is the first time I see both tracking protection and reader view in one run.
In the previous test runs it has been always either one or the other.
I'll whip up a loop then, to account for all the permutations of various pop ups.
I'll submit a new PR asap.
Updated by vsvecova about 7 years ago
New PR based on feedback: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3183
Passed on my machine: http://dreamyhamster.suse.cz/tests/285#step/firefox/5
Updated by vsvecova about 7 years ago
Ok, so as I see in https://openqa.suse.de/tests/1031692#step/firefox/6, the test is still failing because of another corner case.
The failure is due to the fact that the test asserts the reader view window, which however is immediately replaced with tracking protection window, while the test still obsesses over the reader view match and wants to close it.
I'll tweak this.
Updated by okurz about 7 years ago
some recent examples of failures can also be found looking on https://w3.nue.suse.com/~okurz/openqa_suse_de_status.html for the reference to this ticket with poo#19926 , e.g. https://openqa.suse.de/tests/1033308#step/firefox/6
Updated by vsvecova about 7 years ago
It appears to me that these failures are mostly due to isolated cases where the reader view window is immediately followed by the track protection tutorial.
Therefore I added a wait_still_screen into the code, right before the assert checking for the pop ups.
PR: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3202
I was not able to reproduce this particular case on my machine though, as these pesky pop ups don't seem to follow any particular pattern to be triggered, and just appear at random.
Passed on my machine: http://dreamyhamster.suse.cz/tests/342#step/firefox/4
Another thing I did, which is more or less cosmetic, is that I removed the checks from the start_firefox subroutine and instead added two more subroutines: one to check for the default browser pop up, and one for the reader view and tracking info. The reason to split the default browser out and away from the other pop ups is that the default browser check seems to always happen first, regardless of the other pop ups, therefore it did not make sense to keep it inside the loop and waste resources checking for it three times.
Also, another reason is that I am working on implementing the default browser check and pop up check subroutines into the x11regressiontest library, so they can be reused in other tests too. I currently have the first working version ready and testing it on QAM test modules. Once this is out and ready, they can be swapped in the x11/firefox.pm module too.
Updated by okurz about 7 years ago
Hi, I think the "immediate" situation was resolved. We should be honest to ourselves and update an "immediate" ticket at least on a daily basis. Please reduce the priority accordingly to where you see persisting problems. It seems like the gnome-live test on o3 at least still has related problems, please take a look into https://openqa.opensuse.org/tests/453702#step/firefox/8
Updated by okurz about 7 years ago
This is an autogenerated message for openQA integration by the openqa_review script:
This bug is still referenced in a failing openQA test: qam-regression-firefox
http://openqa.suse.de/tests/1124666
Updated by okurz about 7 years ago
Hi, could you see my last comment regarding the priority of the ticket?
Updated by vsvecova about 7 years ago
- Priority changed from Immediate to High
Hi Oliver, thanks for the nudge - indeed I didn't notice your last comment. I didn't mean to neglect updating the ticket but it does seem that it slipped between the cracks.
I saw your helpful comments on my PR (https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3327) and I've reworked the test code accordingly.
Due to issues with my machine, I have not been able to run the verification tests, however this seems resolved now and I will amend my commit ASAP.
Same goes for the needles (https://gitlab.suse.de/openqa/os-autoinst-needles-sles/merge_requests/431).
I'll get back to the opensuse issue too (unfortunately I lost some work I had done on that while I was troubleshooting my machine).
I'll update the ticket again as I continue committing my work.
Updated by okurz almost 7 years ago
This is an autogenerated message for openQA integration by the openqa_review script:
This bug is still referenced in a failing openQA test: qam-regression-firefox
https://openqa.suse.de/tests/1156542
Updated by vsvecova almost 7 years ago
- Status changed from In Progress to Resolved
The PR is merged and the underlying issue has been tackled.
There are currently a few other firefox issues, which however have a different root cause and are being addressed within separate tickets (poo#25690, poo#25856 etc.)
Recent runs:
SP3: https://openqa.suse.de/tests/1213759
SP2: https://openqa.suse.de/tests/1213458
O3: https://openqa.opensuse.org/tests/484911#step/firefox/6
Marking this one as resolved.
In case I overlooked other related issues, I suggest opening a new ticket.