QA » openQA Project » openQA Tests

A lot of failures about this firefox change on Build0453 (RC1C)

https://openqa.suse.de/tests/overview?result=failed&result=incomplete&arch=&failed_modules=&distri=sle&version=12-SP3&build=0453&groupid=55#

The previous PR hasn't fix all tests failures:

• https://openqa.suse.de/tests/1027166#step/firefox/9

Please take a look at this list: https://openqa.suse.de/tests/overview?arch=&failed_modules=firefox&distri=sle&version=12-SP3&build=0453&groupid=55#

Ah, this is the first time I see both tracking protection and reader view in one run.
In the previous test runs it has been always either one or the other.

I'll whip up a loop then, to account for all the permutations of various pop ups.
I'll submit a new PR asap.

New PR based on feedback: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3183

Passed on my machine: http://dreamyhamster.suse.cz/tests/285#step/firefox/5

Ok, so as I see in https://openqa.suse.de/tests/1031692#step/firefox/6, the test is still failing because of another corner case.

The failure is due to the fact that the test asserts the reader view window, which however is immediately replaced with tracking protection window, while the test still obsesses over the reader view match and wants to close it.

I'll tweak this.

some recent examples of failures can also be found looking on https://w3.nue.suse.com/~okurz/openqa_suse_de_status.html for the reference to this ticket with poo#19926 , e.g. https://openqa.suse.de/tests/1033308#step/firefox/6

It appears to me that these failures are mostly due to isolated cases where the reader view window is immediately followed by the track protection tutorial.
Therefore I added a wait_still_screen into the code, right before the assert checking for the pop ups.

PR: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3202

I was not able to reproduce this particular case on my machine though, as these pesky pop ups don't seem to follow any particular pattern to be triggered, and just appear at random.
Passed on my machine: http://dreamyhamster.suse.cz/tests/342#step/firefox/4

Another thing I did, which is more or less cosmetic, is that I removed the checks from the start_firefox subroutine and instead added two more subroutines: one to check for the default browser pop up, and one for the reader view and tracking info. The reason to split the default browser out and away from the other pop ups is that the default browser check seems to always happen first, regardless of the other pop ups, therefore it did not make sense to keep it inside the loop and waste resources checking for it three times.

Also, another reason is that I am working on implementing the default browser check and pop up check subroutines into the x11regressiontest library, so they can be reused in other tests too. I currently have the first working version ready and testing it on QAM test modules. Once this is out and ready, they can be swapped in the x11/firefox.pm module too.

Hi, I think the "immediate" situation was resolved. We should be honest to ourselves and update an "immediate" ticket at least on a daily basis. Please reduce the priority accordingly to where you see persisting problems. It seems like the gnome-live test on o3 at least still has related problems, please take a look into https://openqa.opensuse.org/tests/453702#step/firefox/8

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: qam-regression-firefox
http://openqa.suse.de/tests/1124666

Hi, could you see my last comment regarding the priority of the ticket?

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: qam-regression-firefox
https://openqa.suse.de/tests/1156542