action #20228
closed[qam]stop relying on html5test.com
0%
Description
Observation¶
openQA test in scenario sle-12-SP1-Server-DVD-Updates-x86_64-qam-gnome@64bit fails in
firefox
Relying on external webservers is not optimal. Even visiting the same webpage (in that case html5test.com) has been proven to be quite unstable, since within 24h timeframe, their server triggers different behavior of Firefox' Tracking Protection mechanism. As a result, yesterday, the same test passed, but today is failing. Also, re-adjusting the needle is not a option, since by adding the shield icon, the URL gets shifted to the more right direction (thus the needle with missmatch).
As a solution, I would propose to replace 'html5test.com' with a local webpage generated by a webserver. Some approaches:
zypper in -t pattern lamp_server (this will also install 'apache2-example-pages' pkg which will automatically create an index.html accessible by localhost)
systemctl start apache2
curl -s localhost | grep "It works!"
// Now that you know that your page is accessible, fire Firefox and do your tests
Alternatively, we could build a docker image that provides a pre-configured website by default:
zypper in docker
docker run
Another approach suggested by @asmorodskyi would be to use https://www.lighttpd.net/ . The benefit is that in this approach we will not rely on zypper to install apache2 but just rar.gz as test resource. In that case we are not adding additional reason to fail the test.
What do you think?
Reproducible¶
Fails since (at least) Build 20170703-1 (current job)
Expected result¶
Last good: 20170702-1 (or more recent)
Further details¶
Always latest result in this scenario: latest