action #17442
Updated by okurz about 7 years ago
## observation tests like zypper_lifecycle, sshd, ssh_sleanup, sshxterm and other started to fail much more than some time ago. It is not new issue, but this failures were not so frequent as now. All fails are caused by typing to soon or letters get lost because of very low performance. #16738 is somehow related. ## reproducible seems to happen in some jobs ## problem * H1 REJECTED worker overbooking * H2 qemu loosing keys -> unlikely as it was happening more often (or at all) recently but not in before, see H5; we can not be so sure. Probably it was seen in before (we have tickets for this) but as it persists it stays in people's mind. * H2.1 REJECTED loosing keys in keyboard buffer -> rejected: we would expect "key event buffer full" messages and an early fail in the jobs * H2.2 loosing keys in some part of qemu before it reaches the keyboard buffer (VNC part?) * H3 performance regression in the product * H3.1 … recently detectable on very fast machines * H3.2 REJECTED detectable on any worker -> #17442#note-32 * H4 REJECTED network -> only affecting x11 tests and not console tests so why should we loose just VNC typing events there? (okurz) * H5 updates on the workers * H5.1 REJECTED package updates on the workers (not openQA packages) -> no updates on worker since 3 weeks (rbrown) * H5.2 REJECTED os-autoinst update on the workers -> #17442#note-32 * H5.3 updates of tests * H6 it was there all the time but nobody cared because it was not seen often enough and now with more and more jobs and more thorough review again it stays in everyones scope ## suggestion * E1-1 DONE: check workers of last examples -> O1-1 linked jobs are spread over openqaworker2, …3, …5, …6 -> REJECT H1 * E3-1 DONE: crosscheck with differing parameters (older older build / test on other machines) -> #17442#note-32 -> SPLIT H3 * E5.3-1 crosscheck with older/newer version of tests ## further details Few examples: https://openqa.suse.de/tests/793680#step/yast2_snapper/5 https://openqa.suse.de/tests/793974#step/zypper_lifecycle/55 <del>https://openqa.suse.de/tests/793716#step/ssh_cleanup/3</del> --> #14826 <del>https://openqa.suse.de/tests/793683#step/sshd/16</del> --> #14826 http://openqa.suse.de/t796211 https://openqa.suse.de/tests/796232#step/gnome_terminal/7 https://openqa.suse.de/tests/795241 (Build 0260) ## workaround retrigger job