action #53795
closed[functional][u] oomath: frequent mistyping of the formula
0%
Description
Observation¶
openQA test in scenario opensuse-Tumbleweed-DVD-x86_64-update_Leap_15.1_gnome@64bit fails in
oomath
Test suite description¶
Maintainer: zluo Upgrade test Leap 15.1 to TW
Reproducible¶
Fails since (at least) Build 20190702
Expected result¶
Last good: 20190701 (or more recent)
Further details¶
Always latest result in this scenario: latest
The formula is very frequently mistyped. Isteaf of "E %phi = H %phi" it types variations like "E 5phi" (% mistyped) - or other things.
Updated by SLindoMansilla over 5 years ago
- Subject changed from oomath: frequent mistyping of the formula to [functional][u] oomath: frequent mistyping of the formula
- Priority changed from Normal to High
Updated by StefanBruens over 5 years ago
https://openqa.opensuse.org/tests/976525#step/oocalc/6
"Hello worlD!" - i.e. the "Shift" from the "W" arrives late, or the one from "!" early ...
Updated by okurz over 5 years ago
https://github.com/os-autoinst/os-autoinst/pull/1174 might help and should be tested against.
Updated by okurz over 5 years ago
- Related to action #46190: [functional][u] test fails in user_settings - mistyping in Username (lowercase instead of uppercase) added
Updated by okurz over 5 years ago
PR was merged. We still have mistyping in the formula though, e.g. see https://openqa.opensuse.org/tests/977811#step/oomath/6 so the PR mentioned is not fixing it.
Updated by okurz over 5 years ago
- Status changed from New to In Progress
- Assignee set to okurz
openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 984408 _GROUP=0 TEST=update_Leap_15.1_gnome@okurz/os-autoinst-distri-opensuse#fix/oomath BUILD=okurz/os-autoinst-distri-opensuse#fix/oomath CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse.git#fix/oomath PRODUCTDIR=os-autoinst-distri-opensuse/products/opensuse NEEDLES_DIR=/var/lib/openqa/cache/openqa1-opensuse/tests/opensuse/products/opensuse/needles EXCLUDE_MODULES=shutdown,cleanup_before_shutdown,reboot_gnome,vlc,hexchat,gimp,inkscape,desktop_mainmenu,evolution,gnome_music,nautilus,glxgears,oocalc,chromium,firefox_audio,firefox,gedit,sshxterm,gnome_tweak_tool,gnome_control_center,keymap_or_locale_x11,ssh_cleanup,_sshd,sshfs,yast2_bootloader,yast2_i,glibc_sanity,salt,curl_https
Created job #984498: opensuse-Tumbleweed-DVD-x86_64-Build20190716-update_Leap_15.1_gnome@64bit -> https://openqa.opensuse.org/t984498
The single job worked so now we can trigger some more to find out if the statistics improved. https://openqa.opensuse.org/tests/984408 and the last 25 jobs before that show 13/25 fails in oomath, so triggered 20 jobs:
for i in {01..20}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 984408 _GROUP=0 TEST=update_Leap_15.1_gnome@okurz/os-autoinst-distri-opensuse#fix/oomath_$i BUILD=okurz/os-autoinst-distri-opensuse#fix/oomath CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse.git#fix/oomath PRODUCTDIR=os-autoinst-distri-opensuse/products/opensuse NEEDLES_DIR=/var/lib/openqa/cache/openqa1-opensuse/tests/opensuse/products/opensuse/needles EXCLUDE_MODULES=shutdown,cleanup_before_shutdown,reboot_gnome,vlc,hexchat,gimp,inkscape,desktop_mainmenu,evolution,gnome_music,nautilus,glxgears,oocalc,chromium,firefox_audio,firefox,gedit,sshxterm,gnome_tweak_tool,gnome_control_center,keymap_or_locale_x11,ssh_cleanup,_sshd,sshfs,yast2_bootloader,yast2_i,glibc_sanity,salt,curl_https; done
EDIT: 1/21 failed in oomath which is not perfect bug significantly better. The failed case is https://openqa.opensuse.org/tests/984542#step/oomath/6 where the "I" in "PHI" was typed in lowercase, i.e. "PHi" instead of "PHI". That could happen to humans as well ;) Creating a PR to cover this first part already: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/7918 . Potential next step: Type even slower (I don't have a better idea).
Updated by okurz over 5 years ago
- Status changed from In Progress to Feedback
for i in {001..100}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 984408 _GROUP=0 TEST=update_Leap_15.1_gnome@okurz/os-autoinst-distri-opensuse#fix/oomath2_$i BUILD=okurz/os-autoinst-distri-opensuse#fix/oomath2 CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse.git#fix/oomath2 PRODUCTDIR=os-autoinst-distri-opensuse/products/opensuse NEEDLES_DIR=/var/lib/openqa/cache/openqa1-opensuse/tests/opensuse/products/opensuse/needles EXCLUDE_MODULES=shutdown,cleanup_before_shutdown,reboot_gnome,vlc,hexchat,gimp,inkscape,desktop_mainmenu,evolution,gnome_music,nautilus,glxgears,oocalc,chromium,firefox_audio,firefox,gedit,sshxterm,gnome_tweak_tool,gnome_control_center,keymap_or_locale_x11,ssh_cleanup,_sshd,sshfs,yast2_bootloader,yast2_i,glibc_sanity,salt,curl_https; done
Updated by okurz over 5 years ago
3/100 failures in oomath so even better but not perfect. Because all seemed to have failed already on the first characters I doubt slower typing helps here. Either we wait upfront or we try multiple times in case of failures.
So trying with an additional wait_still_screen at start:
for i in {001..100}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 986478 _GROUP=0 TEST=update_Leap_15.1_gnome@okurz/os-autoinst-distri-opensuse#fix/oomath2_$i BUILD=okurz/os-autoinst-distri-opensuse#fix/oomath2_2 CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse.git#fix/oomath2 PRODUCTDIR=os-autoinst-distri-opensuse/products/opensuse NEEDLES_DIR=/var/lib/openqa/cache/openqa1-opensuse/tests/opensuse/products/opensuse/needles EXCLUDE_MODULES=shutdown,cleanup_before_shutdown,reboot_gnome,vlc,hexchat,gimp,inkscape,desktop_mainmenu,evolution,gnome_music,nautilus,glxgears,oocalc,chromium,firefox_audio,firefox,gedit,sshxterm,gnome_tweak_tool,gnome_control_center,keymap_or_locale_x11,ssh_cleanup,_sshd,sshfs,yast2_bootloader,yast2_i,glibc_sanity,salt,curl_https; done
Created job #986574: opensuse-Tumbleweed-DVD-x86_64-Build20190718-update_Leap_15.1_gnome@64bit -> https://openqa.opensuse.org/t986574 -> https://openqa.opensuse.org/tests/overview?version=Tumbleweed&build=okurz%2Fos-autoinst-distri-opensuse%23fix%2Foomath2_2&distri=opensuse
Updated by okurz over 5 years ago
failed in 1/100 in https://openqa.opensuse.org/tests/986602#step/oomath/7 with a small leading "e" instead of "E". However the wait_still_screen calls all timed out so they were simply waiting for 30s unconditionally e.g. visible in https://openqa.opensuse.org/tests/986635/file/autoinst-log.txt or https://openqa.opensuse.org/tests/986602/file/autoinst-log.txt when searching for "E %PHI". I could use assert_still_screen but this won't fix the problem for good. I guess we should simply retry internally.
for i in {001..004}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 988855 _GROUP=0 TEST=update_Leap_15.1_gnome@okurz/os-autoinst-distri-opensuse#fix/oomath2_$i BUILD=okurz/os-autoinst-distri-opensuse#fix/oomath2_3 CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse.git#fix/oomath2 PRODUCTDIR=os-autoinst-distri-opensuse/products/opensuse NEEDLES_DIR=/var/lib/openqa/cache/openqa1-opensuse/tests/opensuse/products/opensuse/needles EXCLUDE_MODULES=shutdown,cleanup_before_shutdown,reboot_gnome,vlc,hexchat,gimp,inkscape,desktop_mainmenu,evolution,gnome_music,nautilus,glxgears,oocalc,chromium,firefox_audio,firefox,gedit,sshxterm,gnome_tweak_tool,gnome_control_center,keymap_or_locale_x11,ssh_cleanup,_sshd,sshfs,yast2_bootloader,yast2_i,glibc_sanity,salt,curl_https; done
Created job #989583: opensuse-Tumbleweed-DVD-x86_64-Build20190722-update_Leap_15.1_gnome@64bit -> https://openqa.opensuse.org/t989583 -> https://openqa.opensuse.org/tests/overview?build=okurz%2Fos-autoinst-distri-opensuse%23fix%2Foomath2_3
Updated by okurz over 5 years ago
that worked. 1/100 fails in oomath in https://openqa.opensuse.org/tests/990886#step/oomath/5 however this time it looks like really something completely different. The application does not show up at all in time but the process is there . So this means we are good to go with the current state:
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/8034
Updated by okurz over 5 years ago
I am dissapointed how little others care that I have the fix ready for review with no single response within 21 days, merging myself now.
Updated by okurz over 5 years ago
- Status changed from Feedback to Resolved
Not that much but at least in the last 7 jobs preceding https://openqa.opensuse.org/tests/1011014 it looks like oomath did not fail.