Project

General

Profile

action #53795

[functional][u] oomath: frequent mistyping of the formula

Added by dimstar about 2 years ago. Updated about 2 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Bugs in existing tests
Target version:
-
Start date:
2019-07-03
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Observation

openQA test in scenario opensuse-Tumbleweed-DVD-x86_64-update_Leap_15.1_gnome@64bit fails in
oomath

Test suite description

Maintainer: zluo Upgrade test Leap 15.1 to TW

Reproducible

Fails since (at least) Build 20190702

Expected result

Last good: 20190701 (or more recent)

Further details

Always latest result in this scenario: latest

The formula is very frequently mistyped. Isteaf of "E %phi = H %phi" it types variations like "E 5phi" (% mistyped) - or other things.


Related issues

Related to openQA Tests - action #46190: [functional][u] test fails in user_settings - mistyping in Username (lowercase instead of uppercase) Resolved2019-01-15

History

#1 Updated by SLindoMansilla about 2 years ago

  • Subject changed from oomath: frequent mistyping of the formula to [functional][u] oomath: frequent mistyping of the formula
  • Priority changed from Normal to High

#2 Updated by StefanBruens about 2 years ago

https://openqa.opensuse.org/tests/976525#step/oocalc/6

"Hello worlD!" - i.e. the "Shift" from the "W" arrives late, or the one from "!" early ...

#3 Updated by okurz about 2 years ago

https://github.com/os-autoinst/os-autoinst/pull/1174 might help and should be tested against.

#4 Updated by okurz about 2 years ago

  • Related to action #46190: [functional][u] test fails in user_settings - mistyping in Username (lowercase instead of uppercase) added

#5 Updated by okurz about 2 years ago

PR was merged. We still have mistyping in the formula though, e.g. see https://openqa.opensuse.org/tests/977811#step/oomath/6 so the PR mentioned is not fixing it.

#6 Updated by okurz about 2 years ago

  • Status changed from New to In Progress
  • Assignee set to okurz
openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 984408 _GROUP=0 TEST=update_Leap_15.1_gnome@okurz/os-autoinst-distri-opensuse#fix/oomath BUILD=okurz/os-autoinst-distri-opensuse#fix/oomath CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse.git#fix/oomath PRODUCTDIR=os-autoinst-distri-opensuse/products/opensuse NEEDLES_DIR=/var/lib/openqa/cache/openqa1-opensuse/tests/opensuse/products/opensuse/needles EXCLUDE_MODULES=shutdown,cleanup_before_shutdown,reboot_gnome,vlc,hexchat,gimp,inkscape,desktop_mainmenu,evolution,gnome_music,nautilus,glxgears,oocalc,chromium,firefox_audio,firefox,gedit,sshxterm,gnome_tweak_tool,gnome_control_center,keymap_or_locale_x11,ssh_cleanup,_sshd,sshfs,yast2_bootloader,yast2_i,glibc_sanity,salt,curl_https

Created job #984498: opensuse-Tumbleweed-DVD-x86_64-Build20190716-update_Leap_15.1_gnome@64bit -> https://openqa.opensuse.org/t984498

The single job worked so now we can trigger some more to find out if the statistics improved. https://openqa.opensuse.org/tests/984408 and the last 25 jobs before that show 13/25 fails in oomath, so triggered 20 jobs:

for i in {01..20}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 984408 _GROUP=0 TEST=update_Leap_15.1_gnome@okurz/os-autoinst-distri-opensuse#fix/oomath_$i BUILD=okurz/os-autoinst-distri-opensuse#fix/oomath CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse.git#fix/oomath PRODUCTDIR=os-autoinst-distri-opensuse/products/opensuse NEEDLES_DIR=/var/lib/openqa/cache/openqa1-opensuse/tests/opensuse/products/opensuse/needles EXCLUDE_MODULES=shutdown,cleanup_before_shutdown,reboot_gnome,vlc,hexchat,gimp,inkscape,desktop_mainmenu,evolution,gnome_music,nautilus,glxgears,oocalc,chromium,firefox_audio,firefox,gedit,sshxterm,gnome_tweak_tool,gnome_control_center,keymap_or_locale_x11,ssh_cleanup,_sshd,sshfs,yast2_bootloader,yast2_i,glibc_sanity,salt,curl_https; done

-> https://openqa.opensuse.org/tests/overview?distri=opensuse&build=okurz%2Fos-autoinst-distri-opensuse%23fix%2Foomath&version=Tumbleweed

EDIT: 1/21 failed in oomath which is not perfect bug significantly better. The failed case is https://openqa.opensuse.org/tests/984542#step/oomath/6 where the "I" in "PHI" was typed in lowercase, i.e. "PHi" instead of "PHI". That could happen to humans as well ;) Creating a PR to cover this first part already: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/7918 . Potential next step: Type even slower (I don't have a better idea).

#7 Updated by okurz about 2 years ago

  • Status changed from In Progress to Feedback
for i in {001..100}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 984408 _GROUP=0 TEST=update_Leap_15.1_gnome@okurz/os-autoinst-distri-opensuse#fix/oomath2_$i BUILD=okurz/os-autoinst-distri-opensuse#fix/oomath2 CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse.git#fix/oomath2 PRODUCTDIR=os-autoinst-distri-opensuse/products/opensuse NEEDLES_DIR=/var/lib/openqa/cache/openqa1-opensuse/tests/opensuse/products/opensuse/needles EXCLUDE_MODULES=shutdown,cleanup_before_shutdown,reboot_gnome,vlc,hexchat,gimp,inkscape,desktop_mainmenu,evolution,gnome_music,nautilus,glxgears,oocalc,chromium,firefox_audio,firefox,gedit,sshxterm,gnome_tweak_tool,gnome_control_center,keymap_or_locale_x11,ssh_cleanup,_sshd,sshfs,yast2_bootloader,yast2_i,glibc_sanity,salt,curl_https; done

https://openqa.opensuse.org/tests/overview?build=okurz%2Fos-autoinst-distri-opensuse%23fix%2Foomath2&distri=opensuse&version=Tumbleweed

#8 Updated by okurz about 2 years ago

3/100 failures in oomath so even better but not perfect. Because all seemed to have failed already on the first characters I doubt slower typing helps here. Either we wait upfront or we try multiple times in case of failures.
So trying with an additional wait_still_screen at start:

for i in {001..100}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 986478 _GROUP=0 TEST=update_Leap_15.1_gnome@okurz/os-autoinst-distri-opensuse#fix/oomath2_$i BUILD=okurz/os-autoinst-distri-opensuse#fix/oomath2_2 CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse.git#fix/oomath2 PRODUCTDIR=os-autoinst-distri-opensuse/products/opensuse NEEDLES_DIR=/var/lib/openqa/cache/openqa1-opensuse/tests/opensuse/products/opensuse/needles EXCLUDE_MODULES=shutdown,cleanup_before_shutdown,reboot_gnome,vlc,hexchat,gimp,inkscape,desktop_mainmenu,evolution,gnome_music,nautilus,glxgears,oocalc,chromium,firefox_audio,firefox,gedit,sshxterm,gnome_tweak_tool,gnome_control_center,keymap_or_locale_x11,ssh_cleanup,_sshd,sshfs,yast2_bootloader,yast2_i,glibc_sanity,salt,curl_https; done

Created job #986574: opensuse-Tumbleweed-DVD-x86_64-Build20190718-update_Leap_15.1_gnome@64bit -> https://openqa.opensuse.org/t986574 -> https://openqa.opensuse.org/tests/overview?version=Tumbleweed&build=okurz%2Fos-autoinst-distri-opensuse%23fix%2Foomath2_2&distri=opensuse

#9 Updated by okurz about 2 years ago

failed in 1/100 in https://openqa.opensuse.org/tests/986602#step/oomath/7 with a small leading "e" instead of "E". However the wait_still_screen calls all timed out so they were simply waiting for 30s unconditionally e.g. visible in https://openqa.opensuse.org/tests/986635/file/autoinst-log.txt or https://openqa.opensuse.org/tests/986602/file/autoinst-log.txt when searching for "E %PHI". I could use assert_still_screen but this won't fix the problem for good. I guess we should simply retry internally.

for i in {001..004}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 988855 _GROUP=0 TEST=update_Leap_15.1_gnome@okurz/os-autoinst-distri-opensuse#fix/oomath2_$i BUILD=okurz/os-autoinst-distri-opensuse#fix/oomath2_3 CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse.git#fix/oomath2 PRODUCTDIR=os-autoinst-distri-opensuse/products/opensuse NEEDLES_DIR=/var/lib/openqa/cache/openqa1-opensuse/tests/opensuse/products/opensuse/needles EXCLUDE_MODULES=shutdown,cleanup_before_shutdown,reboot_gnome,vlc,hexchat,gimp,inkscape,desktop_mainmenu,evolution,gnome_music,nautilus,glxgears,oocalc,chromium,firefox_audio,firefox,gedit,sshxterm,gnome_tweak_tool,gnome_control_center,keymap_or_locale_x11,ssh_cleanup,_sshd,sshfs,yast2_bootloader,yast2_i,glibc_sanity,salt,curl_https; done

Created job #989583: opensuse-Tumbleweed-DVD-x86_64-Build20190722-update_Leap_15.1_gnome@64bit -> https://openqa.opensuse.org/t989583 -> https://openqa.opensuse.org/tests/overview?build=okurz%2Fos-autoinst-distri-opensuse%23fix%2Foomath2_3

#10 Updated by okurz about 2 years ago

that worked. 1/100 fails in oomath in https://openqa.opensuse.org/tests/990886#step/oomath/5 however this time it looks like really something completely different. The application does not show up at all in time but the process is there . So this means we are good to go with the current state:
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/8034

#11 Updated by okurz about 2 years ago

I am dissapointed how little others care that I have the fix ready for review with no single response within 21 days, merging myself now.

#12 Updated by okurz about 2 years ago

  • Status changed from Feedback to Resolved

Not that much but at least in the last 7 jobs preceding https://openqa.opensuse.org/tests/1011014 it looks like oomath did not fail.

Also available in: Atom PDF