QA » openQA Project

osukup wrote in #note-5:

Table of testapi functions in test distributions. Used are almalinux, fedora, Debian, opensuse, openQA and OBS test distributions with all together and separate openSUSE:

function	global	opensuse	rest	comment
script_run	12995	10798	2197
assert_script_run	9746	8121	1625
get_var	6616	4686	1930
send_key	5813	4715	1098
assert_screen	3810	2513	1297
script_output	3023	2825	198
record_info	2639	2628	11
assert_and_click	2626	805	1821
wait_still_screen	1741	925	816
check_var	1547	1455	92
select_console	1105	1092	13
validate_script_output	1050	909	141
type_string	989	673	316
enter_cmd	897	866	31
get_required_var	886	869	17
upload_logs	814	560	254
wait_screen_change	806	686	120
save_screenshot	770	735	35
check_screen	714	314	400
match_has_tag	704	543	161
set_var	684	644	40
send_key_until_needlematch	491	418	73
record_soft_failure	460	357	103
wait_serial	364	330	34
x11_start_program	324	324	0	distribution implemenation
diag	279	247	32
data_url	265	265	0
type_password	168	143	25
click_lastmatch	134	13	121
autoinst_url	118	118	0
mouse_set	97	30	67
mouse_hide	95	46	49
is_serial_terminal	87	57	30
reset_consoles	70	70	0
become_root	69	62	7	distribution reimplemetation
console	51	51	0
ensure_installed	43	43	0	distribution implemenation
parse_extra_log	38	38	0
upload_asset	36	28	8
assert_and_dclick	31	29	2
script_sudo	30	17	13
assert_shutdown	27	20	7
hashed_string	26	12	14
check_var_array	25	19	6	debian analyse.pm
save_tmp_file	24	24	0
parse_junit_log	21	21	0
mouse_click	18	10	8
get_var_array	16	11	5	debian analyse.pl
hold_key	16	13	3
release_key	15	12	3
current_console	13	13	0
get_test_data	9	9	0
assert_script_sudo	7	3	4	distribution implemenation ... practically nver used
start_audiocapture	7	4	3
assert_screen_change	7	6	1
force_soft_failure	6	6	0
power	5	1	4
check_recorded_sound	4	4	0	one in comment
check_shutdown	4	3	1
eject_cd	4	1	3	somewhere is comment ... command is broken
assert_recorded_sound	2	1	1
resume_vm	2	2	0
mouse_tclick	1	1	0	wow
assert_still_screen	1	1	0	called as testapi::assert_still_screen
mouse_dclick	0	0	0
background_script_run	0	0	0
switch_network	0	0	0	fedora has sub in gnome-panel ...
save_memory_dump	0	0	0
freeze_vm	0	0	0
save_storage	0	0	0
mouse_drag	0	0	0

opensuse: git@github.com:os-autoinst/os-autoinst-distri-opensuse.git
obs: https://github.com/os-autoinst/os-autoinst-distri-obs.git
openqa: git@github.com:os-autoinst/os-autoinst-distri-openQA.git
debian: https://salsa.debian.org/qa/openqa/openqa-tests-debian.git
fedora: https://pagure.io/fedora-qa/os-autoinst-distri-fedora.git
almalinux: git@github.com:AlmaLinux/os-autoinst-distri-almalinux.git

We discussed these least-used functions together in the team:

• freeze_vm/resume_vm:
  • freezing is seemingly never used?
  • does this work? what's the use case? used for snapshots/backup of the disk?
  • merge this with save_storage/save_memory_dump?
  • is it feasible to use in the debugging case?
• mouse_drag
  • maybe dragging is implemented as press/down/up/click instead? likely we would have some tests dragging things with a mouse?
  • It actually has been used in the past, funnily as a workaround: a687332d0ad948bf36375711c5850d5d5f75d8f0 + # On x11, workaround bsc#1191112 by mouse click or drag the dialog. + if (($setup_console =~ /x11/) && (check_var('RMT_TEST', 'rmt_chinese'))) { + record_soft_failure('bsc#1191112 - When navigating through YaST module screens the next screen appears, but its content is not loaded'); + wait_still_screen(10, 15); + mouse_drag(startx => 480, starty => 50, endx => 485, endy => 50, button => 'left'); + }
• switch_network:
  • fedora has an implementation of this under its own distri
  • maybe this was lost and never upstreamed?
• background_script_run:
  • why is this never used? maybe we don't need it? script_run with & or nohup might be enough?
  • it's distrio-specific
  • used in ltp tests
• asset/check_recorded_sound/start_audiocapture:
  • is it worth adding another case to confirm if it's easy enough to use?
  • it's used in multiple distris... but why not more in general?
• mouse_dclick:
  • used internally, arguably redundant with asset_and_click and click_last_match exposing double clicking
• mouse_tclick:
  • used by gedit, trivial code but not made available by any other means and not something one would want to copy (a lot of clicks and sleeps)
  • maybe other code tries to triple-click but doesn't use this api?
• eject_cd:
  • is this usable? is this broken? should be worth checking
• current_console:
  • only used in opensuse
• get_test_data
• assert_script_sudo:
  • distribution implementation practically never used
  • this relies on script_sudo which relies on the distri implementation
  • fedora/alma have an implementation but don't use it: os-autoinst-distri-almalinux/lib/fedoradistribution.pm:18
• assert_screen_change
• force_soft_failure
• power

Action points¶

• Provide a list of repos used as a reference to search the test distributions we consider relevant/covered here @osukup
• Provide a reproducible command line to lookup the uses of testapi functions (rg/git grep) so we can double-check the results, or see if we made some progress down the line @osukup
• After this we can distribute the discussed least-used commands amongst the team to evaluate e.g. spend a couple hours at most to come up with a suggestion (whole team)