Project

General

Profile

Actions

action #138032

closed

coordination #154777: [saga][epic] Shareable os-autoinst and test distribution plugins

coordination #108527: [epic] os-autoinst wheels for scalable code reuse of helper functions and segmented test distributions

Find out most/least used testapi functions to decide about where to extend/cleanup size:M

Added by okurz 8 months ago. Updated 3 months ago.

Status:
Resolved
Priority:
Low
Assignee:
Category:
Feature requests
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:

Description

Motivation

#130369 brought me to the idea that functions like "login" or more generalized "assert_and_click_and_assert" from https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/146/files#diff-1fb9ad4446612c618556273eb389bd68d1643725f366380e154c8619da1b43eeR8 maybe could be directly brought into the testapi of os-autoinst so not needing wheels necessarily. But the testapi is already big and maybe we even have functions that are candidates for removal. So let's look into test distributions that we can find on github, at least os-autoinst-distri-openSUSE and see which testapi functions are used the most and which the least.

Acceptance criteria

  • AC1: We have a list of absolute uses of os-autoinst testapi functions in test distribution(s), at least os-autoinst-distri-opensuse

Suggestions

  • grep testapi for testapi function definitions, e.g. start with grep "sub [^_].*" testapi.pm and compile a list of all functions. For each function lookup on github or local in os-autoinst-distri-opensuse and count occurences
  • Put the list sorted by occurences of function calls in this ticket
  • Present the list to the tools team and initiate a discussion

Files

check.sh (282 Bytes) check.sh script with ripgrep :D osukup, 2024-01-17 11:00
fun.txt (1023 Bytes) fun.txt list of functions osukup, 2024-01-17 11:00

Related issues 2 (1 open1 closed)

Copied from openQA Project - action #130369: [spike][timeboxed:20h] Reduce duplication of openQA-in-openQA tests in os-autoinst-distri-opensuse and os-autoinst-distri-openQA size:SResolvedtinita2023-06-01

Actions
Copied to openQA Project - action #156859: Improve most used testapi functionsNew

Actions
Actions #1

Updated by okurz 8 months ago

  • Copied from action #130369: [spike][timeboxed:20h] Reduce duplication of openQA-in-openQA tests in os-autoinst-distri-opensuse and os-autoinst-distri-openQA size:S added
Actions #2

Updated by tinita 7 months ago

  • Subject changed from Find out most/least used testapi functions to decide about where to extend/cleanup to Find out most/least used testapi functions to decide about where to extend/cleanup size:M
  • Status changed from New to Workable
Actions #3

Updated by osukup 5 months ago

  • Assignee set to osukup
Actions #4

Updated by osukup 5 months ago

  • Status changed from Workable to In Progress

found test repositories for almalinux, fedora and debian ..

Actions #5

Updated by osukup 5 months ago

osukup wrote in #note-5:

Table of testapi functions in test distributions. Used are almalinux, fedora, Debian, opensuse, openQA and OBS test distributions with all together and separate openSUSE:

function global opensuse rest comment
script_run 12995 10798 2197
assert_script_run 9746 8121 1625
get_var 6616 4686 1930
send_key 5813 4715 1098
assert_screen 3810 2513 1297
script_output 3023 2825 198
record_info 2639 2628 11
assert_and_click 2626 805 1821
wait_still_screen 1741 925 816
check_var 1547 1455 92
select_console 1105 1092 13
validate_script_output 1050 909 141
type_string 989 673 316
enter_cmd 897 866 31
get_required_var 886 869 17
upload_logs 814 560 254
wait_screen_change 806 686 120
save_screenshot 770 735 35
check_screen 714 314 400
match_has_tag 704 543 161
set_var 684 644 40
send_key_until_needlematch 491 418 73
record_soft_failure 460 357 103
wait_serial 364 330 34
x11_start_program 324 324 0 distribution implemenation
diag 279 247 32
data_url 265 265 0
type_password 168 143 25
click_lastmatch 134 13 121
autoinst_url 118 118 0
mouse_set 97 30 67
mouse_hide 95 46 49
is_serial_terminal 87 57 30
reset_consoles 70 70 0
become_root 69 62 7 distribution reimplemetation
console 51 51 0
ensure_installed 43 43 0 distribution implemenation
parse_extra_log 38 38 0
upload_asset 36 28 8
assert_and_dclick 31 29 2
script_sudo 30 17 13
assert_shutdown 27 20 7
hashed_string 26 12 14
check_var_array 25 19 6 debian analyse.pm
save_tmp_file 24 24 0
parse_junit_log 21 21 0
mouse_click 18 10 8
get_var_array 16 11 5 debian analyse.pl
hold_key 16 13 3
release_key 15 12 3
current_console 13 13 0
get_test_data 9 9 0
assert_script_sudo 7 3 4 distribution implemenation ... practically nver used
start_audiocapture 7 4 3
assert_screen_change 7 6 1
force_soft_failure 6 6 0
power 5 1 4
check_recorded_sound 4 4 0 one in comment
check_shutdown 4 3 1
eject_cd 4 1 3 somewhere is comment ... command is broken
assert_recorded_sound 2 1 1
resume_vm 2 2 0
mouse_tclick 1 1 0 wow
assert_still_screen 1 1 0 called as testapi::assert_still_screen
mouse_dclick 0 0 0
background_script_run 0 0 0
switch_network 0 0 0 fedora has sub in gnome-panel ...
save_memory_dump 0 0 0
freeze_vm 0 0 0
save_storage 0 0 0
mouse_drag 0 0 0
Actions #6

Updated by osukup 5 months ago

opensuse: git@github.com:os-autoinst/os-autoinst-distri-opensuse.git
obs: https://github.com/os-autoinst/os-autoinst-distri-obs.git
openqa: git@github.com:os-autoinst/os-autoinst-distri-openQA.git
debian: https://salsa.debian.org/qa/openqa/openqa-tests-debian.git
fedora: https://pagure.io/fedora-qa/os-autoinst-distri-fedora.git
almalinux: git@github.com:AlmaLinux/os-autoinst-distri-almalinux.git

Actions #7

Updated by livdywan 5 months ago

We discussed these least-used functions together in the team:

  • freeze_vm/resume_vm:
    • freezing is seemingly never used?
    • does this work? what's the use case? used for snapshots/backup of the disk?
    • merge this with save_storage/save_memory_dump?
    • is it feasible to use in the debugging case?
  • mouse_drag
    • maybe dragging is implemented as press/down/up/click instead? likely we would have some tests dragging things with a mouse?
    • It actually has been used in the past, funnily as a workaround: a687332d0ad948bf36375711c5850d5d5f75d8f0 + # On x11, workaround bsc#1191112 by mouse click or drag the dialog. + if (($setup_console =~ /x11/) && (check_var('RMT_TEST', 'rmt_chinese'))) { + record_soft_failure('bsc#1191112 - When navigating through YaST module screens the next screen appears, but its content is not loaded'); + wait_still_screen(10, 15); + mouse_drag(startx => 480, starty => 50, endx => 485, endy => 50, button => 'left'); + }
  • switch_network:
    • fedora has an implementation of this under its own distri
    • maybe this was lost and never upstreamed?
  • background_script_run:
    • why is this never used? maybe we don't need it? script_run with & or nohup might be enough?
    • it's distrio-specific
    • used in ltp tests
  • asset/check_recorded_sound/start_audiocapture:
    • is it worth adding another case to confirm if it's easy enough to use?
    • it's used in multiple distris... but why not more in general?
  • mouse_dclick:
    • used internally, arguably redundant with asset_and_click and click_last_match exposing double clicking
  • mouse_tclick:
    • used by gedit, trivial code but not made available by any other means and not something one would want to copy (a lot of clicks and sleeps)
    • maybe other code tries to triple-click but doesn't use this api?
  • eject_cd:
    • is this usable? is this broken? should be worth checking
  • current_console:
    • only used in opensuse
  • get_test_data
  • assert_script_sudo:
    • distribution implementation practically never used
    • this relies on script_sudo which relies on the distri implementation
    • fedora/alma have an implementation but don't use it: os-autoinst-distri-almalinux/lib/fedoradistribution.pm:18
  • assert_screen_change
  • force_soft_failure
  • power

Action points

  • Provide a list of repos used as a reference to search the test distributions we consider relevant/covered here @osukup
  • Provide a reproducible command line to lookup the uses of testapi functions (rg/git grep) so we can double-check the results, or see if we made some progress down the line @osukup
  • After this we can distribute the discussed least-used commands amongst the team to evaluate e.g. spend a couple hours at most to come up with a suggestion (whole team)

Updated by osukup 5 months ago

point 2 of actions

The list of repositories is in the comment https://progress.opensuse.org/issues/138032#note-6

Actions #9

Updated by osukup 5 months ago

  • Status changed from In Progress to Workable
  • Assignee deleted (osukup)
Actions #10

Updated by okurz 4 months ago

  • Target version changed from Tools - Next to Ready
Actions #11

Updated by okurz 4 months ago

  • Priority changed from Normal to Low
Actions #12

Updated by jbaier_cz 4 months ago

  • Assignee set to jbaier_cz
Actions #13

Updated by jbaier_cz 3 months ago

  • Status changed from Workable to In Progress
Actions #14

Updated by jbaier_cz 3 months ago

  • Status changed from In Progress to Resolved

Script added to the repo in https://github.com/os-autoinst/scripts/pull/298

With that I believe this is done.

Actions #15

Updated by okurz 3 months ago

Actions

Also available in: Atom PDF