action #135521
Updated by livdywan 8 months ago
## Observations
```
os-autoinst $ prove -v -I. t/14-isotovideo.t
t/14-isotovideo.t ..
# Subtest: get the version number
ok 1 - version printed
ok 2 - no state file was written
1..2
ok 1 - get the version number
# Subtest: color output can be configured via the command-line
# Starting isotovideo with: /usr/bin/perl /home/okurz/local/os-autoinst/os-autoinst/isotovideo --workdir /tmp/14-isotovideo.t-iG1X/pool -d backend=null --color=yes casedir=/home/okurz/local/os-autoinst/os-autoinst/t/data/tests schedule=module1,bar/module2 _exit_after_schedule=1
ok 1 - isotovideo exit code
ok 2 - logs use colors when requested
# Starting isotovideo with: /usr/bin/perl /home/okurz/local/os-autoinst/os-autoinst/isotovideo --workdir /tmp/14-isotovideo.t-iG1X/pool -d backend=null --color=no casedir=/home/okurz/local/os-autoinst/os-autoinst/t/data/tests schedule=module1,bar/module2 _exit_after_schedule=1
ok 3 - isotovideo exit code
ok 4 - no colors in logs
1..4
ok 2 - color output can be configured via the command-line
# Subtest: standalone isotovideo without any parameters
# Starting isotovideo with: /usr/bin/perl /home/okurz/local/os-autoinst/os-autoinst/isotovideo --workdir /tmp/14-isotovideo.t-iG1X/pool -d backend=null
ok 1 - isotovideo exit code
ok 2 - initialization error printed to user
1..2
ok 3 - standalone isotovideo without any parameters
# Subtest: standalone isotovideo without vars.json file and only command line parameters
# Starting isotovideo with: /usr/bin/perl /home/okurz/local/os-autoinst/os-autoinst/isotovideo --workdir /tmp/14-isotovideo.t-iG1X/pool -d backend=null casedir=/home/okurz/local/os-autoinst/os-autoinst/t/data/tests schedule=module1,bar/module2 _exit_after_schedule=1
ok 1 - isotovideo exit code
ok 2 - requested module module1 scheduled
ok 3 - requested module bar/module2 scheduled
1..3
ok 4 - standalone isotovideo without vars.json file and only command line parameters
# Subtest: standard tests based on simple vars.json file
# Starting isotovideo with: /usr/bin/perl /home/okurz/local/os-autoinst/os-autoinst/isotovideo --workdir /tmp/14-isotovideo.t-iG1X/pool -d backend=null
ok 1 - isotovideo exit code
ok 2 - shutdown scheduled
1..2
ok 5 - standard tests based on simple vars.json file
# Subtest: isotovideo with custom git repo parameters specified
ok 1 - initialized test repo
# Starting isotovideo with: /usr/bin/perl /home/okurz/local/os-autoinst/os-autoinst/isotovideo --workdir /tmp/14-isotovideo.t-iG1X/pool -d backend=null casedir=file:///tmp/14-isotovideo.t-iG1X/pool/repo.git#foo needles_dir=/home/okurz/local/os-autoinst/os-autoinst/t/data _exit_after_schedule=1
Bailout called. Further testing stopped: test 't/14-isotovideo.t' exceeds runtime limit of '20' seconds
Bail out! test 't/14-isotovideo.t' exceeds runtime limit of '20' seconds
FAILED--Further testing stopped: test 't/14-isotovideo.t' exceeds runtime limit of '20' seconds
```
## Acceptance criteria
* **AC1:** t/14-isotvideo.t passes `prove -I. t/14-isotovideo.t` consistently without running into timeout locally
## Suggestions
* Use https://github.com/okurz/scripts/blob/master/count_fail_ratio or `retry` to find out failure statistics
* Bisect where a performance regression was introduced
* Fix the overly long runtime of the test
* As applicable refactor further to improve runtime
* Adjust timeout value as applicable