QA » openQA Project

Observations¶

os-autoinst $ prove -v -I. t/14-isotovideo.t 
t/14-isotovideo.t .. 
# Subtest: get the version number
    ok 1 - version printed
    ok 2 - no state file was written
    1..2
ok 1 - get the version number
# Subtest: color output can be configured via the command-line
    # Starting isotovideo with: /usr/bin/perl /home/okurz/local/os-autoinst/os-autoinst/isotovideo --workdir /tmp/14-isotovideo.t-iG1X/pool -d backend=null --color=yes casedir=/home/okurz/local/os-autoinst/os-autoinst/t/data/tests schedule=module1,bar/module2 _exit_after_schedule=1
    ok 1 - isotovideo exit code
    ok 2 - logs use colors when requested
    # Starting isotovideo with: /usr/bin/perl /home/okurz/local/os-autoinst/os-autoinst/isotovideo --workdir /tmp/14-isotovideo.t-iG1X/pool -d backend=null --color=no casedir=/home/okurz/local/os-autoinst/os-autoinst/t/data/tests schedule=module1,bar/module2 _exit_after_schedule=1
    ok 3 - isotovideo exit code
    ok 4 - no colors in logs
    1..4
ok 2 - color output can be configured via the command-line
# Subtest: standalone isotovideo without any parameters
    # Starting isotovideo with: /usr/bin/perl /home/okurz/local/os-autoinst/os-autoinst/isotovideo --workdir /tmp/14-isotovideo.t-iG1X/pool -d backend=null
    ok 1 - isotovideo exit code
    ok 2 - initialization error printed to user
    1..2
ok 3 - standalone isotovideo without any parameters
# Subtest: standalone isotovideo without vars.json file and only command line parameters
    # Starting isotovideo with: /usr/bin/perl /home/okurz/local/os-autoinst/os-autoinst/isotovideo --workdir /tmp/14-isotovideo.t-iG1X/pool -d backend=null casedir=/home/okurz/local/os-autoinst/os-autoinst/t/data/tests schedule=module1,bar/module2 _exit_after_schedule=1
    ok 1 - isotovideo exit code
    ok 2 - requested module module1 scheduled
    ok 3 - requested module bar/module2 scheduled
    1..3
ok 4 - standalone isotovideo without vars.json file and only command line parameters
# Subtest: standard tests based on simple vars.json file
    # Starting isotovideo with: /usr/bin/perl /home/okurz/local/os-autoinst/os-autoinst/isotovideo --workdir /tmp/14-isotovideo.t-iG1X/pool -d backend=null
    ok 1 - isotovideo exit code
    ok 2 - shutdown scheduled
    1..2
ok 5 - standard tests based on simple vars.json file
# Subtest: isotovideo with custom git repo parameters specified
    ok 1 - initialized test repo
    # Starting isotovideo with: /usr/bin/perl /home/okurz/local/os-autoinst/os-autoinst/isotovideo --workdir /tmp/14-isotovideo.t-iG1X/pool -d backend=null casedir=file:///tmp/14-isotovideo.t-iG1X/pool/repo.git#foo needles_dir=/home/okurz/local/os-autoinst/os-autoinst/t/data _exit_after_schedule=1
Bailout called.  Further testing stopped:  test 't/14-isotovideo.t' exceeds runtime limit of '20' seconds
Bail out!  test 't/14-isotovideo.t' exceeds runtime limit of '20' seconds
FAILED--Further testing stopped: test 't/14-isotovideo.t' exceeds runtime limit of '20' seconds

Acceptance criteria¶

• AC1: t/14-isotvideo.t passes prove -I. t/14-isotovideo.t consistently without running into timeout locally

Suggestions¶

• Use https://github.com/okurz/scripts/blob/master/count_fail_ratio or retry to find out failure statistics
• Bisect where a performance regression was introduced
• Fix the overly long runtime of the test
• As applicable refactor further to improve runtime
• Adjust timeout value as applicable