QA » openQA Project » openQA Tests » Containers

Currently, BCI tests are executed by the environments defined by the variable BCI_TEST_ENVS.
The current code is running all the commands in a loop

    for my $env (split(/,/, $test_envs)) {
        record_info($env);
        my $ret = script_run("timeout $bci_timeout tox -e $env $cmd_options", timeout => ($bci_timeout + 3));
        if ($ret == 124) {
            # man timeout: If  the command times out, and --preserve-status is not set, then exit with status 124.
            record_soft_failure("The command <tox -e $env $cmd_options> timed out.");
            $error_count += 1;
        } elsif ($ret != 0) {
            record_soft_failure("The command <tox -e $env $cmd_options> failed.");
            $error_count += 1;
        } else {
            record_info('PASSED');
        }
    }

The current execution time is around 2 hours.

The problem of this is that openQA stays at 100% during most part of that execution since the progress bar looks at the number of modules executed.

We could follow the approach of LTP and other tests that use loadtest during runtime. This way, we make openQA treat every test as a new module, so the progress bar is more realistic.