QA » openQA Project » openQA Tests

Problem¶

The qaset utility (package qa_testset_automation) allows for suspending and resuming execution, but there appears to be no official interface for monitoring what happens.

Context¶

poo#46463, openQA pull request 8539

Details¶

The PR8539 openQA code, file

/var/lib/openqa/share/tests/sle/tests/qa_automation/kernel_multipath.pm

actually needs the possibility of suspending and resuming qaset-driven tests.
It comments on the challenge presented by the desire to properly monitor this as follows:

# FIXME: The status queries for qaset (pkg qa_testset_automation) in
# the subsequent two subs are rather awkward "inofficial" workarounds:
# the current qaset API does not seem to offer such queries.
#
# A request to implement API commands like, e.g.,
#
#     /usr/share/qa/qaset/qaset status          # clean/running/stopped/done
#     /usr/share/qa/qaset/qaset waitstop        # wait until status is "stopped"
#     /usr/share/qa/qaset/qaset waitdone        # wait until status is "done"
#
# is under way.
#
sub qaset_waitdone {
# qaset has been observed to create file /var/log/qaset/control/DONE
# as soon as
# -  no testsuite run is in progress anymore _and_
# -  there is no testsuite waiting for execution anymore
#    (file /var/log/qaset/control/NEXT_RUN)
#
my ($timeout) = @_;
assert_script_run(
"until [ -f /var/log/qaset/control/DONE ]; do sleep 5; done",
timeout      => $timeout,
fail_message => "qaset failed to announce overall completion within $timeout s");
}
# Invocation: qaset_waitstop(testname => $testname, timeout => $timeout [,prewait => $prewait]);
#
sub qaset_waitstop {
#
# FIXME: This sub is particularly awkward due to the need for $prewait.
# Observed: the following qaset behavior:
#
# qaset creates file /var/log/qaset/control/SYSTEM_DIRTY as soon
# as no testsuite run is in progress anymore (In contrast to
# DONE further testsuites-to-execute may be left waiting, like
# after a qaset stop).
#
# If some testsuite is left waiting for execution (file NEXT_RUN),
# starting another qaset run will delete an existing file SYSTEM_DIRTY:
# WARNING: but it has been observed to do so only after a delay of a few
# (FIXME: how many?) seconds. It is thus unsafe to start polling for
# SYSTEM_DIRTY immediately after such a restart. Hence $prewait.
#
my %args     = @_;
my $testname = $args{testname};
my $timeout  = $args{timeout};
my $prewait  = $args{prewait} // 15;    # 15 is guessed to be enough :-/
sleep $prewait if $prewait > 0;
assert_script_run(
"until [ -f /var/log/qaset/control/SYSTEM_DIRTY ]; do sleep 5; done",
timeout      => $timeout,
fail_message => "$testname: qaset run failed to complete in $timeout s");
}

The main point of this request is to obtain a reliable official way to carry out such monitoring

Hope is that for somebody familiar with qaset's process management the implementation of additional subcommands like the ones in the comment might be fairly straightforward.