QA » openQA Project

Observation¶

At the time of writing openQA "t" tests have some "unhandled test output":

[11:45:14] t/23-amqp.t ............................................... 3/? [2021-12-07 11:45:21.67648] [459] [debug] [pid:459] Sending AMQP event: some.topic
[2021-12-07 11:45:21.67689] [459] [debug] [pid:459] AMQP URL: amqp://guest:guest@localhost:5672/?exchange=pubsub&cacertfile=%2Fsome%2Fcacert.pem&certfile=%2Fsome%2Fcert.pem&keyfile=%2Fsome%2Fkey.pem
[11:45:14] t/23-amqp.t ............................................... ok    12621 ms ( 0.02 usr  0.00 sys + 11.91 cusr  0.44 csys = 12.37 CPU)
…
[11:45:39] t/24-worker-jobs.t ........................................ 20/? [WARN] Job 1 stopped because it exceeded MAX_JOB_TIME
[WARN] Found poolCKgM/base_state.json but failed to parse the JSON: 'false' expected, at character offset 0 (before "foo boo") at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/JSON.pm line 31.

[11:45:39] t/24-worker-jobs.t ........................................ ok    11737 ms ( 0.08 usr  0.01 sys + 11.24 cusr  0.47 csys = 11.80 CPU)
[11:45:50] t/24-worker-overall.t ..................................... The signatures feature is experimental at t/24-worker-overall.t line 739.
[11:45:50] t/24-worker-overall.t ..................................... ok     4910 ms ( 0.05
…
[11:49:49] t/35-script_clone_job.t ................................... The signatures feature is experimental at t/35-script_clone_job.t line 31.
[11:49:49] t/35-script_clone_job.t ................................... ok     2555 ms ( 0.02 usr  0.00 sys +  2.34 cusr  0.17 csys =  2.53 CPU)

Acceptance criteria¶

• AC1: No unhandled output from succeeding tests present

Suggestions¶

• Reproduce individual test outputs locally, cover expected strings with Test::Output as we do in many other test modules
• Add opt-in variable to make tests fail on unexpected output
• Use Test::Warnings