Project

General

Profile

Actions
Copy link

action #133112

open

Switch unit tests to Test2-Suite

Added by tinita 10 months ago. Updated about 2 months ago.

Status:
New
Priority:
Low
Assignee:
-
Category:
Feature requests
Target version:
QA - future
Start date:
2023-07-20
Due date:
% Done:

0%

Estimated time:

Description

Motivation

We are using a lot of different test modules in different distributions.

  • Test::Most
  • Test::Warnings
  • Test::Fatal
  • Test::MockModule
  • Test::MockObject
  • OpenQA::Test::TimeLimit

The new Test2 suite comes with a lot of builtin stuff:
https://metacpan.org/pod/Test2::V0

The yath tool (alternative to prove) has a builtin timeout functionality: https://metacpan.org/pod/App::Yath#HARNESS-NO-TIMEOUT

https://metacpan.org/pod/Test2::Mock has some interesting features like call tracing. 18-backend-qemu.t is an example where we would get the tracking for free instead of creating our own hash. in another example we use the %called hash even differently, so this is inconsistent.

https://metacpan.org/pod/Test2::Tools::AsyncSubtest / https://metacpan.org/pod/Test2::AsyncSubtest makes it easy to run tests asynchronously or in a fork.

Also the is function already handles deep comparison and even lists all top level mismatches instead of just reporting the first mismatch and then stop.

We might want to try out if we can switch to Test2 in one of our tests and see if it makes sense.
No need to switch all tests at once.

Only downside: The version in Leap 15.5 is quite old (2017!), so we would have to link it in devel:openQA

Actions
Copy link
 #1

Updated by tinita 10 months ago

  • Description updated (diff)
Actions
Copy link
 #2

Updated by tinita 10 months ago

  • Description updated (diff)
Actions
Copy link
 #3

Updated by okurz 10 months ago

  • Status changed from New to Blocked
  • Assignee set to okurz
  • Target version set to Ready

tinita wrote:

Only downside: The version in Leap 15.5 is quite old (2017!), so we would have to link it in devel:openQA

Why not just try with a maintenance update before we want to use it. I did osc mr -m "Planned future dependency for openQA development" openSUSE:Factory perl-Test2-Suite openSUSE:Leap:15.5 -> https://build.opensuse.org/request/show/1099719

Actions
Copy link
 #4

Updated by tinita 10 months ago

  • Description updated (diff)
Actions
Copy link
 #5

Updated by tinita 10 months ago

  • Description updated (diff)
Actions
Copy link
 #6

Updated by tinita 10 months ago

okurz wrote:

I did osc mr -m "Planned future dependency for openQA development" openSUSE:Factory perl-Test2-Suite openSUSE:Leap:15.5 -> https://build.opensuse.org/request/show/1099719

It was declined with:

comes from SUSE:SLE-15:Update, so needs to upgrade on the SLE sider

Actions
Copy link
 #8

Updated by tinita 9 months ago

Actions
Copy link
 #10

Updated by okurz 9 months ago

tracking outside of backlog. We do not necessarily need to care right now if we have Test2-Suite or not

Actions
Copy link
 #11

Updated by tinita 9 months ago

  • Target version changed from Ready to future

okurz wrote:

tracking outside of backlog. We do not necessarily need to care right now if we have Test2-Suite or not

I guess you wanted to set that to future?

Actions
Copy link
 #12

Updated by okurz about 2 months ago

  • Status changed from Blocked to New
  • Assignee deleted (okurz)

https://build.opensuse.org/request/show/1100039 was revoked but we had https://build.suse.de/request/show/315797 within IBS to have perl-Test2-Suite in SLE15-SP6 which was now accepted. This is for https://jira.suse.com/browse/PED-7021 which I created for #127028. In podman run --rm -it --pull=newer registry.opensuse.org/opensuse/leap:15.5 I can still see the old version perl-Test2-Suite-0.000097-1.17 though. registry.opensuse.org/opensuse/leap:15.6 does not yet exist. I guess we would need to wait some months until we can fully use Leap 15.6

Actions
Copy link

Also available in: Atom PDF