Project

General

Profile

action #92497

Perl test execution in os-autoinst runs succesfully but then fails with error

Added by VANASTASIADIS 5 months ago. Updated 5 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Concrete Bugs
Target version:
Start date:
2021-05-11
Due date:
2021-06-02
% Done:

0%

Estimated time:
Difficulty:

Description

Observation

When running a single perl test (or a series of tests), following the steps described in https://github.com/os-autoinst/os-autoinst#conducting-the-build, the tests run as expected, but afterwards encounters the following error and exits:

Cannot detect source of '17-basetest.t'! at /usr/lib/perl5/5.26.1/TAP/Parser/IteratorFactory.pm line 261.
        TAP::Parser::IteratorFactory::detect_source(TAP::Parser::IteratorFactory=HASH(0x5563f4c92c20), TAP::Parser::Source=HASH(0x5563f4c92ab8)) called at /usr/lib/perl5/5.26.1/TAP/Parser/IteratorFactory.pm line 211
        TAP::Parser::IteratorFactory::make_iterator(TAP::Parser::IteratorFactory=HASH(0x5563f4c92c20), TAP::Parser::Source=HASH(0x5563f4c92ab8)) called at /usr/lib/perl5/5.26.1/TAP/Parser.pm line 472
        TAP::Parser::_initialize(TAP::Parser=HASH(0x5563f4c92818), HASH(0x5563f4a1dff0)) called at /usr/lib/perl5/5.26.1/TAP/Object.pm line 55
        TAP::Object::new("TAP::Parser", HASH(0x5563f4a1dff0)) called at /usr/lib/perl5/5.26.1/TAP/Object.pm line 130
        TAP::Object::_construct(TAP::Harness=HASH(0x5563f445fe50), "TAP::Parser", HASH(0x5563f4a1dff0)) called at /usr/lib/perl5/5.26.1/TAP/Harness.pm line 852
        TAP::Harness::make_parser(TAP::Harness=HASH(0x5563f445fe50), TAP::Parser::Scheduler::Job=HASH(0x5563f4c62df8)) called at /usr/lib/perl5/5.26.1/TAP/Harness.pm line 651
        TAP::Harness::_aggregate_single(TAP::Harness=HASH(0x5563f445fe50), TAP::Parser::Aggregator=HASH(0x5563f4900670), TAP::Parser::Scheduler=HASH(0x5563f4c62d98)) called at /usr/lib/perl5/5.26.1/TAP/Harness.pm line 743
        TAP::Harness::aggregate_tests(TAP::Harness=HASH(0x5563f445fe50), TAP::Parser::Aggregator=HASH(0x5563f4900670), "17-basetest.t") called at /usr/lib/perl5/5.26.1/TAP/Harness.pm line 558
        TAP::Harness::__ANON__() called at /usr/lib/perl5/5.26.1/TAP/Harness.pm line 571
        TAP::Harness::runtests(TAP::Harness=HASH(0x5563f445fe50), "17-basetest.t") called at /usr/lib/perl5/5.26.1/App/Prove.pm line 546
        App::Prove::_runtests(App::Prove=HASH(0x5563f44524e8), HASH(0x5563f48ee168), "17-basetest.t") called at /usr/lib/perl5/5.26.1/App/Prove.pm line 504
        App::Prove::run(App::Prove=HASH(0x5563f44524e8)) called at /usr/bin/prove line 13
make[3]: *** [CMakeFiles/test-perl-testsuite.dir/build.make:74: CMakeFiles/test-perl-testsuite] Error 2
make[2]: *** [CMakeFiles/Makefile2:540: CMakeFiles/test-perl-testsuite.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:547: CMakeFiles/test-perl-testsuite.dir/rule] Error 2
make: *** [Makefile:348: test-perl-testsuite] Error 2

Steps to reproduce

  • Navigate to your build folder for os-autoinst (i.e. where you ran cmake)
  • run make test-perl-testsuite TESTS="17-basetest.t, replacing 17-basetest.t with any single test or group of tests contained in the t/ directory

Problem

H1. The problem seems to persist regardless of the test the user tries to run
H2. In the case of multiple tests, after all tests have completed, the error refers to the first test listed by the user in the make test-perl-testsuite TESTS="test1, test2 ... testN" command and exits

Suggestion

Since this problem is unlikely to have existed for long without anyone noticing, it is probably a regression. Locating a commit that caused this behaviour could be a good start.

Workaround

Running the tests will still produce all expected results, so no special action is needed

History

#1 Updated by VANASTASIADIS 5 months ago

  • Description updated (diff)

#2 Updated by cdywan 5 months ago

  • Description updated (diff)
  • Status changed from New to Workable

As discussed in Rocket.Chat, I can reproduce the same with the latest master and different tests. So this must be a regression.

#3 Updated by VANASTASIADIS 5 months ago

  • Assignee set to VANASTASIADIS

#4 Updated by VANASTASIADIS 5 months ago

  • Status changed from Workable to In Progress

#5 Updated by VANASTASIADIS 5 months ago

This error was introduced with commit https://github.com/os-autoinst/os-autoinst/commit/090ad748b226d6322cea8e5029c3dde66f7675da . It runs all tests in both t and xt directories, and the crash occurs when a test that is located in the t dir is executed while in the xt dir and vice versa.

My solution: split the testname arguments in 2 strings, one for tests in the t dir and on for thests in the xt dir, and run the appropriate tests in their respective dirs.

Since this commit dates to dec 2020, I'm surprised it went unnoticed for so long. The order of execution was t and then xt, so trying to execute any specific xt test with make test-perl-testsuite TESTS=<testnames> should result in an error.

#6 Updated by openqa_review 5 months ago

  • Due date set to 2021-06-02

Setting due date based on mean cycle time of SUSE QE Tools

#7 Updated by okurz 5 months ago

  • Status changed from In Progress to Resolved

https://github.com/os-autoinst/os-autoinst/pull/1674 merged. I can confirm that make test-perl-testsuite TESTS=t/01-test_needle.t works now fine. Good job!

#8 Updated by VANASTASIADIS 5 months ago

Thank you okurz! Glad to contribute.

Also available in: Atom PDF