Project

General

Profile

Actions
Copy link

action #181358

closed

coordination #127031: [saga][epic] openQA for SUSE customers

coordination #165393: [epic] Improved code coverage in openQA

[openQA][CI] Prevent unhandled output in prove test calls with automatic checks in os-autoinst

Added by gpuliti about 1 month ago. Updated 6 days ago.

Status:
Resolved
Priority:
Low
Assignee:
gpuliti
Category:
Regressions/Crashes
Target version:
Ready
Start date:
2025-03-01
Due date:
% Done:

0%

Estimated time:
Tags:
openQA

Description

Motivation

https://app.circleci.com/pipelines/gh/os-autoinst/openQA/16015/workflows/93b25d9c-129a-40d0-8dec-82b2390fb56b/jobs/154230 shows a problem of unhandled output. We want to have a clean TAP output with all output from tested application code to be captured:

[23:50:36] t/05-scheduler-restart-and-duplicate.t .................... 22/?     # {
    #   '100006' => 100012,
    #   '100008' => 100013,
    #   '100009' => 100014,
    #   '100010' => 100015
    # }
[23:50:36] t/05-scheduler-restart-and-duplicate.t .................... ok    14366 ms ( 0.04 usr  0.01 sys + 13.61 cusr  0.37 csys = 14.03 CPU)
[

Acceptance criteria

  • AC1: Unhandled output from any tests called with prove is prevented by automatic checks in os-autoinst
  • AC2: Output of the tests should still be immediate and not printed with delay

Suggestions

Related issues 3 (2 open — 1 closed)

Related to openQA Project (public) - action #182402: [ci] Warning in test output about Devel::Cover: It was not rebuilt for a perl patch version update size:SFeedbacktinita2025-05-14

Actions
Copied from openQA Project (public) - action #178117: [openQA][CI] Prevent unhandled output in prove test calls with automatic checks size:SResolvedgpuliti2025-03-01

Actions
Copied to openQA Project (public) - action #182111: [openQA][CI] Prevent unhandled output in prove test calls with automatic checks in any other projectsNew2025-03-01

Actions
Actions
Copy link
 #1

Updated by gpuliti about 1 month ago

  • Copied from action #178117: [openQA][CI] Prevent unhandled output in prove test calls with automatic checks size:S added
Actions
Copy link
 #2

Updated by gpuliti 19 days ago

  • Assignee set to gpuliti
Actions
Copy link
 #3

Updated by gpuliti 19 days ago

  • Status changed from New to In Progress
Actions
Copy link
 #4

Updated by okurz 16 days ago

  • Copied to action #182111: [openQA][CI] Prevent unhandled output in prove test calls with automatic checks in any other projects added
Actions
Copy link
 #5

Updated by okurz 16 days ago

  • Subject changed from [openQA][CI] Prevent unhandled output in prove test calls with automatic checks in other project to [openQA][CI] Prevent unhandled output in prove test calls with automatic checks in os-autoinst
  • Description updated (diff)

Split out "other projects" ticket and focussing this on os-autoinst only

Actions
Copy link
 #6

Updated by gpuliti 13 days ago ยท Edited

  • Status changed from In Progress to Feedback

I've open a pr https://github.com/os-autoinst/os-autoinst/pull/2701 to run the tests directly to the ci.

I've created the tools/prove_wrapper script by using the one build for os-autoinst/openqa repository.

I can't understand how the PROVE_PATH is handled by find_programs in the cmake/test-targets.cmake file. For test pourpose I've set it with tools/prove_wrapper and it work as expected by using the wrapper, but if I set it using -DPROVE_PATH=\"/tools/prove_wrapper\" on the tools/container_run_ci the variable overwriting is ignored and the default one, the unwrapped, is being use.

I've left some debug output in order to understand where the problem could be.

  • print working directory for tools/container_run_ci:
Run tools/container_run_ci
...
PWD: /home/runner/work/os-autoinst/os-autoinst
...
  • pwd and arguments of tools/invoke-tests:
3: PWD: /opt
3: ARGS[1]: --prove-tool, ARGS[2]:tools/prove_wrapper

Now that I'm writing I'm realizing that maybe the problem is the invoke-tests pwd.

I've apply the pwd as prefix to the prove_wrapper script, let see the results... (edit: result gives error)

Actions
Copy link
 #7

Updated by gpuliti 12 days ago

  • Status changed from Feedback to In Progress
Actions
Copy link
 #8

Updated by gpuliti 12 days ago

Thanks to @mkittler I'm back to working on this!

Previously I was adding the PROVE_PATH directly to the last cmake call, but the configuration is been build with the first cmake call. I've moved the PROVE_PATH to the first one:

cmake -G Ninja -DPROVE_PATH=\"$PWD/tools/prove_wrapper\" -DCMAKE_BUILD_TYPE=Release -S . -B \"$build_dir\"
cmake --build \"$build_dir\" --verbose --target check-pkg-build
cmake --build \"$build_dir\" --verbose --target $target
Actions
Copy link
 #9

Updated by gpuliti 12 days ago

  • Description updated (diff)
Actions
Copy link
 #10

Updated by gpuliti 11 days ago

  • Status changed from In Progress to Workable
Actions
Copy link
 #11

Updated by gpuliti 11 days ago

  • Status changed from Workable to Feedback
Actions
Copy link
 #12

Updated by gpuliti 11 days ago

  • Related to action #182402: [ci] Warning in test output about Devel::Cover: It was not rebuilt for a perl patch version update size:S added
Actions
Copy link
 #13

Updated by gpuliti 10 days ago

  • Status changed from Feedback to In Progress
Actions
Copy link
 #14

Updated by gpuliti 10 days ago

  • Status changed from In Progress to Feedback
Actions
Copy link
 #15

Updated by gpuliti 10 days ago

Moving prove_wrapper script to https://github.com/os-autoinst/os-autoinst-common/pull/57 to avoid duplication.

Once the pr is merged I'll update the PRs: https://github.com/os-autoinst/os-autoinst/pull/2701 and the followup new https://github.com/os-autoinst/openQA/pull/6459

Actions
Copy link
 #16

Updated by gpuliti 9 days ago

Actions
Copy link
 #17

Updated by gpuliti 6 days ago

  • Status changed from Feedback to Resolved

All merged, Resolve!

Actions
Copy link

Also available in: Atom PDF