action #158688: Display only failures in bats-user-local_tap - Containers and images - openSUSE Project Management Tool

Actions

Copy link

action #158688

closed

Display only failures in bats-user-local_tap

Added by ph03nix 7 months ago. Updated 7 days ago.

Status:

Rejected

Priority:

Normal

Assignee:

Target version:

Start date:

2024-04-09

Due date:

% Done:

Estimated time:

Tags:

containers

Description

When the podman test "bats-user-local_tap" fails, the "Show only failures" function in openQA still displays all subtests. It would be great, if when using "Show only failures" only the failing tests are being displayed, in the same way as for LTP.

Example: The failures in bats-user-local_tap - when setting "Show only failures" I still get the full list of test failures:

Acceptance criteria¶

Ensure the "Show only failures" function of openQA works with the subresults of the "bats-user-local_tap" test

Files

Download all files

Screenshot 2024-04-09 at 13-21-06 openQA sle-15-SP4-Server-DVD-Updates-aarch64-Build20240408-1-podman_testsuite@aarch64-virtio test results.png (109 KB) Screenshot 2024-04-09 at 13-21-06 openQA sle-15-SP4-Server-DVD-Updates-aarch64-Build20240408-1-podman_testsuite@aarch64-virtio test results.png		ph03nix, 2024-04-09 11:21
Screenshot 2024-04-09 at 13-21-32 openQA sle-15-SP4-Server-DVD-Updates-aarch64-Build20240408-1-podman_testsuite@aarch64-virtio test results.png (35.6 KB) Screenshot 2024-04-09 at 13-21-32 openQA sle-15-SP4-Server-DVD-Updates-aarch64-Build20240408-1-podman_testsuite@aarch64-virtio test results.png		ph03nix, 2024-04-09 11:21

Related issues 1 (1 open — 0 closed)

Actions

Copy link

Updated by rbranco 6 months ago · Edited

Perhaps related:
bats uses the TAP format while img_proof & ltp use IPA. Perhaps a limitation in the parser?

Actions

Copy link

Updated by rbranco 6 months ago

Fwiw, it works on the External tab.

Asked in #eng-testing:
https://suse.slack.com/archives/C02CANHLANP/p1713275967130139

Actions

Copy link

Updated by rbranco 6 months ago

This would need some clever refactoring since we're uploading 4 files instead of just one:

Matthias Griessmeier
yes, because it treats bats-user-local_tap as a single testmodule which as a whole can fail, but not the "external results" inside the test module.
(for that there is the second tab)
I didn't know that feature before, but this is expected behaviour for me

Oliver Kurz
yeah, the checkbox is effectively an "Show only test modules with failures", not "show only failed test steps". The question I have is rather why there are 600+ test steps and if that can be better segmented. I don't know how those tests are structured so I don't have an answer

Actions

Copy link

Updated by ph03nix 6 months ago

@rbranco this is just a nice-to-have. If the required effort exceeds the rather cosmetic advantage, then I think we should not bother too much.
However I want to leave the decision on where to draw the line between worthwhile and too much effort to you.

Actions

Copy link