Project

General

Profile

Actions
Copy link

action #177321

closed

[BCI] Catch soft-failures from BCI-Tests

Added by ph03nix about 2 months ago. Updated 7 days ago.

Status:
Resolved
Priority:
High
Assignee:
pdostal
Target version:
-
Start date:
2025-02-17
Due date:
% Done:

0%

Estimated time:
Tags:
beginner-friendly, bci

Description

Soft-failures from BCI-Tests are not being shown as softfailures in openQA.

See e.g. https://openqa.opensuse.org/tests/4859876#step/_root_BCI-tests_minimal_/1, where a soft-failure is just displayed as passed

Acceptance criteria

  • AC1: Softfailed tests from BCI-Tests are shown as soft-failures in openQA

Files

softfailure.png (40.8 KB) softfailure.png ph03nix, 2025-02-17 07:59
Actions
Copy link
 #1

Updated by ph03nix about 1 month ago

  • Priority changed from Normal to High
Actions
Copy link
 #2

Updated by pdostal about 1 month ago

  • Status changed from Workable to In Progress
  • Assignee set to pdostal
Actions
Copy link
 #5

Updated by pdostal about 1 month ago

  • Status changed from In Progress to Blocked

Blocked until OSD upgrades.

Actions
Copy link
 #6

Updated by pdostal 29 days ago

  • Status changed from Blocked to In Progress

Deployed in OSD as well as on O3. My test code pull request is ready to be merged.

Actions
Copy link
 #8

Updated by ph03nix 23 days ago

Example xml document for a softfailure: https://openqa.opensuse.org/tests/4857401/file/bci_logs-junit_minimal_.xml

<testsuites>
<testsuite name="pytest" errors="0" failures="0" skipped="1" tests="3" time="14.221" timestamp="2025-02-15T09:01:29.183753-05:00" hostname="susetest">
<testcase classname="tests.test_minimal" name="test_minimal_image_size[bci/bci-minimal:latest from registry.opensuse.org/opensuse/factory/totest/containers/opensuse/bci/bci-minimal:latest]" time="2.671">
<skipped type="pytest.xfail" message="Tumbleweed Minimal image exceeds limit (boo#1236736)"/>
</testcase>
<testcase classname="tests.test_minimal" name="test_rpm_present_in_minimal[bci/bci-minimal:latest from registry.opensuse.org/opensuse/factory/totest/containers/opensuse/bci/bci-minimal:latest]" time="12.947"/>
<testcase classname="tests.test_minimal" name="test_fat_packages_absent[bci/bci-minimal:latest from registry.opensuse.org/opensuse/factory/totest/containers/opensuse/bci/bci-minimal:latest]" time="11.156"/>
</testsuite>
</testsuites>
Actions
Copy link
 #10

Updated by pdostal 10 days ago

  • Status changed from In Progress to Resolved
Actions
Copy link
 #11

Updated by ph03nix 7 days ago

Thank you!

Actions
Copy link

Also available in: Atom PDF