action #165692
closed
[openQA-in-openQA] test does not fail if state is done but incomplete auto_review:"no candidate needle.*openqa-dashboard.*matched" size:S
Added by jbaier_cz 9 months ago.
Updated 8 months ago.
Category:
Feature requests
Description
Motivation¶
During my work on #155413 I found out, that we are looking for state running or done, but we are also considering incomplete jobs as not problematic which can hide other issues (like in my case the AppArmor rule missing). See the attached screen. We should only allow running and passed jobs and fail otherwise.
Acceptance criteria¶
-
AC1: If the test scheduled is done but not passed the test should fail
-
AC2: The overview of running/finished tests is visible and we click on the test
Files
- Related to action #155413: Ensure apparmor is enforced in openQA-in-openQA tests size:M added
- Related to action #165408: Unreviewed issue (Group 24 openQA) test-running in openQA failing with error detecting default remote branch size:S added
- Category set to Feature requests
That's exactly the kind of ticket I wanted to create myself today!
I also thought it would be another improvement to go to /tests in the dashboard and click on the test, so we can see how the test looks like.
It can be much easier to see what's going on there instead of looking at a JSON file without line breaks.
- Subject changed from [openQA-in-openQA] test does not fail if state is done but incomplete to [openQA-in-openQA] test does not fail if state is done but incomplete size:S
- Description updated (diff)
- Status changed from New to Workable
- Assignee set to jbaier_cz
It looks like the solution for AC1 is almost already in the code. I will try to unify the code paths.
- Status changed from Workable to In Progress
- Due date set to 2024-09-13
Setting due date based on mean cycle time of SUSE QE Tools
- Due date deleted (
2024-09-13)
- Status changed from In Progress to Resolved
Both changes can be seen in production:
- Status changed from Resolved to Feedback
- Status changed from Feedback to In Progress
- Due date set to 2024-09-20
Setting due date based on mean cycle time of SUSE QE Tools
- Subject changed from [openQA-in-openQA] test does not fail if state is done but incomplete size:S to [openQA-in-openQA] test does not fail if state is done but incomplete auto_review:"o candidate needle with tag(s) .*openqa-test-details.*matched" size:S
- Subject changed from [openQA-in-openQA] test does not fail if state is done but incomplete auto_review:"o candidate needle with tag(s) .*openqa-test-details.*matched" size:S to [openQA-in-openQA] test does not fail if state is done but incomplete auto_review:"o candidate needle with tag(s) .*openqa-(test-details|search-results).*matched" size:S
- Subject changed from [openQA-in-openQA] test does not fail if state is done but incomplete auto_review:"o candidate needle with tag(s) .*openqa-(test-details|search-results).*matched" size:S to [openQA-in-openQA] test does not fail if state is done but incomplete size:S
- Related to action #166445: [openQA-in-openQA][sporadic] test fails in tests, simple_boot incomplete auto_review:"no candidate needle.*openqa-test-details.*matched":retry added
- Status changed from In Progress to Feedback
- Subject changed from [openQA-in-openQA] test does not fail if state is done but incomplete size:S to [openQA-in-openQA] test does not fail if state is done but incomplete auto_review:"no candidate needle.*openqa-dashboard.*matched" size:S
- Due date deleted (
2024-09-20)
- Status changed from Feedback to Resolved
Also available in: Atom
PDF
