action #114451
closedIncidents from all test issues variables are collected during bisect size:M
Description
Observation¶
https://openqa.suse.de/tests/9184859#comments shows a comment from openqa-investigate. As this job is an aggregate test I would have expected also a result from https://github.com/os-autoinst/scripts/blob/master/openqa-trigger-bisect-jobs but we only have one entry in the job setting OS_TEST_ISSUES. Likely we should also trigger based on entries in BASE_TEST_ISSUES, DESKTOP_TEST_ISSUES, etc.
Acceptance criteria¶
- AC1: Results from openqa-trigger-bisect-jobs are shown in job comments whenever an aggregate test includes entries in multiple TEST_ISSUES variables
- AC2: Incidents are pulled from all *_TEST_ISSUES variables
Suggestions¶
- Understand the meaning of the different "TEST_ISSUES" variables We don't care, we just assume we can treat all the same
- Extend https://github.com/os-autoinst/scripts/blob/master/openqa-trigger-bisect-jobs to look into multiple TEST_ISSUES variables instead of just OS_TEST_ISSUES
Out of scope¶
- Do not care about performance impact of "too many openQA jobs" triggered, we should have enough capacity
Updated by livdywan about 2 years ago
- Subject changed from gimme more bisect jobs! to Incidents from all test issues variables are collected during bisect size:M
- Description updated (diff)
- Status changed from New to Workable
Updated by tinita about 2 years ago
- Status changed from Workable to In Progress
- Assignee set to tinita
Updated by openqa_review about 2 years ago
- Due date set to 2022-08-16
Setting due date based on mean cycle time of SUSE QE Tools
Updated by tinita about 2 years ago
- Status changed from In Progress to Workable
- Assignee deleted (
tinita)
Currently I'm not able to work on it
Updated by tinita about 2 years ago
- Status changed from Workable to In Progress
- Assignee set to tinita
Updated by tinita about 2 years ago
I think I have the basic implementation and will now add unit tests
https://github.com/perlpunk/scripts/tree/bisect-all-issue-kinds
Updated by tinita about 2 years ago
PR: https://github.com/os-autoinst/scripts/pull/177 - Bisect all *_TEST_ISSUES variables
Updated by tinita about 2 years ago
- Status changed from In Progress to Feedback
Updated by tinita about 2 years ago
PR was merged, I will check if it works in production
Updated by osukup about 2 years ago
think there will be small problem with #111710 on OSD
Updated by tinita about 2 years ago
I was able to search for jobs with this query:
select j.id, jg.name, j.clone_id, js.key, js.value,j.t_created
from job_settings js join jobs j on js.job_id=j.id left join job_groups jg on j.group_id=jg.id
where js.key like '%_TEST_ISSUES' and js.value like '%,%' and j.result='failed'
and j.t_created >= '2022-08-10' and j.clone_id is null
and jg.name !~ '(Development|Public Cloud|Released|Others|Kernel|Virtualization).*';
I looked at this job https://openqa.suse.de/tests/9298857 but there were only the normal last_good_* investigate jobs.
I ran the hook manually but commented out line 18 echo "$test" | "$script_dir/openqa-investigate"
:
env script_dir=/opt/os-autoinst-scripts enable_force_result=true host=openqa.suse.de exclude_group_regex='.*(Development|Public Cloud|Released|Others|Kernel|Virtualization).*' grep_timeout=60 nice ionice -c idle /tmp/openqa-label-known-issues-and-investigate-hook 9298857
and now the investigation jobs have been created.
I enabled line 18 again and got this:
Skipping investigation of job 9298857: job cluster is already being investigated, see comment on job 9298857
403 Forbidden
404 Not Found
{"error_status":404}
So a new task is to improve error handling, so we can actually see where this comes from.
But no bisect jobs were created this time. I think the openqa-investigate exits with non-zero and aborts the script.
To search for actual bisect jobs I did:
select id, clone_id, test, t_created, state, result from jobs where t_created >= '2022-08-10' and test like '%:bisect_without_%';
and got 19 jobs for today, so in general it works. Example: https://openqa.suse.de/tests/9296997#comments
Updated by tinita about 2 years ago
- Copied to action #115178: openqa-investigate: Ensure proper error handling size:M added
Updated by tinita about 2 years ago
- Status changed from Feedback to Resolved
It works, so I resolve it.
Created #115178 as a followup.
Updated by tinita about 2 years ago
- Status changed from Resolved to Workable
The logic needs to be a bit different from what I implemented:
- An issue can appear in several *_TEST_ISSUES variables
- When removing it, it has to be removed from all the variables
- https://github.com/os-autoinst/scripts/pull/182 exists to support _TEST_REPOS
Updated by livdywan about 2 years ago
- Status changed from Workable to In Progress
- Assignee set to osukup
Updated by openqa_review about 2 years ago
- Due date set to 2022-09-07
Setting due date based on mean cycle time of SUSE QE Tools
Updated by livdywan about 2 years ago
- Status changed from Feedback to Resolved
@osukup confirmed this can be resolved.