action #72292
closedopenqa-review fails to reference bugs from soft-fail references
Description
Observation¶
reports on https://progress.opensuse.org/projects/qa/wiki#Test-results-overview should reference bugs or tickets in the soft-fail references but they don't . This happened in the past and was fixed e.g. with https://github.com/os-autoinst/openqa_review/commit/935b63b9a536b3aa6eb76cc184fd22b3a720df86 but should be checked again.
Impact¶
QA SLE Project Manager and release manager review the report often or on a daily basis and want to see the bug references linked to soft fail cases. The issue was originally reported by mawerner that "the report is missing soft fail references", i.e. at least a significant amount
Acceptance criteria¶
- AC1: Reports from osd and o3 include bug references for all soft fail references (i.e. linked to
record_soft_failurecalls as well as soft-fail needles)
Suggestions¶
- Maybe https://gitlab.suse.de/openqa/openqa-review/pipelines also shows some errors
- Call
openqa-reviewlocally with according parameters to include soft fails and parse bugrefs and maybe-vvvvto show more logging output and reproduce the problem - Check if this affects all soft fail references or only some
- Fix the problem :)
Workaround¶
Follow the links to the openQA jobs directly and read out the soft fail references from there, e.g. bugzilla links
Updated by okurz over 3 years ago
- Description updated (diff)
- Status changed from New to Workable
- Target version set to Ready
Updated by okurz over 3 years ago
Reproduced with a small set with
python openqa_review/openqa_review.py --host https://openqa.suse.de -vvvv -n -r -T --query-issue-status --no-empty-sections --include-softfails --running-threshold=2 -J https://openqa.suse.de/group_overview/129 --arch aarch64
log messages:
DEBUG:openqa_review/openqa_review.py:job_url /tests/4796197
DEBUG:openqa_review/openqa_review.py:Found older openQA, before https://github.com/os-autoinst/openQA/pull/2932
DEBUG:openqa_review/openqa_review.py:Could not find soft failed info box, looking for workaround needle in job /tests/4796197
INFO:openqa_review/openqa_review.py:Could find neither soft failed info box nor needle, assuming an old openQA job, skipping.
Obviously we don't have an older openQA installation. This means that "/module_components_ajax" can not be found. https://github.com/os-autoinst/openQA/commit/fac9d6dc2b425af6803183d9505a275a28c4d4ab has removed that.
We could try with the following change:
def _get_url_to_softfailed_module(self, job_url):
log.debug('job_url %s' % job_url)
- url = job_url + '/module_components_ajax'
try:
- test_details_html = self.test_browser.get_soup(url).find(title='Soft Failed')
- if test_details_html is None: # pragma: no cover
- log.debug('Found older openQA, before https://github.com/os-autoinst/openQA/pull/3080')
- url = job_url + '/details_ajax'
+ details = self.test_browser.get_json('/api/v1/jobs/' + job_url.split('/')[-1] + '/details')
+ text_data = [j for j in [i for i in details['job']['testresults'] if i['result'] == 'softfailed'][0]['details'] if j['result'] == 'softfail'][0]['text_data']
…
but "text_data" has text like # Soft Failure:\nbsc#1028774: needle(s) scc-registration,yast2-windowborder-corner,registration-online-repos not found within 100\n and we wanted to parse the clickable URL. I could not understand how to find this data. With the "requests" module we should be able to fully simulate a browser.
@mkittler maybe we can look into this together next week?
Updated by mkittler over 3 years ago
The "rendered" link which also contains the URL is part of the preview. With curl you can get it like this: curl -H 'X-Requested-With: XMLHttpRequest' 'https://openqa.opensuse.org/tests/1432333/modules/bootloader_uefi/steps/2'
Updated by okurz over 3 years ago
- Status changed from Workable to In Progress
- Assignee set to okurz
ok, I will try two things: Add the special request header in the request in openqa-review and try to remove the special redirect in openQA code.
Updated by okurz over 3 years ago
- Status changed from In Progress to Feedback
Actually it's easier. I thought we rely on openQA parsing bug references into full URLs which we parse with openqa-review but this is actually not try, we do that conversion in openqa-review itself. So we only need to parse the failing test module name from "details_ajax" and "fake" an old module URL. Fixed in https://github.com/os-autoinst/openqa_review/commit/1091909b91d31d48415ef232b389f586bb3252b9 .
Waiting for verification in the automatic jobs, e.g. tomorrow. Afterwards I can create an update of the package in devel:languages:python as well.
See https://github.com/os-autoinst/openQA/pull/3466 for the suggestion to simplify openQA code a bit.
Updated by okurz over 3 years ago
- Status changed from Feedback to Resolved
this problem is resolved and we have soft-fails with bugrefs again but openqa-review shows errors in the logs, e.g. in https://gitlab.suse.de/openqa/openqa-review/-/jobs/272515 . this is likely related to #52655