openqa-review fails to reference bugs from soft-fail references
reports on https://progress.opensuse.org/projects/qa/wiki#Test-results-overview should reference bugs or tickets in the soft-fail references but they don't . This happened in the past and was fixed e.g. with https://github.com/os-autoinst/openqa_review/commit/935b63b9a536b3aa6eb76cc184fd22b3a720df86 but should be checked again.
QA SLE Project Manager and release manager review the report often or on a daily basis and want to see the bug references linked to soft fail cases. The issue was originally reported by mawerner that "the report is missing soft fail references", i.e. at least a significant amount
- AC1: Reports from osd and o3 include bug references for all soft fail references (i.e. linked to
record_soft_failurecalls as well as soft-fail needles)
- Maybe https://gitlab.suse.de/openqa/openqa-review/pipelines also shows some errors
openqa-reviewlocally with according parameters to include soft fails and parse bugrefs and maybe
-vvvvto show more logging output and reproduce the problem
- Check if this affects all soft fail references or only some
- Fix the problem :)
Follow the links to the openQA jobs directly and read out the soft fail references from there, e.g. bugzilla links
Reproduced with a small set with
python openqa_review/openqa_review.py --host https://openqa.suse.de -vvvv -n -r -T --query-issue-status --no-empty-sections --include-softfails --running-threshold=2 -J https://openqa.suse.de/group_overview/129 --arch aarch64
DEBUG:openqa_review/openqa_review.py:job_url /tests/4796197 DEBUG:openqa_review/openqa_review.py:Found older openQA, before https://github.com/os-autoinst/openQA/pull/2932 DEBUG:openqa_review/openqa_review.py:Could not find soft failed info box, looking for workaround needle in job /tests/4796197 INFO:openqa_review/openqa_review.py:Could find neither soft failed info box nor needle, assuming an old openQA job, skipping.
Obviously we don't have an older openQA installation. This means that "/module_components_ajax" can not be found. https://github.com/os-autoinst/openQA/commit/fac9d6dc2b425af6803183d9505a275a28c4d4ab has removed that.
We could try with the following change:
def _get_url_to_softfailed_module(self, job_url): log.debug('job_url %s' % job_url) - url = job_url + '/module_components_ajax' try: - test_details_html = self.test_browser.get_soup(url).find(title='Soft Failed') - if test_details_html is None: # pragma: no cover - log.debug('Found older openQA, before https://github.com/os-autoinst/openQA/pull/3080') - url = job_url + '/details_ajax' + details = self.test_browser.get_json('/api/v1/jobs/' + job_url.split('/')[-1] + '/details') + text_data = [j for j in [i for i in details['job']['testresults'] if i['result'] == 'softfailed']['details'] if j['result'] == 'softfail']['text_data'] …
but "text_data" has text like
# Soft Failure:\nbsc#1028774: needle(s) scc-registration,yast2-windowborder-corner,registration-online-repos not found within 100\n and we wanted to parse the clickable URL. I could not understand how to find this data. With the "requests" module we should be able to fully simulate a browser.
mkittler maybe we can look into this together next week?
- Status changed from In Progress to Feedback
Actually it's easier. I thought we rely on openQA parsing bug references into full URLs which we parse with openqa-review but this is actually not try, we do that conversion in openqa-review itself. So we only need to parse the failing test module name from "details_ajax" and "fake" an old module URL. Fixed in https://github.com/os-autoinst/openqa_review/commit/1091909b91d31d48415ef232b389f586bb3252b9 .
Waiting for verification in the automatic jobs, e.g. tomorrow. Afterwards I can create an update of the package in devel:languages:python as well.
See https://github.com/os-autoinst/openQA/pull/3466 for the suggestion to simplify openQA code a bit.
- Status changed from Feedback to Resolved
this problem is resolved and we have soft-fails with bugrefs again but openqa-review shows errors in the logs, e.g. in https://gitlab.suse.de/openqa/openqa-review/-/jobs/272515 . this is likely related to #52655