Project

General

Profile

action #72292

openqa-review fails to reference bugs from soft-fail references

Added by okurz 10 months ago. Updated 9 months ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Concrete Bugs
Target version:
Start date:
2020-10-06
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Observation

reports on https://progress.opensuse.org/projects/qa/wiki#Test-results-overview should reference bugs or tickets in the soft-fail references but they don't . This happened in the past and was fixed e.g. with https://github.com/os-autoinst/openqa_review/commit/935b63b9a536b3aa6eb76cc184fd22b3a720df86 but should be checked again.

Impact

QA SLE Project Manager and release manager review the report often or on a daily basis and want to see the bug references linked to soft fail cases. The issue was originally reported by mawerner that "the report is missing soft fail references", i.e. at least a significant amount

Acceptance criteria

  • AC1: Reports from osd and o3 include bug references for all soft fail references (i.e. linked to record_soft_failure calls as well as soft-fail needles)

Suggestions

  • Maybe https://gitlab.suse.de/openqa/openqa-review/pipelines also shows some errors
  • Call openqa-review locally with according parameters to include soft fails and parse bugrefs and maybe -vvvv to show more logging output and reproduce the problem
  • Check if this affects all soft fail references or only some
  • Fix the problem :)

Workaround

Follow the links to the openQA jobs directly and read out the soft fail references from there, e.g. bugzilla links

History

#1 Updated by okurz 10 months ago

  • Description updated (diff)
  • Status changed from New to Workable
  • Target version set to Ready

#2 Updated by okurz 10 months ago

  • Description updated (diff)

#3 Updated by okurz 10 months ago

  • Description updated (diff)

#4 Updated by okurz 10 months ago

  • Description updated (diff)

#5 Updated by okurz 10 months ago

Reproduced with a small set with

python openqa_review/openqa_review.py --host https://openqa.suse.de -vvvv -n -r -T --query-issue-status --no-empty-sections --include-softfails --running-threshold=2 -J https://openqa.suse.de/group_overview/129 --arch aarch64

log messages:

DEBUG:openqa_review/openqa_review.py:job_url /tests/4796197
DEBUG:openqa_review/openqa_review.py:Found older openQA, before https://github.com/os-autoinst/openQA/pull/2932
DEBUG:openqa_review/openqa_review.py:Could not find soft failed info box, looking for workaround needle in job /tests/4796197
INFO:openqa_review/openqa_review.py:Could find neither soft failed info box nor needle, assuming an old openQA job, skipping.

Obviously we don't have an older openQA installation. This means that "/module_components_ajax" can not be found. https://github.com/os-autoinst/openQA/commit/fac9d6dc2b425af6803183d9505a275a28c4d4ab has removed that.

We could try with the following change:

     def _get_url_to_softfailed_module(self, job_url):                                                                                                       
         log.debug('job_url %s' % job_url)                                                                                                                   
-        url = job_url + '/module_components_ajax'                                                                                                           
         try:                                                                                                                                                
-            test_details_html = self.test_browser.get_soup(url).find(title='Soft Failed')                                                                   
-            if test_details_html is None:  # pragma: no cover                                                                                               
-                log.debug('Found older openQA, before https://github.com/os-autoinst/openQA/pull/3080')                                                     
-                url = job_url + '/details_ajax'                                                                                                             
+            details = self.test_browser.get_json('/api/v1/jobs/' + job_url.split('/')[-1] + '/details')                                                     
+            text_data = [j for j in [i for i in details['job']['testresults'] if i['result'] == 'softfailed'][0]['details'] if j['result'] == 'softfail'][0]['text_data']
…

but "text_data" has text like # Soft Failure:\nbsc#1028774: needle(s) scc-registration,yast2-windowborder-corner,registration-online-repos not found within 100\n and we wanted to parse the clickable URL. I could not understand how to find this data. With the "requests" module we should be able to fully simulate a browser.

mkittler maybe we can look into this together next week?

#6 Updated by mkittler 10 months ago

The "rendered" link which also contains the URL is part of the preview. With curl you can get it like this: curl -H 'X-Requested-With: XMLHttpRequest' 'https://openqa.opensuse.org/tests/1432333/modules/bootloader_uefi/steps/2'

#7 Updated by okurz 10 months ago

  • Status changed from Workable to In Progress
  • Assignee set to okurz

ok, I will try two things: Add the special request header in the request in openqa-review and try to remove the special redirect in openQA code.

#8 Updated by okurz 10 months ago

  • Status changed from In Progress to Feedback

Actually it's easier. I thought we rely on openQA parsing bug references into full URLs which we parse with openqa-review but this is actually not try, we do that conversion in openqa-review itself. So we only need to parse the failing test module name from "details_ajax" and "fake" an old module URL. Fixed in https://github.com/os-autoinst/openqa_review/commit/1091909b91d31d48415ef232b389f586bb3252b9 .

Waiting for verification in the automatic jobs, e.g. tomorrow. Afterwards I can create an update of the package in devel:languages:python as well.

See https://github.com/os-autoinst/openQA/pull/3466 for the suggestion to simplify openQA code a bit.

#9 Updated by okurz 9 months ago

  • Status changed from Feedback to Resolved

this problem is resolved and we have soft-fails with bugrefs again but openqa-review shows errors in the logs, e.g. in https://gitlab.suse.de/openqa/openqa-review/-/jobs/272515 . this is likely related to #52655

Also available in: Atom PDF