action #100709
closedopenqa-review pipeline failed because details-* JSON is empty
Description
https://gitlab.suse.de/openqa/openqa-review/-/jobs/633333
Traceback (most recent call last):
File "/usr/bin/openqa-review", line 33, in <module>
sys.exit(load_entry_point('openqa-review==0.0.0', 'console_scripts', 'openqa-review')())
File "/usr/lib/python3.8/site-packages/openqa_review/openqa_review.py", line 1557, in main
report = generate_report(args)
File "/usr/lib/python3.8/site-packages/openqa_review/openqa_review.py", line 1488, in generate_report
return Report(browser, args, root_url, job_groups)
File "/usr/lib/python3.8/site-packages/openqa_review/openqa_review.py", line 1437, in __init__
self.report[k] = self._one_report(v)
File "/usr/lib/python3.8/site-packages/openqa_review/openqa_review.py", line 1448, in _one_report
return ProductReport(self.browser, job_group_url, self.root_url, self.args)
File "/usr/lib/python3.8/site-packages/openqa_review/openqa_review.py", line 1162, in __init__
self.reports[arch] = ArchReport(arch, results, args, root_url, progress_browser, bugzilla_browser, browser)
File "/usr/lib/python3.8/site-packages/openqa_review/openqa_review.py", line 898, in __init__
self._search_for_bugrefs_for_softfailures(results)
File "/usr/lib/python3.8/site-packages/openqa_review/openqa_review.py", line 945, in _search_for_bugrefs_for_softfailures
v["bugref"] = self._get_bugref_for_softfailed_module(v, module_name)
File "/usr/lib/python3.8/site-packages/openqa_review/openqa_review.py", line 1003, in _get_bugref_for_softfailed_module
details_json = self.test_browser.get_json(details_url)
File "/usr/lib/python3.8/site-packages/openqa_review/browser.py", line 77, in get_json
return self.get_page(url, as_json=True, cache=cache)
File "/usr/lib/python3.8/site-packages/openqa_review/browser.py", line 104, in get_page
content = self._get(absolute_url, as_json=as_json)
File "/usr/lib/python3.8/site-packages/openqa_review/browser.py", line 159, in _get
content = r.json() if as_json else r.content.decode("utf8")
File "/usr/lib/python3.8/site-packages/requests/models.py", line 900, in json
return complexjson.loads(self.text, **kwargs)
File "/usr/lib64/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
+ save_report=
+ cleanup
+ rm -r /tmp/tmp.H6jSa7iqg0
+++ kill %1
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: command terminated with exit code 1
Updated by tinita over 2 years ago
Restarted twice, but always failed with the same error
Updated by livdywan over 2 years ago
This seems to file in browser.py in this spot:
content = r.json() if as_json else r.content.decode("utf8")
So I'm guessing we're not getting JSON but something else. And the something else wasn't caught as a HTTPError.
Here's a small PR to expose the error to start with: https://github.com/os-autoinst/openqa_review/pull/180
Updated by tinita over 2 years ago
Thanks to the User-Agent header, from the log I could determine which url it was:
https://openqa.opensuse.org/tests/1961715/file/details-libreoffice_recent_documents.json
It's empty, and all details-*.json files for this test are empty:
ls -l /var/lib/openqa/testresults/01961/01961715-opensuse-Tumbleweed-DVD-x86_64-Build20211008-desktopapps-documentation@64bit/details-*
Updated by livdywan over 2 years ago
- Subject changed from openqa-review pipeline failed to openqa-review pipeline failed because details-* JSON is empty
- Status changed from New to In Progress
- Assignee set to livdywan
I'm taking this ticket. With Tina's assessment I think it's pretty clear what we need to do here.
Updated by openqa_review over 2 years ago
- Due date set to 2021-10-26
Setting due date based on mean cycle time of SUSE QE Tools
Updated by okurz over 2 years ago
brought up in weekly. @cdywan Please check if you have a related PR or something prepared locally.
Updated by livdywan over 2 years ago
Here's a small PR to expose the error to start with: https://github.com/os-autoinst/openqa_review/pull/180
Not merged yet, because I was having issues locally runnings the tests. And apparently deprecation warnings are breaking one of the Travis tests.
I'm taking this ticket. With Tina's assessment I think it's pretty clear what we need to do here.
Next time I better type out what I was going to, since I can't find the branch I thought I had.
That said, I had another thought: # Reason: isotovideo done: Can't use an undefined value as a symbol reference at /home/squamata/os-autoinst/myjsonrpc.pm line 54. can prevent some jobs from writing the file. So that might be the underlying cause.
Updated by okurz over 2 years ago
interesting. Then https://github.com/os-autoinst/os-autoinst/pull/1828 could help
Updated by livdywan over 2 years ago
- Status changed from In Progress to Feedback
https://github.com/os-autoinst/openqa_review/pull/180 got merged as did https://github.com/os-autoinst/os-autoinst/pull/1828 so in theory I expect:
- empty files don't fail the pipeline
- invalid file descriptors fail affected jobs with
myjsonprc: called on undefined file descriptor
Updated by tinita over 2 years ago
The title of https://github.com/os-autoinst/os-autoinst/pull/1828 was a bit misleading. I changed it from
"myjsonrpc: Prevent warning 'Can't use an undefined value as a symbol reference'"
to
"myjsonrpc: Prevent error 'Can't use an undefined value as a symbol reference'"
as this is actually already a fatal error in perl and not a warning.
So affected jobs have been failing before, just now with a better error message.
Updated by livdywan over 2 years ago
- Due date changed from 2021-10-26 to 2021-11-05
tinita wrote:
The title of https://github.com/os-autoinst/os-autoinst/pull/1828 was a bit misleading. I changed it from
"myjsonrpc: Prevent warning 'Can't use an undefined value as a symbol reference'"
to
"myjsonrpc: Prevent error 'Can't use an undefined value as a symbol reference'"
as this is actually already a fatal error in perl and not a warning.So affected jobs have been failing before, just now with a better error message.
Ack. Note that I'm bumping the Due date to reflect both the latest and earliest I will be able to check that reports are fine. Maybe somebody can check it earlier.
Updated by tinita over 2 years ago
https://github.com/os-autoinst/openqa_review/pull/180 introduced an error, and a fix was merged: https://github.com/os-autoinst/openqa_review/pull/188