Project

General

Profile

action #100709

openqa-review pipeline failed because details-* JSON is empty

Added by tinita about 2 months ago. Updated about 1 month ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Concrete Bugs
Target version:
Start date:
2021-10-11
Due date:
2021-11-05
% Done:

0%

Estimated time:
Difficulty:

Description

https://gitlab.suse.de/openqa/openqa-review/-/jobs/633333

Traceback (most recent call last):
  File "/usr/bin/openqa-review", line 33, in <module>
    sys.exit(load_entry_point('openqa-review==0.0.0', 'console_scripts', 'openqa-review')())
  File "/usr/lib/python3.8/site-packages/openqa_review/openqa_review.py", line 1557, in main
    report = generate_report(args)
  File "/usr/lib/python3.8/site-packages/openqa_review/openqa_review.py", line 1488, in generate_report
    return Report(browser, args, root_url, job_groups)
  File "/usr/lib/python3.8/site-packages/openqa_review/openqa_review.py", line 1437, in __init__
    self.report[k] = self._one_report(v)
  File "/usr/lib/python3.8/site-packages/openqa_review/openqa_review.py", line 1448, in _one_report
    return ProductReport(self.browser, job_group_url, self.root_url, self.args)
  File "/usr/lib/python3.8/site-packages/openqa_review/openqa_review.py", line 1162, in __init__
    self.reports[arch] = ArchReport(arch, results, args, root_url, progress_browser, bugzilla_browser, browser)
  File "/usr/lib/python3.8/site-packages/openqa_review/openqa_review.py", line 898, in __init__
    self._search_for_bugrefs_for_softfailures(results)
  File "/usr/lib/python3.8/site-packages/openqa_review/openqa_review.py", line 945, in _search_for_bugrefs_for_softfailures
    v["bugref"] = self._get_bugref_for_softfailed_module(v, module_name)
  File "/usr/lib/python3.8/site-packages/openqa_review/openqa_review.py", line 1003, in _get_bugref_for_softfailed_module
    details_json = self.test_browser.get_json(details_url)
  File "/usr/lib/python3.8/site-packages/openqa_review/browser.py", line 77, in get_json
    return self.get_page(url, as_json=True, cache=cache)
  File "/usr/lib/python3.8/site-packages/openqa_review/browser.py", line 104, in get_page
    content = self._get(absolute_url, as_json=as_json)
  File "/usr/lib/python3.8/site-packages/openqa_review/browser.py", line 159, in _get
    content = r.json() if as_json else r.content.decode("utf8")
  File "/usr/lib/python3.8/site-packages/requests/models.py", line 900, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/lib64/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python3.8/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python3.8/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
+ save_report=
+ cleanup
+ rm -r /tmp/tmp.H6jSa7iqg0
+++ kill %1
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: command terminated with exit code 1

History

#1 Updated by tinita about 2 months ago

Restarted twice, but always failed with the same error

#2 Updated by cdywan about 2 months ago

This seems to file in browser.py in this spot:

content = r.json() if as_json else r.content.decode("utf8")

So I'm guessing we're not getting JSON but something else. And the something else wasn't caught as a HTTPError.

Here's a small PR to expose the error to start with: https://github.com/os-autoinst/openqa_review/pull/180

#3 Updated by tinita about 2 months ago

Thanks to the User-Agent header, from the log I could determine which url it was:
https://openqa.opensuse.org/tests/1961715/file/details-libreoffice_recent_documents.json

It's empty, and all details-*.json files for this test are empty:

ls -l /var/lib/openqa/testresults/01961/01961715-opensuse-Tumbleweed-DVD-x86_64-Build20211008-desktopapps-documentation@64bit/details-*

#4 Updated by cdywan about 2 months ago

  • Subject changed from openqa-review pipeline failed to openqa-review pipeline failed because details-* JSON is empty
  • Status changed from New to In Progress
  • Assignee set to cdywan

I'm taking this ticket. With Tina's assessment I think it's pretty clear what we need to do here.

#5 Updated by okurz about 2 months ago

  • Target version set to Ready

#6 Updated by openqa_review about 2 months ago

  • Due date set to 2021-10-26

Setting due date based on mean cycle time of SUSE QE Tools

#7 Updated by okurz about 2 months ago

brought up in weekly. cdywan Please check if you have a related PR or something prepared locally.

#8 Updated by cdywan about 2 months ago

Here's a small PR to expose the error to start with: https://github.com/os-autoinst/openqa_review/pull/180

Not merged yet, because I was having issues locally runnings the tests. And apparently deprecation warnings are breaking one of the Travis tests.

I'm taking this ticket. With Tina's assessment I think it's pretty clear what we need to do here.

Next time I better type out what I was going to, since I can't find the branch I thought I had.

That said, I had another thought: # Reason: isotovideo done: Can't use an undefined value as a symbol reference at /home/squamata/os-autoinst/myjsonrpc.pm line 54. can prevent some jobs from writing the file. So that might be the underlying cause.

#9 Updated by okurz about 2 months ago

#10 Updated by cdywan about 1 month ago

  • Status changed from In Progress to Feedback

https://github.com/os-autoinst/openqa_review/pull/180 got merged as did https://github.com/os-autoinst/os-autoinst/pull/1828 so in theory I expect:

  • empty files don't fail the pipeline
  • invalid file descriptors fail affected jobs with myjsonprc: called on undefined file descriptor

#11 Updated by tinita about 1 month ago

The title of https://github.com/os-autoinst/os-autoinst/pull/1828 was a bit misleading. I changed it from
"myjsonrpc: Prevent warning 'Can't use an undefined value as a symbol reference'"
to
"myjsonrpc: Prevent error 'Can't use an undefined value as a symbol reference'"
as this is actually already a fatal error in perl and not a warning.

So affected jobs have been failing before, just now with a better error message.

#12 Updated by cdywan about 1 month ago

  • Due date changed from 2021-10-26 to 2021-11-05

tinita wrote:

The title of https://github.com/os-autoinst/os-autoinst/pull/1828 was a bit misleading. I changed it from
"myjsonrpc: Prevent warning 'Can't use an undefined value as a symbol reference'"
to
"myjsonrpc: Prevent error 'Can't use an undefined value as a symbol reference'"
as this is actually already a fatal error in perl and not a warning.

So affected jobs have been failing before, just now with a better error message.

Ack. Note that I'm bumping the Due date to reflect both the latest and earliest I will be able to check that reports are fine. Maybe somebody can check it earlier.

#14 Updated by tinita about 1 month ago

  • Status changed from Feedback to Resolved

Also available in: Atom PDF