QA » openQA Project

needle git hash available in vars.json as well as in output of autoinst-log.txt. Also https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3141 can serve as a nice example how more information can be extracted from within the SUT by looking into details of y2log and popping up with a record_info and more details so that one does not need to transcribe screenshots or download the logfiles to look into details.

The following can help further to get details then:

import requests
test_url = 'https://openqa.suse.de/tests/1033327'
j = requests.get(test_url + '/file/details-yast2_lan_restart.json').json()
soft_failed_name = [i['text'] for i in j if 'title' in i.keys() and 'Soft Fail' in i['title']][0]
out = requests.get(test_url + '/file/' + soft_failed_name).content
print(out.decode('utf-8'))

-> # Soft Failure:
bsc#992113

Since the last update openqa-review improved by parsing soft-fail info boxes as well as soft-fail needles including reminder comments on tickets for these cases as well. Some further ideas are mentioned in the description for anyone to pick up.

back to workable as it's not really "In Progress" now.

https://github.com/os-autoinst/openQA/pull/2612

idea from #61103 included. Last change was the HTML table. Some things crossed of the list. With this I unassign again.

https://github.com/os-autoinst/scripts/pull/23 to cover "See https://github.com/okurz/openqa_review/tree/feature/investigate especially https://github.com/okurz/openqa_review/blob/feature/investigate/openqa_review/investigate.py"

crossed off two other points and deleted "* os-autoinst version in vars.json" as this should either not be necessary considering that the version is in the log file or implicitly covered by "all package changes, e.g. save rpm -qa in file and provide diff and/or changelog" which one can do on the worker just as well as within the SUT.

I have tested the call to rpm -qa on all osd workers:

# salt -l error --no-color -C  'G@roles:worker' --state-output=changes cmd.run "time rpm -qa > /tmp/rpm_qa; ls -lh /tmp/rpm_qa; rm /tmp/rpm_qa"
openqaworker3.suse.de: 0m1.187s, 68K
openqaworker2.suse.de: 0m1.191s, 64K
openqaworker6.suse.de: 0m1.210s, 67K
openqaworker8.suse.de: 0m1.210s, 63K
openqaworker5.suse.de: 0m1.226s, 67K
openqaworker9.suse.de: 0m1.510s, 63K
malbec.arch.suse.de: 0m1.730s, 53K
openqaworker10.suse.de: 0m1.715s, 60K
QA-Power8-5-kvm.qa.suse.de: 0m1.826s, 65K
openqaworker13.suse.de: 0m1.765s, 64K
QA-Power8-4-kvm.qa.suse.de: 0m1.884s, 67K
powerqaworker-qam-1: 0m2.244s, 93K
openqaworker-arm-1.suse.de: 0m4.072s, 63K
openqaworker-arm-2.suse.de: 0m4.076s, 62K
openqaworker-arm-3.suse.de: 0m4.639s, 72K
grenache-1.qa.suse.de: 0m4.859s, 92K

So 1-5s, recording 60-100kB. We could record this in every job but feels not efficient. Parsing zypper log only in case of an error with the diff computed between the date of "last good" and "first bad" feels smarter albeit of course a little bit more complicated :) The philosophy to follow would be to make the happy path (test passes) as efficient as possible and treat failures as exceptions, hence I consider recording rpm -qa in every job regardless of the status as not the right option.

os-autoinst/os-autoinst#1367 merged, can now continue with https://github.com/os-autoinst/scripts/pull/23

With https://github.com/os-autoinst/scripts/pull/24 I added a script to look for "investigation candidates" and added a feature to "openqa-investigate" to post automatic comments to openQA with links to triggered investigation jobs. With https://gitlab.suse.de/openqa/auto-review/-/merge_requests/3 I run this daily (for now) so automatic investigation jobs get triggered now for all failed openQA jobs that do not have a label, e.g. new failures: https://openqa.opensuse.org/tests/1218545#comments . Based on the results one should be able to more easily determine if issues are sporadic, test regressions, product regressions. Based on that I await any feedback. We can also continue the work in openQA itself.

I guess by now I can update the bug as I have not received further feedback lately. Results and observations so far:

• Some people like the "Investigation" tab, some have not even found it and got lost in their own investigation :(
• The script solution in #19720#note-19 kinda works and can be replaced by a solution within openQA which I have prepared some steps for but did not manage to continue further so far
• The other mentioned ideas are also still valid and can be done anytime

@ilausuch assigning to you. Even if you don't plan to cover all ideas you can track this ticket and after being done with the parts that you planned you can set back to "Workable" and unassign again

discussed with ilausuch, clarified the approach regarding saving changes from worker/SUT, updated description.

due to changes in a related task: #69085

due to changes in a related task: #69088

Updated description with links to individual PRs for already covered items, reworked to be "epic" with subtasks for current WIP items

I release this ticket for now

fine. For blocked tickets however we should have an assignee. In this case the ticket was assigned to you as you are assigned to the subtask #69088 . For now I can do the same myself of course.

the current only left subtask is for future so moving this epic there as well

All subtasks done with two remotely related ones moved to #102912 which covers the rest. I check the description and we have covered really the most of it can call this epic done. Yay! Thanks to all contributors!