action #73366
closedauto-review: Improve output
Description
Motivation¶
We run auto_review aka openqa_review in GitLab CI. The output of the script can be seen in the log of the pipeline, which can be hard to read because one has to find the relevant snippets first within the verbose output that we currently use.
Acceptance Criteria¶
TBD
Suggestions¶
- Look into pipelines on https://gitlab.suse.de/openqa/openqa-review/ as part of the Monitor step in our DevOps workflow
- GitLab CI has support for JUnit artifacts i.e. XML files that contain sections for passed or failed tests. Those could be (ab)used to list the found jobs. The script just needs to add the XML boilerplate and write to a file (this is the same concept we use with CircleCI to render TAP output).
- Found jobs could be shown as pipeline failures in GitLab CI w/o going into the verbose logs
- JUnit output is optional and the script output remains available for e.g. local usage
Updated by livdywan over 3 years ago
- Description updated (diff)
- Category set to Feature requests
Updated by okurz over 3 years ago
- Subject changed from Implement JUnit output in auto_review to auto-review: Improve output
- Description updated (diff)
- Target version set to Ready
Hi, I changed the description to better conform to the template of feature requests instead of defects :)
In particular I changed the subject line to not prescribe the implementation but leave it open to the developer to choose how to cover the user story.
Also I moved the existing acceptance criteria into suggestions because I think it is actually not feasible for us to achieve AC1. To my understanding a gitlab CI pipeline can not dynamically define new jobs that would represent failed results. Also the optional junit part should not be an acceptance criterion if it is optional.
@cdywan WDYT?
Updated by mkittler over 3 years ago
PR for printing a summary of jobs to be reviewed at the end: https://github.com/os-autoinst/scripts/pull/38
That's just a simple change but already a huge improvement as one no longer needs to go though all the verbose output. I made it work with set -x by using only a single echo. Unfortunately the links will still not be clickable from GitLab's CI output. I suppose using JUnit would only be beneficial if it leads to clickable links.
Updated by livdywan over 3 years ago
I thought we agreed to brainstorm and collect ideas here before jumping right into code when the three of us were discussing it 😉
Although I like the idea of having a text summary.
Updated by livdywan over 3 years ago
okurz wrote:
Also I moved the existing acceptance criteria into suggestions because I think it is actually not feasible for us to achieve AC1. To my understanding a gitlab CI pipeline can not dynamically define new jobs that would represent failed results. Also the optional junit part should not be an acceptance criterion if it is optional.
@cdywan WDYT?
So far JUnit had been the only proposed output. With the exception of the text summary by way of a proof of concept @mkittler now proposed.
I don't get the part about dynamically defining jobs.
Updated by okurz over 3 years ago
cdywan wrote:
I thought we agreed to brainstorm and collect ideas here before jumping right into code when the three of us were discussing it 😉
Yeah, I told mkittler of for that already ;)
Updated by mkittler over 3 years ago
- Assignee deleted (
mkittler)
I put the summary in and with https://github.com/os-autoinst/scripts/pull/40 it also works when the shutdown line can not be determined. Since the last GitLab update the links are also clickable. For me that's good enough for now so I'm unassigning.
Updated by okurz over 3 years ago
- Status changed from New to Resolved
- Assignee set to mkittler
@cdywan I hope you agree that we basically achieved our main goal by being lucky and having received gitlab 13.5 just in time to have clickable URLs. Feel free to disagree, add more details to the ticket and reopen however I think currently that we are good.