Project

General

Profile

action #73366

auto-review: Improve output

Added by cdywan 3 months ago. Updated 3 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Start date:
2020-10-14
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Motivation

We run auto_review aka openqa_review in GitLab CI. The output of the script can be seen in the log of the pipeline, which can be hard to read because one has to find the relevant snippets first within the verbose output that we currently use.

Acceptance Criteria

TBD

Suggestions

  • Look into pipelines on https://gitlab.suse.de/openqa/openqa-review/ as part of the Monitor step in our DevOps workflow
  • GitLab CI has support for JUnit artifacts i.e. XML files that contain sections for passed or failed tests. Those could be (ab)used to list the found jobs. The script just needs to add the XML boilerplate and write to a file (this is the same concept we use with CircleCI to render TAP output).
  • Found jobs could be shown as pipeline failures in GitLab CI w/o going into the verbose logs
  • JUnit output is optional and the script output remains available for e.g. local usage

History

#1 Updated by cdywan 3 months ago

  • Description updated (diff)
  • Category set to Feature requests

#2 Updated by okurz 3 months ago

  • Subject changed from Implement JUnit output in auto_review to auto-review: Improve output
  • Description updated (diff)
  • Target version set to Ready

Hi, I changed the description to better conform to the template of feature requests instead of defects :)

In particular I changed the subject line to not prescribe the implementation but leave it open to the developer to choose how to cover the user story.

Also I moved the existing acceptance criteria into suggestions because I think it is actually not feasible for us to achieve AC1. To my understanding a gitlab CI pipeline can not dynamically define new jobs that would represent failed results. Also the optional junit part should not be an acceptance criterion if it is optional.

cdywan WDYT?

#3 Updated by mkittler 3 months ago

  • Assignee set to mkittler

#4 Updated by mkittler 3 months ago

PR for printing a summary of jobs to be reviewed at the end: https://github.com/os-autoinst/scripts/pull/38

That's just a simple change but already a huge improvement as one no longer needs to go though all the verbose output. I made it work with set -x by using only a single echo. Unfortunately the links will still not be clickable from GitLab's CI output. I suppose using JUnit would only be beneficial if it leads to clickable links.

#5 Updated by cdywan 3 months ago

I thought we agreed to brainstorm and collect ideas here before jumping right into code when the three of us were discussing it 😉

Although I like the idea of having a text summary.

#6 Updated by cdywan 3 months ago

okurz wrote:

Also I moved the existing acceptance criteria into suggestions because I think it is actually not feasible for us to achieve AC1. To my understanding a gitlab CI pipeline can not dynamically define new jobs that would represent failed results. Also the optional junit part should not be an acceptance criterion if it is optional.

cdywan WDYT?

So far JUnit had been the only proposed output. With the exception of the text summary by way of a proof of concept @MKittler now proposed.

I don't get the part about dynamically defining jobs.

#7 Updated by okurz 3 months ago

cdywan wrote:

I thought we agreed to brainstorm and collect ideas here before jumping right into code when the three of us were discussing it 😉

Yeah, I told mkittler of for that already ;)

#8 Updated by mkittler 3 months ago

  • Assignee deleted (mkittler)

I put the summary in and with https://github.com/os-autoinst/scripts/pull/40 it also works when the shutdown line can not be determined. Since the last GitLab update the links are also clickable. For me that's good enough for now so I'm unassigning.

#9 Updated by okurz 3 months ago

  • Status changed from New to Resolved
  • Assignee set to mkittler

cdywan I hope you agree that we basically achieved our main goal by being lucky and having received gitlab 13.5 just in time to have clickable URLs. Feel free to disagree, add more details to the ticket and reopen however I think currently that we are good.

Also available in: Atom PDF