Project

General

Profile

Actions

action #91356

closed

openQA Project - coordination #39719: [saga][epic] Detection of "known failures" for stable tests, easy test results review and easy tracking of known issues

action #52655: [epic] Move openqa-review from cron-jobs on lord.arch to a more sustainable long-term solution

Save openqa-review reports as gitlab CI artifacts

Added by okurz almost 3 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Target version:
Start date:
2021-04-19
Due date:
% Done:

0%

Estimated time:

Description

Motivation

Within #52655 we plan to move from okurz's personal cron jobs on lord.arch to a generic solution for running openqa-review and publishing the test overview report documents. https://gitlab.suse.de/openqa/openqa-review/-/pipeline_schedules already runs openqa-review on a regular base, we just need a way to store the generated HTML documents so that they are accessible as referenced so far in https://progress.opensuse.org/projects/qa/wiki#Test-results-overview

Acceptance criteria

  • AC1: The html document generated by openqa-review is stored in a webpage accessible by other users

Suggestions

  • Research how gitlab CI uses artifacts and configure the gitlab CI configuration accordingly
  • Update short links to the gitlab CI permalinks or copy the files within the gitlab CI jobs to a persistent path

Related issues 1 (0 open1 closed)

Copied to QA - action #93710: Reference individual openqa-review reports in gitlab CI artifacts, e.g. using gitlab pagesResolvedlivdywan

Actions
Actions #1

Updated by livdywan almost 3 years ago

Some thoughts on this:

  • Accessible could mean simply via gitlab? So maybe the GitLab Pages mechanism might be the easiest option
  • As results are linked on the public wiki, maybe they should be exposed outside the VPN?
  • openqa_suse_de_status.html is written by bin/openqa-review-daily-email via .gitlab-ci.yml
Actions #2

Updated by okurz almost 3 years ago

cdywan wrote:

Some thoughts on this:

  • Accessible could mean simply via gitlab? So maybe the GitLab Pages mechanism might be the easiest option

Well, I think it can be even easier with just https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html#access-the-latest-job-artifacts-by-url

  • As results are linked on the public wiki, maybe they should be exposed outside the VPN?

That's out of scope for the current story.

  • openqa_suse_de_status.html is written by bin/openqa-review-daily-email via .gitlab-ci.yml

correct :)

Actions #3

Updated by osukup almost 3 years ago

  • Assignee set to osukup

i'll try

Actions #4

Updated by livdywan almost 3 years ago

osukup wrote:

i'll try

Do you have a branch? Iirc you mentioned it on Rocket.Chat but no update here yet

Actions #5

Updated by osukup almost 3 years ago

  • Status changed from Workable to In Progress

@okurz for render as html artifact needs enabled Gitlab Pages in project. And it should automatically render saved artifact

Actions #6

Updated by openqa_review almost 3 years ago

  • Due date set to 2021-05-27

Setting due date based on mean cycle time of SUSE QE Tools

Actions #7

Updated by okurz almost 3 years ago

osukup wrote:

@okurz for render as html artifact needs enabled Gitlab Pages in project. And it should automatically render saved artifact

True. But AFAICS in https://gitlab.suse.de/openqa/openqa-review/pages there is nothing to "enable" but one would need to add a domain. I don't know if gitlab pages is enabled for the instance gitlab.suse.de and if yes, what domain we can or should use. Also, as you are a member of the parent project scope https://gitlab.suse.de/openqa/ can you access https://gitlab.suse.de/openqa/openqa-review/pages ? should I need to add you as owner/maintainer in the project itself to check?

If the plain html output files are not rendered as such directly from artifact URLs and if gitlab pages does not work for us then maybe we should scp the files to a host that can provide a web server, e.g. w3.nue.suse.com

Actions #8

Updated by osukup almost 3 years ago

okurz wrote:

osukup wrote:

@okurz for render as html artifact needs enabled Gitlab Pages in project. And it should automatically render saved artifact

True. But AFAICS in https://gitlab.suse.de/openqa/openqa-review/pages there is nothing to "enable" but one would need to add a domain. I don't know if gitlab pages is enabled for the instance gitlab.suse.de and if yes, what domain we can or should use. Also, as you are a member of the parent project scope https://gitlab.suse.de/openqa/ can you access https://gitlab.suse.de/openqa/openqa-review/pages ? should I need to add you as owner/maintainer in the project itself to check?

unfortuanetly I'm only developer ... so I can't set this and I dont see pages

Yes , in settings>pages it needs add domain and then it shows what to set for dns records needed for gitlab pages

If the plain html output files are not rendered as such directly from artifact URLs and if gitlab pages does not work for us then maybe we should scp the files to a host that can provide a web server, e.g. w3.nue.suse.com

Actions #9

Updated by okurz almost 3 years ago

I bumped your permissions in https://gitlab.suse.de/groups/openqa/-/group_members so feel free to experiment with gitlab pages and a domain for openqa-review.

You also referenced https://gitlab.suse.de/OPS-Service/salt/-/merge_requests/1780 for DNS entries.

openqa-review has an entry for http://openqa-review.suse.de/ which so far yields error 301 with no content behind. Maybe we need for the next CI job to publish something. The jobs already store the html document as artifact, e.g. https://gitlab.suse.de/openqa/openqa-review/-/jobs/428496/artifacts/file/openqa_suse_de_status.html

For proper display we also need https://github.com/os-autoinst/openqa_review/tree/master/dashboard_files published on the "gitlab pages"

Actions #10

Updated by osukup almost 3 years ago

--> looks like Gitlab Pages aren't working in our instance correctly --> http://openqa-review.suse.de redirects in all cases to localhost :(

--> plan ask infra + @jbaier_cz what with this

or use plan b) SCP artifact outside gitlab to publish on different server

Actions #11

Updated by livdywan almost 3 years ago

osukup wrote:

--> looks like Gitlab Pages aren't working in our instance correctly --> http://openqa-review.suse.de redirects in all cases to localhost :(

--> plan ask infra + @jbaier_cz what with this

Brief reminder on the recommended workflow:

For EngInfra tickets first create tracker ticket in https://progress.opensuse.org/projects/openqa-infrastructure/issues/ , then create EngInfra ticket with "[openqa] …" in subject, optional "[openqa][urgent] …", reference progress ticket, CC osd-admins@suse.de . Use the tracker ticket for internal notes

Actions #13

Updated by livdywan almost 3 years ago

  • Due date changed from 2021-05-27 to 2021-06-04
  • Status changed from In Progress to Blocked

This is probably Blocked for now, some conversation happening in the infra ticket.

Actions #15

Updated by livdywan almost 3 years ago

osukup wrote:

for now this link works: https://openqa.io.suse.de/openqa-review/openqa_suse_de_status.html

I get ERR_TOO_MANY_REDIRECTS there

Actions #16

Updated by osukup almost 3 years ago

cdywan wrote:

osukup wrote:

for now this link works: https://openqa.io.suse.de/openqa-review/openqa_suse_de_status.html

I get ERR_TOO_MANY_REDIRECTS there

hmm, something new, few days ago it worked. And @tinita updated ticket

Actions #17

Updated by okurz almost 3 years ago

  • Status changed from Blocked to In Progress

osukup wrote:

for now this link works: https://openqa.io.suse.de/openqa-review/openqa_suse_de_status.html

https://infra.nue.suse.com/SelfService/Display.html?id=189390 still open but
works for me right now but is missing the CSS data from https://github.com/os-autoinst/openqa_review/tree/master/dashboard_files so work can be continued for that part at least.

Actions #18

Updated by osukup almost 3 years ago

  • Status changed from In Progress to Blocked

https://openqa.io.suse.de/openqa-review/ -> with CSS etc, so for now to blocked

Actions #19

Updated by okurz almost 3 years ago

this looks awesome and it also seems have proper https. So what is actually that you want to have on top? If there is nothing important left just call it done because the acceptance criterion is fulfilled? We want to continue with gitlab pages in further work in the epic anyway, e.g. to reference individual reports.

Actions #20

Updated by okurz almost 3 years ago

  • Copied to action #93710: Reference individual openqa-review reports in gitlab CI artifacts, e.g. using gitlab pages added
Actions #21

Updated by osukup almost 3 years ago

okurz wrote:

this looks awesome and it also seems have proper https. So what is actually that you want to have on top?

own custom domain, as you see it uses https://GROUP.io.suse.de/REPO/ --> we have in dns prepared https://openqa-review.suse.de

Actions #22

Updated by osukup almost 3 years ago

modified to publish only osd run

Actions #23

Updated by okurz almost 3 years ago

are you also getting reminders? Do we need to escalate https://infra.nue.suse.com/SelfService/Display.html?id=189390 since there was no reaction since 21 days?

Actions #24

Updated by livdywan over 2 years ago

  • Due date changed from 2021-06-04 to 2021-07-02

okurz wrote:

are you also getting reminders? Do we need to escalate https://infra.nue.suse.com/SelfService/Display.html?id=189390 since there was no reaction since 21 days?

Answered in Rocket.Chat. @osukup got no reminder emails. I also asked for an update on the SD ticket.

Actions #25

Updated by okurz over 2 years ago

please split the "use another custom domain" part into a separate ticket to wait for the SUSE-IT SD ticket. I consider the original ACs done and would for now go with what we have, i.e. https://openqa.io.suse.de/openqa-review/

Actions #26

Updated by okurz over 2 years ago

Same as for parent ticket moving to "QA" project as I would like to simplify access to others even though this is a "SUSE-internal task".

Actions #27

Updated by livdywan over 2 years ago

  • Due date changed from 2021-07-02 to 2021-07-07

Moving due date due to public holidays

Actions #28

Updated by okurz over 2 years ago

  • Due date changed from 2021-07-07 to 2021-07-02

@cdywan a public holiday should not "surprise" us and is a bad reason to bump our time expectation. As I already explained in #91356#note-25 before the due-date has passed there as in easy way to call the story done. If osukup does not want to do that then anyone else in the team can do that as well and should do before a due-date has passed.

Actions #29

Updated by okurz over 2 years ago

  • Status changed from Blocked to Resolved

See #91356#note-25 . If the custom domain is not feasible we don't need it. https://openqa.io.suse.de/openqa-review/ is good enough.

Actions #30

Updated by okurz over 2 years ago

  • Due date deleted (2021-07-02)
Actions #31

Updated by osukup over 2 years ago

eng infra for custom domain tracking https://jira.suse.com/browse/ENGINFRA-724

Actions

Also available in: Atom PDF