Project

General

Profile

action #91356

openQA Project - coordination #39719: [saga][epic] Detection of "known failures" for stable tests, easy test results review and easy tracking of known issues

action #52655: [epic] Move openqa-review from cron-jobs on lord.arch to a more sustainable long-term solution

Save openqa-review reports as gitlab CI artifacts

Added by okurz 8 months ago. Updated 4 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Target version:
Start date:
2021-04-19
Due date:
% Done:

0%

Estimated time:

Description

Motivation

Within #52655 we plan to move from okurz's personal cron jobs on lord.arch to a generic solution for running openqa-review and publishing the test overview report documents. https://gitlab.suse.de/openqa/openqa-review/-/pipeline_schedules already runs openqa-review on a regular base, we just need a way to store the generated HTML documents so that they are accessible as referenced so far in https://progress.opensuse.org/projects/qa/wiki#Test-results-overview

Acceptance criteria

  • AC1: The html document generated by openqa-review is stored in a webpage accessible by other users

Suggestions

  • Research how gitlab CI uses artifacts and configure the gitlab CI configuration accordingly
  • Update short links to the gitlab CI permalinks or copy the files within the gitlab CI jobs to a persistent path

Related issues

Copied to QA - action #93710: Reference individual openqa-review reports in gitlab CI artifacts, e.g. using gitlab pagesResolved

History

#1 Updated by cdywan 8 months ago

Some thoughts on this:

  • Accessible could mean simply via gitlab? So maybe the GitLab Pages mechanism might be the easiest option
  • As results are linked on the public wiki, maybe they should be exposed outside the VPN?
  • openqa_suse_de_status.html is written by bin/openqa-review-daily-email via .gitlab-ci.yml

#2 Updated by okurz 8 months ago

cdywan wrote:

Some thoughts on this:

  • Accessible could mean simply via gitlab? So maybe the GitLab Pages mechanism might be the easiest option

Well, I think it can be even easier with just https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html#access-the-latest-job-artifacts-by-url

  • As results are linked on the public wiki, maybe they should be exposed outside the VPN?

That's out of scope for the current story.

  • openqa_suse_de_status.html is written by bin/openqa-review-daily-email via .gitlab-ci.yml

correct :)

#3 Updated by osukup 7 months ago

  • Assignee set to osukup

i'll try

#4 Updated by cdywan 7 months ago

osukup wrote:

i'll try

Do you have a branch? Iirc you mentioned it on Rocket.Chat but no update here yet

#5 Updated by osukup 7 months ago

  • Status changed from Workable to In Progress

okurz for render as html artifact needs enabled Gitlab Pages in project. And it should automatically render saved artifact

#6 Updated by openqa_review 7 months ago

  • Due date set to 2021-05-27

Setting due date based on mean cycle time of SUSE QE Tools

#7 Updated by okurz 7 months ago

osukup wrote:

okurz for render as html artifact needs enabled Gitlab Pages in project. And it should automatically render saved artifact

True. But AFAICS in https://gitlab.suse.de/openqa/openqa-review/pages there is nothing to "enable" but one would need to add a domain. I don't know if gitlab pages is enabled for the instance gitlab.suse.de and if yes, what domain we can or should use. Also, as you are a member of the parent project scope https://gitlab.suse.de/openqa/ can you access https://gitlab.suse.de/openqa/openqa-review/pages ? should I need to add you as owner/maintainer in the project itself to check?

If the plain html output files are not rendered as such directly from artifact URLs and if gitlab pages does not work for us then maybe we should scp the files to a host that can provide a web server, e.g. w3.nue.suse.com

#8 Updated by osukup 7 months ago

okurz wrote:

osukup wrote:

okurz for render as html artifact needs enabled Gitlab Pages in project. And it should automatically render saved artifact

True. But AFAICS in https://gitlab.suse.de/openqa/openqa-review/pages there is nothing to "enable" but one would need to add a domain. I don't know if gitlab pages is enabled for the instance gitlab.suse.de and if yes, what domain we can or should use. Also, as you are a member of the parent project scope https://gitlab.suse.de/openqa/ can you access https://gitlab.suse.de/openqa/openqa-review/pages ? should I need to add you as owner/maintainer in the project itself to check?

unfortuanetly I'm only developer ... so I can't set this and I dont see pages

Yes , in settings>pages it needs add domain and then it shows what to set for dns records needed for gitlab pages

If the plain html output files are not rendered as such directly from artifact URLs and if gitlab pages does not work for us then maybe we should scp the files to a host that can provide a web server, e.g. w3.nue.suse.com

#9 Updated by okurz 7 months ago

I bumped your permissions in https://gitlab.suse.de/groups/openqa/-/group_members so feel free to experiment with gitlab pages and a domain for openqa-review.

You also referenced https://gitlab.suse.de/OPS-Service/salt/-/merge_requests/1780 for DNS entries.

openqa-review has an entry for http://openqa-review.suse.de/ which so far yields error 301 with no content behind. Maybe we need for the next CI job to publish something. The jobs already store the html document as artifact, e.g. https://gitlab.suse.de/openqa/openqa-review/-/jobs/428496/artifacts/file/openqa_suse_de_status.html

For proper display we also need https://github.com/os-autoinst/openqa_review/tree/master/dashboard_files published on the "gitlab pages"

#10 Updated by osukup 6 months ago

--> looks like Gitlab Pages aren't working in our instance correctly --> http://openqa-review.suse.de redirects in all cases to localhost :(

--> plan ask infra + jbaier_cz what with this

or use plan b) SCP artifact outside gitlab to publish on different server

#11 Updated by cdywan 6 months ago

osukup wrote:

--> looks like Gitlab Pages aren't working in our instance correctly --> http://openqa-review.suse.de redirects in all cases to localhost :(

--> plan ask infra + jbaier_cz what with this

Brief reminder on the recommended workflow:

For EngInfra tickets first create tracker ticket in https://progress.opensuse.org/projects/openqa-infrastructure/issues/ , then create EngInfra ticket with "[openqa] …" in subject, optional "[openqa][urgent] …", reference progress ticket, CC osd-admins@suse.de . Use the tracker ticket for internal notes

#13 Updated by cdywan 6 months ago

  • Due date changed from 2021-05-27 to 2021-06-04
  • Status changed from In Progress to Blocked

This is probably Blocked for now, some conversation happening in the infra ticket.

#15 Updated by cdywan 6 months ago

osukup wrote:

for now this link works: https://openqa.io.suse.de/openqa-review/openqa_suse_de_status.html

I get ERR_TOO_MANY_REDIRECTS there

#16 Updated by osukup 6 months ago

cdywan wrote:

osukup wrote:

for now this link works: https://openqa.io.suse.de/openqa-review/openqa_suse_de_status.html

I get ERR_TOO_MANY_REDIRECTS there

hmm, something new, few days ago it worked. And tinita updated ticket

#17 Updated by okurz 6 months ago

  • Status changed from Blocked to In Progress

osukup wrote:

for now this link works: https://openqa.io.suse.de/openqa-review/openqa_suse_de_status.html

https://infra.nue.suse.com/SelfService/Display.html?id=189390 still open but
works for me right now but is missing the CSS data from https://github.com/os-autoinst/openqa_review/tree/master/dashboard_files so work can be continued for that part at least.

#18 Updated by osukup 6 months ago

  • Status changed from In Progress to Blocked

https://openqa.io.suse.de/openqa-review/ -> with CSS etc, so for now to blocked

#19 Updated by okurz 6 months ago

this looks awesome and it also seems have proper https. So what is actually that you want to have on top? If there is nothing important left just call it done because the acceptance criterion is fulfilled? We want to continue with gitlab pages in further work in the epic anyway, e.g. to reference individual reports.

#20 Updated by okurz 6 months ago

  • Copied to action #93710: Reference individual openqa-review reports in gitlab CI artifacts, e.g. using gitlab pages added

#21 Updated by osukup 6 months ago

okurz wrote:

this looks awesome and it also seems have proper https. So what is actually that you want to have on top?

own custom domain, as you see it uses https://GROUP.io.suse.de/REPO/ --> we have in dns prepared https://openqa-review.suse.de

#22 Updated by osukup 6 months ago

modified to publish only osd run

#23 Updated by okurz 5 months ago

are you also getting reminders? Do we need to escalate https://infra.nue.suse.com/SelfService/Display.html?id=189390 since there was no reaction since 21 days?

#24 Updated by cdywan 5 months ago

  • Due date changed from 2021-06-04 to 2021-07-02

okurz wrote:

are you also getting reminders? Do we need to escalate https://infra.nue.suse.com/SelfService/Display.html?id=189390 since there was no reaction since 21 days?

Answered in Rocket.Chat. osukup got no reminder emails. I also asked for an update on the SD ticket.

#25 Updated by okurz 5 months ago

please split the "use another custom domain" part into a separate ticket to wait for the SUSE-IT SD ticket. I consider the original ACs done and would for now go with what we have, i.e. https://openqa.io.suse.de/openqa-review/

#26 Updated by okurz 5 months ago

Same as for parent ticket moving to "QA" project as I would like to simplify access to others even though this is a "SUSE-internal task".

#27 Updated by cdywan 5 months ago

  • Due date changed from 2021-07-02 to 2021-07-07

Moving due date due to public holidays

#28 Updated by okurz 5 months ago

  • Due date changed from 2021-07-07 to 2021-07-02

cdywan a public holiday should not "surprise" us and is a bad reason to bump our time expectation. As I already explained in #91356#note-25 before the due-date has passed there as in easy way to call the story done. If osukup does not want to do that then anyone else in the team can do that as well and should do before a due-date has passed.

#29 Updated by okurz 5 months ago

  • Status changed from Blocked to Resolved

See #91356#note-25 . If the custom domain is not feasible we don't need it. https://openqa.io.suse.de/openqa-review/ is good enough.

#30 Updated by okurz 5 months ago

  • Due date deleted (2021-07-02)

#31 Updated by osukup 4 months ago

eng infra for custom domain tracking https://jira.suse.com/browse/ENGINFRA-724

Also available in: Atom PDF