Project

General

Profile

Actions
Copy link

action #126950

closed

coordination #58184: [saga][epic][use case] full version control awareness within openQA

coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull request

[openQA-in-openQA] openQA tests in pull requests to github.com/os-autoinst/os-autoinst-distri-openQA/ size:M

Added by okurz over 1 year ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
jbaier_cz
Category:
Feature requests
Target version:
Ready
Start date:
2023-03-30
Due date:
% Done:

0%

Estimated time:

Description

Motivation

As http://open.qa/docs/#_running_openqa_jobs_as_ci_checks now explains how openQA jobs can be used as part of CI workflows we should use that within github.com/os-autoinst/os-autoinst-distri-openQA/ so that whenever we create changes to the distribution we benefit and also consume our own dogfood.

Acceptance criteria

  • AC1: New pull requests in github.com/os-autoinst/os-autoinst-distri-openQA/ provide feedback to the pull request creator and reviewers from all openQA-in-openQA tests executed for the pull request before merge

Suggestions

Related issues 2 (0 open — 2 closed)

Blocked by openQA Project (public) - action #128360: Supporting fork based development model size:MResolvedmkittler

Actions
Copied to openQA Project (public) - action #150992: [timeboxed][spike solution:20h] openQA tests in pull requests to github.com/os-autoinst/os-autoinst-distri-opensuse/ size:MResolvedmkittler

Actions
Actions
Copy link
 #1

Updated by livdywan over 1 year ago

  • Subject changed from [openQA-in-openQA] openQA tests in pull requests to github.com/os-autoinst/os-autoinst-distri-openQA/ to [openQA-in-openQA] openQA tests in pull requests to github.com/os-autoinst/os-autoinst-distri-openQA/ size:M
  • Description updated (diff)
  • Status changed from New to Workable
Actions
Copy link
 #2

Updated by jbaier_cz over 1 year ago

  • Status changed from Workable to In Progress
  • Assignee set to jbaier_cz

I will take this as a learning experience.

Actions
Copy link
 #3

Updated by openqa_review over 1 year ago

  • Due date set to 2023-05-02

Setting due date based on mean cycle time of SUSE QE Tools

Actions
Copy link
 #4

Updated by jbaier_cz over 1 year ago

I drafted https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/114 which should schedule openqa_from_git test inside a pull request, one such attempt can be seen here or also inside the pull request itself: https://openqa.opensuse.org/tests/3235605

What is still missing is to add correct HDD_1 (which I can probably take from the Jenkins job scheduling the "regular" tests) and also use the pull request branch inside the test itself (or maybe create a different test suite for that?).

Actions
Copy link
 #5

Updated by jbaier_cz over 1 year ago

  • Status changed from In Progress to Feedback

After some tinkering I was able to find out the proper setting. I learn a fer key aspects:

  1. You do not need scenario definitions, unless you need to override group id (scenarios defined in the job group)
  2. Keys/values in scenario definitions (i.e. job templates) are slightly different than in job groups
  3. version: "*" does not work as expected
  4. distri: openqa has to be in lowercase or it does not match (on the other hand, one can use DISTRI=openQA in the call and it will still work)
Actions
Copy link
 #6

Updated by livdywan over 1 year ago

  • Due date changed from 2023-05-02 to 2023-05-12

A sporadic failure occurred in one of the test runs (https://openqa.opensuse.org/tests/3238356#step/start_test/9). The CI behaves as expected but we should have a ticket about the issue before resolving this.

The mentioned learnings above would also be good to add to existing docs, or file tickets if these are considered bugs. At least to avoid users running into these.

Actions
Copy link
 #7

Updated by livdywan over 1 year ago

  • Due date changed from 2023-05-12 to 2023-05-05
Actions
Copy link
 #8

Updated by jbaier_cz over 1 year ago

Does not look sporadic to me, after a force push a new test was created with the very same error: https://openqa.opensuse.org/tests/3248361, maybe I have an error in the job configuration?

Actions
Copy link
 #9

Updated by jbaier_cz over 1 year ago

All test issues solved, the request is ready to be merged.

Actions
Copy link
 #10

Updated by jbaier_cz over 1 year ago

  • Blocked by action #128360: Supporting fork based development model size:M added
Actions
Copy link
 #11

Updated by okurz over 1 year ago

  • Due date deleted (2023-05-05)
  • Status changed from Feedback to Blocked
Actions
Copy link
 #12

Updated by livdywan over 1 year ago

We'll still consider this blocked for now, as Marius is enabling the "fork" based approach in the example distri first, then we can have a new version of https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/114

Actions
Copy link
 #13

Updated by jbaier_cz over 1 year ago

As explained it the PR, the webhook approach might not work in this case.

Actions
Copy link
 #14

Updated by okurz over 1 year ago

  • Status changed from Blocked to Workable

#128360 was resolved so one way or another this should now continue :)

Actions
Copy link
 #15

Updated by jbaier_cz over 1 year ago

  • Status changed from Workable to Feedback

PR with action merged, the action should be triggered on pull_request_target which should in theory work well with fork-based development

Actions
Copy link
 #16

Updated by jbaier_cz over 1 year ago

  • Status changed from Feedback to Resolved
Actions
Copy link
 #18

Updated by okurz about 1 year ago

  • Copied to action #150992: [timeboxed][spike solution:20h] openQA tests in pull requests to github.com/os-autoinst/os-autoinst-distri-opensuse/ size:M added
Actions
Copy link

Also available in: Atom PDF