Project

General

Profile

Actions

action #126950

closed

coordination #58184: [saga][epic][use case] full version control awareness within openQA

coordination #48641: [epic] Trigger openQA tests in pull requests of any product github pull request

[openQA-in-openQA] openQA tests in pull requests to github.com/os-autoinst/os-autoinst-distri-openQA/ size:M

Added by okurz 12 months ago. Updated 10 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Start date:
2023-03-30
Due date:
% Done:

0%

Estimated time:

Description

Motivation

As http://open.qa/docs/#_running_openqa_jobs_as_ci_checks now explains how openQA jobs can be used as part of CI workflows we should use that within github.com/os-autoinst/os-autoinst-distri-openQA/ so that whenever we create changes to the distribution we benefit and also consume our own dogfood.

Acceptance criteria

  • AC1: New pull requests in github.com/os-autoinst/os-autoinst-distri-openQA/ provide feedback to the pull request creator and reviewers from all openQA-in-openQA tests executed for the pull request before merge

Suggestions


Related issues 2 (0 open2 closed)

Blocked by openQA Project - action #128360: Supporting fork based development model size:MResolvedmkittler

Actions
Copied to openQA Project - action #150992: [timeboxed][spike solution:20h] openQA tests in pull requests to github.com/os-autoinst/os-autoinst-distri-opensuse/ size:MResolvedmkittler

Actions
Actions #1

Updated by livdywan 12 months ago

  • Subject changed from [openQA-in-openQA] openQA tests in pull requests to github.com/os-autoinst/os-autoinst-distri-openQA/ to [openQA-in-openQA] openQA tests in pull requests to github.com/os-autoinst/os-autoinst-distri-openQA/ size:M
  • Description updated (diff)
  • Status changed from New to Workable
Actions #2

Updated by jbaier_cz 12 months ago

  • Status changed from Workable to In Progress
  • Assignee set to jbaier_cz

I will take this as a learning experience.

Actions #3

Updated by openqa_review 12 months ago

  • Due date set to 2023-05-02

Setting due date based on mean cycle time of SUSE QE Tools

Actions #4

Updated by jbaier_cz 11 months ago

I drafted https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/114 which should schedule openqa_from_git test inside a pull request, one such attempt can be seen here or also inside the pull request itself: https://openqa.opensuse.org/tests/3235605

What is still missing is to add correct HDD_1 (which I can probably take from the Jenkins job scheduling the "regular" tests) and also use the pull request branch inside the test itself (or maybe create a different test suite for that?).

Actions #5

Updated by jbaier_cz 11 months ago

  • Status changed from In Progress to Feedback

After some tinkering I was able to find out the proper setting. I learn a fer key aspects:

  1. You do not need scenario definitions, unless you need to override group id (scenarios defined in the job group)
  2. Keys/values in scenario definitions (i.e. job templates) are slightly different than in job groups
  3. version: "*" does not work as expected
  4. distri: openqa has to be in lowercase or it does not match (on the other hand, one can use DISTRI=openQA in the call and it will still work)
Actions #6

Updated by livdywan 11 months ago

  • Due date changed from 2023-05-02 to 2023-05-12

A sporadic failure occurred in one of the test runs (https://openqa.opensuse.org/tests/3238356#step/start_test/9). The CI behaves as expected but we should have a ticket about the issue before resolving this.

The mentioned learnings above would also be good to add to existing docs, or file tickets if these are considered bugs. At least to avoid users running into these.

Actions #7

Updated by livdywan 11 months ago

  • Due date changed from 2023-05-12 to 2023-05-05
Actions #8

Updated by jbaier_cz 11 months ago

Does not look sporadic to me, after a force push a new test was created with the very same error: https://openqa.opensuse.org/tests/3248361, maybe I have an error in the job configuration?

Actions #9

Updated by jbaier_cz 11 months ago

All test issues solved, the request is ready to be merged.

Actions #10

Updated by jbaier_cz 11 months ago

  • Blocked by action #128360: Supporting fork based development model size:M added
Actions #11

Updated by okurz 11 months ago

  • Due date deleted (2023-05-05)
  • Status changed from Feedback to Blocked
Actions #12

Updated by livdywan 10 months ago

We'll still consider this blocked for now, as Marius is enabling the "fork" based approach in the example distri first, then we can have a new version of https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/114

Actions #13

Updated by jbaier_cz 10 months ago

As explained it the PR, the webhook approach might not work in this case.

Actions #14

Updated by okurz 10 months ago

  • Status changed from Blocked to Workable

#128360 was resolved so one way or another this should now continue :)

Actions #15

Updated by jbaier_cz 10 months ago

  • Status changed from Workable to Feedback

PR with action merged, the action should be triggered on pull_request_target which should in theory work well with fork-based development

Actions #16

Updated by jbaier_cz 10 months ago

  • Status changed from Feedback to Resolved
Actions #18

Updated by okurz 4 months ago

  • Copied to action #150992: [timeboxed][spike solution:20h] openQA tests in pull requests to github.com/os-autoinst/os-autoinst-distri-opensuse/ size:M added
Actions

Also available in: Atom PDF