coordination #58184: [saga][epic][use case] full version control awareness within openQA, e.g. user forks and branches, fully versioned test schedules and configuration settings
action #48641: [epic] Trigger openQA tests in pull requests of any product github pull request
[openQA-in-openQA] openQA tests in pull requests to github.com/os-autoinst/os-autoinst-distri-openQA/ size:M
As http://open.qa/docs/#_running_openqa_jobs_as_ci_checks now explains how openQA jobs can be used as part of CI workflows we should use that within github.com/os-autoinst/os-autoinst-distri-openQA/ so that whenever we create changes to the distribution we benefit and also consume our own dogfood.
- AC1: New pull requests in github.com/os-autoinst/os-autoinst-distri-openQA/ provide feedback to the pull request creator and reviewers from all openQA-in-openQA tests executed for the pull request before merge
- This will most likely not work in forks since GitHub doesn't allow secrets to be accessible to forks by design
- Follow the documentation and consider the existing use case in https://github.com/os-autoinst/os-autoinst-distri-example/blob/main/.github/workflows/openqa.yml
- Maybe it would be good for somebody with no prior experience with the triggering of openqa tests from GitHub to take this as a learning experience
#1 Updated by cdywan about 2 months ago
- Subject changed from [openQA-in-openQA] openQA tests in pull requests to github.com/os-autoinst/os-autoinst-distri-openQA/ to [openQA-in-openQA] openQA tests in pull requests to github.com/os-autoinst/os-autoinst-distri-openQA/ size:M
- Description updated (diff)
- Status changed from New to Workable
#2 Updated by jbaier_cz about 2 months ago
- Status changed from Workable to In Progress
- Assignee set to jbaier_cz
I will take this as a learning experience.
#3 Updated by openqa_review about 2 months ago
- Due date set to 2023-05-02
Setting due date based on mean cycle time of SUSE QE Tools
#4 Updated by jbaier_cz about 2 months ago
I drafted https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/114 which should schedule
openqa_from_git test inside a pull request, one such attempt can be seen here or also inside the pull request itself: https://openqa.opensuse.org/tests/3235605
What is still missing is to add correct HDD_1 (which I can probably take from the Jenkins job scheduling the "regular" tests) and also use the pull request branch inside the test itself (or maybe create a different test suite for that?).
#5 Updated by jbaier_cz about 2 months ago
- Status changed from In Progress to Feedback
After some tinkering I was able to find out the proper setting. I learn a fer key aspects:
- You do not need scenario definitions, unless you need to override group id (scenarios defined in the job group)
- Keys/values in scenario definitions (i.e. job templates) are slightly different than in job groups
version: "*"does not work as expected
distri: openqahas to be in lowercase or it does not match (on the other hand, one can use
DISTRI=openQAin the call and it will still work)
#6 Updated by cdywan about 1 month ago
- Due date changed from 2023-05-02 to 2023-05-12
A sporadic failure occurred in one of the test runs (https://openqa.opensuse.org/tests/3238356#step/start_test/9). The CI behaves as expected but we should have a ticket about the issue before resolving this.
The mentioned learnings above would also be good to add to existing docs, or file tickets if these are considered bugs. At least to avoid users running into these.
#7 Updated by cdywan about 1 month ago
- Due date changed from 2023-05-12 to 2023-05-05
#8 Updated by jbaier_cz about 1 month ago
Does not look sporadic to me, after a force push a new test was created with the very same error: https://openqa.opensuse.org/tests/3248361, maybe I have an error in the job configuration?
#9 Updated by jbaier_cz about 1 month ago
All test issues solved, the request is ready to be merged.
#10 Updated by jbaier_cz about 1 month ago
- Blocked by action #128360: Supporting fork based development model size:M added
#11 Updated by okurz about 1 month ago
- Due date deleted (
- Status changed from Feedback to Blocked
#12 Updated by cdywan 13 days ago
We'll still consider this blocked for now, as Marius is enabling the "fork" based approach in the example distri first, then we can have a new version of https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/114