action #48641
coordination #58184: [saga][epic][use case] full version control awareness within openQA, e.g. user forks and branches, fully versioned test schedules and configuration settings
[epic] Trigger openQA tests in pull requests of any product github pull request
25%
Description
User Story¶
As a developer of any software on github I want to execute an openQA test on a production server based on a pull request of my software to not need a local openQA instance
Further details¶
okurz:
Had a nice discussion with trenn: https://github.com/cobbler/cobbler/pull/2024 is a nice example of what one would desire to do in openQA instead, e.g.
- pull request in github triggers openQA test
- openQA test executes a custom test defined in the github source repo of the product under test (cobbler in this case)
- openQA test only boots a VM image, e.g. Tumbleweed, and executes the custom module
- test result is fed back to the github PR
implementation suggestions:
- could be done with polling bot for now
- using latest os-autoinst-distri-opensuse plus – as custom assets currently do not work – a custom test module that same what we tried already for HPC gets the defined test script, could be anything downloadable and executable, and runs it
- use
SCHEDULEparameter with e.g.SCHEDULE=tests/boot/boot_to_desktop,tests/run_custom - github API access relying on test variables previously provided on trigger
Subtasks
Related issues
History
#1
Updated by okurz about 2 years ago
Updated by - Copied from action #44327: Trigger tests based on git refspec/branch added
#2
Updated by okurz over 1 year ago
- Parent task set to #58184
#3
Updated by mkittler over 1 year ago
Updated by - This use-case is also impaired by the inability to use custom needles (https://progress.opensuse.org/issues/56789 #56789).
- Besides, the implementation suggestion
use SCHEDULE parameter with e.g. SCHEDULE=tests/boot/boot_to_desktop,tests/run_customimplies that a composability of test distributions would be required. This is of course possible by somehow embedding the base test distribution (e.g. os-autoinst-distri-opensuse) into the product repository (e.g. cobbler) adding own tests on top of it. But this sounds rather hacky and inconvenient to use. So having a light test distribution which can simply be referred to as "base test distribution" seems a desirable for this use-case.
These points are likely the tricky part. Triggering the test execution itself is likely not that hard. Instead of writing a "polling bot" I would add an API route in openQA which can be added as GitHub hook.
#4
Updated by okurz about 1 year ago
mkittler wrote:
- This use-case is also impaired by the inability to use custom needles (https://progress.opensuse.org/issues/56789 #56789).
Maybe we can still consider any need for needles an independant requirement of the part about triggering test code which I find more important.
Regarding the "hacky approach" or "base test distribution" I think we have a ticket for having something like a "linux" middleware between os-autoinst and os-autoinst-distri-opensuse that can be extracted but for now and to have a proof-of-concept on which to improve upon I recommend the "hacky" approach
[…] I would add an API route in openQA which can be added as GitHub hook.
What would you add as an API route here?
#7
Updated by okurz 6 months ago
- Related to coordination #77698: [epic] synchronous qemu based system level test in pull request CI runs, e.g. standalone isotovideo or openQA tests added