coordination #58184: [saga][epic][use case] full version control awareness within openQA, e.g. user forks and branches, fully versioned test schedules and configuration settings
[epic] Trigger openQA tests in pull requests of any product github pull request
As a developer of any software on github I want to execute an openQA test on a production server based on a pull request of my software to not need a local openQA instance
Had a nice discussion with trenn: https://github.com/cobbler/cobbler/pull/2024 is a nice example of what one would desire to do in openQA instead, e.g.
- pull request in github triggers openQA test
- openQA test executes a custom test defined in the github source repo of the product under test (cobbler in this case)
- openQA test only boots a VM image, e.g. Tumbleweed, and executes the custom module
- test result is fed back to the github PR
- could be done with polling bot for now
- using latest os-autoinst-distri-opensuse plus – as custom assets currently do not work – a custom test module that same what we tried already for HPC gets the defined test script, could be anything downloadable and executable, and runs it
SCHEDULEparameter with e.g.
- github API access relying on test variables previously provided on trigger
#3 Updated by mkittler over 1 year ago
- This use-case is also impaired by the inability to use custom needles (https://progress.opensuse.org/issues/56789 #56789).
- Besides, the implementation suggestion
use SCHEDULE parameter with e.g. SCHEDULE=tests/boot/boot_to_desktop,tests/run_customimplies that a composability of test distributions would be required. This is of course possible by somehow embedding the base test distribution (e.g. os-autoinst-distri-opensuse) into the product repository (e.g. cobbler) adding own tests on top of it. But this sounds rather hacky and inconvenient to use. So having a light test distribution which can simply be referred to as "base test distribution" seems a desirable for this use-case.
These points are likely the tricky part. Triggering the test execution itself is likely not that hard. Instead of writing a "polling bot" I would add an API route in openQA which can be added as GitHub hook.
#4 Updated by okurz about 1 year ago
Maybe we can still consider any need for needles an independant requirement of the part about triggering test code which I find more important.
Regarding the "hacky approach" or "base test distribution" I think we have a ticket for having something like a "linux" middleware between os-autoinst and os-autoinst-distri-opensuse that can be extracted but for now and to have a proof-of-concept on which to improve upon I recommend the "hacky" approach
[…] I would add an API route in openQA which can be added as GitHub hook.
What would you add as an API route here?