coordination #58184: [saga][epic][use case] full version control awareness within openQA, e.g. user forks and branches, fully versioned test schedules and configuration settings
[epic] Trigger openQA tests in pull requests of any product github pull request
As a developer of any software on github I want to execute an openQA test on a production server based on a pull request of my software to not need a local openQA instance
Had a nice discussion with trenn: https://github.com/cobbler/cobbler/pull/2024 is a nice example of what one would desire to do in openQA instead, e.g.
- pull request in github triggers openQA test
- openQA test executes a custom test defined in the github source repo of the product under test (cobbler in this case)
- openQA test only boots a VM image, e.g. Tumbleweed, and executes the custom module
- test result is fed back to the github PR
- could be done with polling bot for now
- using latest os-autoinst-distri-opensuse plus – as custom assets currently do not work – a custom test module that same what we tried already for HPC gets the defined test script, could be anything downloadable and executable, and runs it
SCHEDULEparameter with e.g.
- github API access relying on test variables previously provided on trigger
#3 Updated by mkittler over 1 year ago
- This use-case is also impaired by the inability to use custom needles (https://progress.opensuse.org/issues/56789 #56789).
- Besides, the implementation suggestion
use SCHEDULE parameter with e.g. SCHEDULE=tests/boot/boot_to_desktop,tests/run_customimplies that a composability of test distributions would be required. This is of course possible by somehow embedding the base test distribution (e.g. os-autoinst-distri-opensuse) into the product repository (e.g. cobbler) adding own tests on top of it. But this sounds rather hacky and inconvenient to use. So having a light test distribution which can simply be referred to as "base test distribution" seems a desirable for this use-case.
These points are likely the tricky part. Triggering the test execution itself is likely not that hard. Instead of writing a "polling bot" I would add an API route in openQA which can be added as GitHub hook.
Maybe we can still consider any need for needles an independant requirement of the part about triggering test code which I find more important.
Regarding the "hacky approach" or "base test distribution" I think we have a ticket for having something like a "linux" middleware between os-autoinst and os-autoinst-distri-opensuse that can be extracted but for now and to have a proof-of-concept on which to improve upon I recommend the "hacky" approach
[…] I would add an API route in openQA which can be added as GitHub hook.
What would you add as an API route here?