QA » openQA Project

User Story¶

As a developer of any software on github I want to execute an openQA test on a production server based on a pull request of my software to not need a local openQA instance

Further details¶

okurz:
Had a nice discussion with trenn: https://github.com/cobbler/cobbler/pull/2024 is a nice example of what one would desire to do in openQA instead, e.g.

1. pull request in github triggers openQA test
2. openQA test executes a custom test defined in the github source repo of the product under test (cobbler in this case)
3. openQA test only boots a VM image, e.g. Tumbleweed, and executes the custom module
4. test result is fed back to the github PR

implementation suggestions:

1. could be done with polling bot for now
2. using latest os-autoinst-distri-opensuse plus – as custom assets currently do not work – a custom test module that same what we tried already for HPC gets the defined test script, could be anything downloadable and executable, and runs it
3. use SCHEDULE parameter with e.g. SCHEDULE=tests/boot/boot_to_desktop,tests/run_custom
4. github API access relying on test variables previously provided on trigger

Further ideas¶

With #128360 a webhook-based approach has been implemented. The following aspects of it could still be improved:

1. Better scheduled product pages: The PR check "Details" link leads to the page of the related scheduled product. It would make sense to list all jobs belonging to that scheduled product there in a nice table so one can easily access the jobs. It would also be great if it would show the latest jobs in the restart/clone chain (and not just the initial jobs). Maybe we could also show the overall status of the scheduled product reusing the code we use to report it to the CI check. Note that this would also be an improvement in general as it helps not just for the CI use case but when dealing with scheduled products in general.
2. Better error handling when reporting back: possibly add a retry, add an audit event in case something goes wrong
3. Finer/better permissions: So far we require an API key/secret pair where the associated user is at least operator. That already includes many permissions. We could allow the creation of API key/secret pairs that fewer permissions.
4. Have an extra token as secret for signing of the webhook payload: This extra token could be generated for the used API key/secret pair so the secret for signing would not just basically be the API user/credentials again.