Project

General

Profile

action #48641

coordination #58184: [saga][epic][use case] full version control awareness within openQA, e.g. user forks and branches, fully versioned test schedules and configuration settings

[epic] Trigger openQA tests in pull requests of any product github pull request

Added by okurz almost 2 years ago. Updated 7 days ago.

Status:
Blocked
Priority:
Low
Assignee:
Category:
Feature requests
Target version:
Start date:
2021-01-13
Due date:
% Done:

0%

Estimated time:
(Total: 0.00 h)
Difficulty:

Description

User Story

As a developer of any software on github I want to execute an openQA test on a production server based on a pull request of my software to not need a local openQA instance

Further details

okurz:
Had a nice discussion with trenn: https://github.com/cobbler/cobbler/pull/2024 is a nice example of what one would desire to do in openQA instead, e.g.

  1. pull request in github triggers openQA test
  2. openQA test executes a custom test defined in the github source repo of the product under test (cobbler in this case)
  3. openQA test only boots a VM image, e.g. Tumbleweed, and executes the custom module
  4. test result is fed back to the github PR

implementation suggestions:

  1. could be done with polling bot for now
  2. using latest os-autoinst-distri-opensuse plus – as custom assets currently do not work – a custom test module that same what we tried already for HPC gets the defined test script, could be anything downloadable and executable, and runs it
  3. use SCHEDULE parameter with e.g. SCHEDULE=tests/boot/boot_to_desktop,tests/run_custom
  4. github API access relying on test variables previously provided on trigger

Subtasks

action #86063: [epic] Add possibility to trigger openQA API calls, e.g. single "jobs", without the need of the client / over the webUI / with curlBlockedokurz

action #87698: openQA jobs with arbitrary parameters can be triggered over the webUI for authenticated users with right permissions (operator+)Workable

action #87695: [epic] full openQA test development, maintenance and administration from browser without the need of a local terminalNew


Related issues

Related to openQA Project - coordination #77698: [epic] synchronous qemu based system level test in pull request CI runs, e.g. standalone isotovideo or openQA testsNew2020-11-15

Copied from openQA Project - action #44327: Trigger tests based on git refspec/branchResolved2018-11-25

History

#1 Updated by okurz almost 2 years ago

  • Copied from action #44327: Trigger tests based on git refspec/branch added

#2 Updated by okurz over 1 year ago

  • Parent task set to #58184

#3 Updated by mkittler over 1 year ago

  • This use-case is also impaired by the inability to use custom needles (https://progress.opensuse.org/issues/56789 #56789).
  • Besides, the implementation suggestion use SCHEDULE parameter with e.g. SCHEDULE=tests/boot/boot_to_desktop,tests/run_custom implies that a composability of test distributions would be required. This is of course possible by somehow embedding the base test distribution (e.g. os-autoinst-distri-opensuse) into the product repository (e.g. cobbler) adding own tests on top of it. But this sounds rather hacky and inconvenient to use. So having a light test distribution which can simply be referred to as "base test distribution" seems a desirable for this use-case.

These points are likely the tricky part. Triggering the test execution itself is likely not that hard. Instead of writing a "polling bot" I would add an API route in openQA which can be added as GitHub hook.

#4 Updated by okurz 9 months ago

mkittler wrote:

Maybe we can still consider any need for needles an independant requirement of the part about triggering test code which I find more important.

Regarding the "hacky approach" or "base test distribution" I think we have a ticket for having something like a "linux" middleware between os-autoinst and os-autoinst-distri-opensuse that can be extracted but for now and to have a proof-of-concept on which to improve upon I recommend the "hacky" approach

[…] I would add an API route in openQA which can be added as GitHub hook.

What would you add as an API route here?

#5 Updated by okurz 6 months ago

  • Target version set to Ready

#7 Updated by okurz 2 months ago

  • Related to coordination #77698: [epic] synchronous qemu based system level test in pull request CI runs, e.g. standalone isotovideo or openQA tests added

#8 Updated by okurz 2 months ago

  • Subject changed from Trigger openQA tests in pull requests of any product github pull request to [epic] Trigger openQA tests in pull requests of any product github pull request
  • Status changed from New to Blocked
  • Assignee set to okurz

waiting for #77698 first

Also available in: Atom PDF