coordination #58184
Updated by okurz over 4 years ago
## Motivation This is linked to [Use case 4](https://progress.opensuse.org/projects/openqav3/wiki#Use-case-4) and motivated by a discussion by the QA tools team in the weekly meeting 2019-10-15 ## User story As a test case contributor during test case development I want to run tests on production instances with all necessary changes recorded in version control before merging to master so that my change will have minimal unexpected impact (test regressions) on existing tests ## Further user stories (from https://confluence.suse.com/pages/viewpage.action?pageId=365527173) 1. I want to start a job based on a modified test in production (In production tests can behave differently, for example because of the heavier load) 1. I want to edit needles and test if they work before proposing changes. 1. I want to compare the results of a certain job group between two of my branches 1. I want to schedule a test 100 times without it showing up in the group overview 1. I want to trigger multiple cloned jobs for each pull-request (Sometimes you want to trigger VR for different jobs against the same PR. it would be nice to do that in one command line) 1. I want to trigger the relevant tests automatically by creating a PR ## Implications and suggestions * The usual test contributor workflows should be supported and made easier by making openQA fully aware of tests triggered for development purposes without negatively impacting existing validation tests * Potential impact on asset management * No pollution of validation test reports by development tests * If there are new/modified needles involved, the existing workflow cannot handle that. The current practice is: * Test your changes (and possibly needle changes) locally and create PR(s) * Edit needles online and save them (then they will be committed to master). Requires admin rights * Cloning cancelled or incomplete jobs currently does not work as openqa-clone-custom-git-refspec requires the vars.json file from a completed job with this file uploaded * Replace "fetchneedles" by inherent git support * Provide support for github pull request validation * Extend openqa-clone-custom-git-refspec to accept list of source tests to clone * openqa-clone-custom-git-refspec: Output in markdown format for easy copy/pasting into git commit messages and github PR comments * openqa-clone-custom-git-refspec: Provide link to /tests/overview page for the custom build when multiple tests have been cloned * Make the trigger source of test jobs apparent, e.g. the source git repositories * #14818#note-18 : "Tim got a ticket from Ray that the docker test failed and wants openQA to reproduce the issue and pause at the beginning of the docker test. Afterwards he wants openQA to make a disk snapshot and step through the test execution to find out where the problem is. After he found out, he reloads the snapshot to tweak the execution. During this process, openQA records his steps and allows to add needles."