action #8542
closed
Added by ancorgs over 8 years ago.
Updated almost 8 years ago.
Category:
Feature requests
Description
Recently, some breakage has been introduced in openqa.opensuse.org by externally contributed tests ("externally" meaning me in this case ;-) ). External contributors cannot afford to verify the test in all the scenarios run by openqa.opensuse.org, so this will happen in the future again. It would be useful to use tortuga or any other machine to run a subset of the scenarios run in the official instance to make sure the new tests can be deployed with some degree of confidence.
- Status changed from New to Resolved
any "staging openQA instance" would not be able to run everything which is run in production. It just does not scale. So anyway only a subset can be run and there can be always something missing. Also, we don't have the hardware capacity to cover everything twice and also consider SLE plus openSUSE. Our DOD should cover some important steps so that external contributors are motivated to test something locally first. We have a good test review process and it has to be decided by the reviewer if he accepts the risk of a new test with or without a local verification and covering which scenarios. Depending on the contributors it might make sense to setup a staging server with a subset of tests which is used by multiple test developers to share the burden of openQA setup and administration. E.g. since recently also the YaST team has one available: https://wiki.microfocus.net/index.php/YAST/openQA
I added a paragraph on https://progress.opensuse.org/projects/openqatests/wiki/Wiki#Test-development-instances-staging-openQA-instances to describe this (mainly same text as this comment). With this, I would like to resolve this ticket.
Also available in: Atom
PDF