action #10202
closed
Test openQA in openQA on o3/osd
Added by okurz over 8 years ago.
Updated over 7 years ago.
Category:
Feature requests
Description
User story¶
As a admin of openQA server I want to test new versions of openQA itself to ensure new versions don't break productive instances on deployment
Acceptance criteria¶
- openQA "releases" are tested automatically ("releases" being e.g. new packages in OBS or git commits from github)
- New releases trigger new builds
tasks¶
- test repositories https://github.com/os-autoinst/os-autoinst-distri-openQA and https://github.com/os-autoinst/os-autoinst-needles-openQA locally
- make sure the required asset is available, e.g. an installed SLE12-SP1-Server qcow harddisk image
- make these repos available on openqa.suse.de
- Mangle with database to make it known to openQA
- Come up with a trigger solution (like our rsync.pl but another one more suitable for openQA itself)
- Inform every SUSE/openSUSE openqa admin that we have this solution
- (optional) Come up with cool monitoring solution and tell if test is passed or failed before we deploy new versions
further notes¶
We already have a good set of tests available, we just need to put them to good use :-)
It would be nice to run this on openqa.opensuse.org but we don't have enough worker capacity available there
With https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/4 TW, SLES and Leap are basically supported as base versions for running a current openQA version. http://lord.arch/tests/2344 shows a successful installation and a running openQA within openQA on TW.
As we have higher worker capacity on o3 now it should be feasible to run tests there. For now I propose to trigger these tests from the jenkins instance on http://lord.arch.suse.de:8080 to run on o3. Using jenkins for triggering allows e.g. IRC notifications on job results. For now running in i.s.d#openqa-test
Would anyone mind if I
- create a new job group on o3
- checkout os-autoinst-distri-openQA on o3
- trigger tests on each new OBS build of openQA on o3
?
- Status changed from New to In Progress
- Subject changed from Test openQA in openQA on openqa.suse.de to Test openQA in openQA on o3/osd
Need to replace real needles repository with dummy one. It will dramatically decrease test run time
- Priority changed from Low to Normal
After we had some recent problems which IMHO could all have been catched by this if we would have it I guess it makes sense to higher prio a bit.
note though that the problems we had were with tests relying on uptodate deployment.
test distribution on o3 checked out and kept up to date with new cron rule:
/etc/cron.d/openqa-update-git
-7 * * * * geekotest (cd /var/lib/openqa/share/tests/openqa && git pull --rebase origin master)
- Status changed from In Progress to Resolved
here we go: http://lord.arch.suse.de:8080/job/monitor-openQA_in_openQA-TW/1054/console
and in irc.suse.de#openqa-test:
[6 Dec 2016 17:10:01] <jenkins-bot> Yippee, build fixed!
[6 Dec 2016 17:10:01] <jenkins-bot> Project monitor-openQA_in_openQA-TW build #1054: FIXED in 1 hr 30 min: http://lord.arch.suse.de:8080/job/monitor-openQA_in_openQA-TW/1054/
[6 Dec 2016 17:10:02] <jenkins-bot> Starting build #1055 for job monitor-openQA_in_openQA-TW (previous build: FIXED)
In case a test fails it will (or should) write a comment on https://build.opensuse.org/project/show/devel:openQA
The current tests are based on Tumbleweed. We can now harmonize the structure, collect feedback, extend by other distributions, e.g. openSUSE Leap and do more about the monitoring. All of this should end up in new tickets though.
Also available in: Atom
PDF