action #10202
closedTest openQA in openQA on o3/osd
0%
Description
User story¶
As a admin of openQA server I want to test new versions of openQA itself to ensure new versions don't break productive instances on deployment
Acceptance criteria¶
- openQA "releases" are tested automatically ("releases" being e.g. new packages in OBS or git commits from github)
- New releases trigger new builds
tasks¶
- test repositories https://github.com/os-autoinst/os-autoinst-distri-openQA and https://github.com/os-autoinst/os-autoinst-needles-openQA locally
- make sure the required asset is available, e.g. an installed SLE12-SP1-Server qcow harddisk image
- make these repos available on openqa.suse.de
- Mangle with database to make it known to openQA
- Come up with a trigger solution (like our rsync.pl but another one more suitable for openQA itself)
- Inform every SUSE/openSUSE openqa admin that we have this solution
- (optional) Come up with cool monitoring solution and tell if test is passed or failed before we deploy new versions
further notes¶
We already have a good set of tests available, we just need to put them to good use :-)
It would be nice to run this on openqa.opensuse.org but we don't have enough worker capacity available there
Updated by okurz over 7 years ago
- Assignee set to okurz
With https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/4 TW, SLES and Leap are basically supported as base versions for running a current openQA version. http://lord.arch/tests/2344 shows a successful installation and a running openQA within openQA on TW.
As we have higher worker capacity on o3 now it should be feasible to run tests there. For now I propose to trigger these tests from the jenkins instance on http://lord.arch.suse.de:8080 to run on o3. Using jenkins for triggering allows e.g. IRC notifications on job results. For now running in i.s.d#openqa-test
Would anyone mind if I
- create a new job group on o3
- checkout os-autoinst-distri-openQA on o3
- trigger tests on each new OBS build of openQA on o3
?
Updated by okurz over 7 years ago
idea: Use https://api.opensuse.org/apidocs to write back notification to corresponding OBS project
Updated by okurz over 7 years ago
- Subject changed from Test openQA in openQA on openqa.suse.de to Test openQA in openQA on o3/osd
latest shows the latest incarnation of openQA test on o3. It fails downloading the latest TW snapshot from o3 itself. Why can it reach the external network but not o3? --> https://openqa.opensuse.org/tests/latest?machine=64bit&test=install_on_tw&flavor=dev&distri=openQA&arch=x86_64&version=Tumbleweed-#step/start_test/1
Updated by asmorodskyi over 7 years ago
Need to replace real needles repository with dummy one. It will dramatically decrease test run time
Updated by okurz over 7 years ago
- Priority changed from Low to Normal
After we had some recent problems which IMHO could all have been catched by this if we would have it I guess it makes sense to higher prio a bit.
Updated by coolo over 7 years ago
note though that the problems we had were with tests relying on uptodate deployment.
Updated by okurz over 7 years ago
asmorodskyi wrote:
Need to replace real needles repository with dummy one. It will dramatically decrease test run time
https://github.com/os-autoinst/openQA/pull/1061 might help.
Updated by okurz over 7 years ago
test distribution on o3 checked out and kept up to date with new cron rule:
/etc/cron.d/openqa-update-git
-7 * * * * geekotest (cd /var/lib/openqa/share/tests/openqa && git pull --rebase origin master)
Updated by okurz over 7 years ago
successful job on o3: https://openqa.opensuse.org/tests/315296
Now onto monitoring like in #12956#note-10
Updated by okurz over 7 years ago
- Status changed from In Progress to Resolved
here we go: http://lord.arch.suse.de:8080/job/monitor-openQA_in_openQA-TW/1054/console
and in irc.suse.de#openqa-test:
[6 Dec 2016 17:10:01] <jenkins-bot> Yippee, build fixed!
[6 Dec 2016 17:10:01] <jenkins-bot> Project monitor-openQA_in_openQA-TW build #1054: FIXED in 1 hr 30 min: http://lord.arch.suse.de:8080/job/monitor-openQA_in_openQA-TW/1054/
[6 Dec 2016 17:10:02] <jenkins-bot> Starting build #1055 for job monitor-openQA_in_openQA-TW (previous build: FIXED)
In case a test fails it will (or should) write a comment on https://build.opensuse.org/project/show/devel:openQA
The current tests are based on Tumbleweed. We can now harmonize the structure, collect feedback, extend by other distributions, e.g. openSUSE Leap and do more about the monitoring. All of this should end up in new tickets though.