action #9760
closed"Finish Testing a Build" (aka Un-Obsolete) and "Retest entire build"
50%
Description
We have two requirements which are shared by both release management and QA - and I believe the implementation is linked, hence treating this as a single issue
These are for the following use cases
"Finish Testing a Build" - openQA currently only considers the latest build as relevant, but this is simply not true. For example Rel Mgrs may check in a very dangerous/dubious package as part of Build #1, and then checkin something far less important for Builds #2, #3, and #4 all quickly after each other. In this case, openQA will obsolete testing for builds #1-3, and only test build #4 properly. But because Build#1 contained the risky change, we NEED openQA to be able to be able to 'unobsolete' a build and complete it's testing of that important build, which would then give us a meaningful point of comparision between Build #1 and #4
"Retest entire Build" - There's times, however rare, that either due to hardware, OBS, or other issues, that an entire set of results can be invalidated and the best option is to reschedule everything. This can be done by the CLI or by lots of clicking on the WebUI, but a single button to trigger a retest of an entire build would be nicer
I actually envision this as two new buttons on the build results screen
For each Arch (I think it makes sense to split this up based on arch) I believe we should have two buttons 'Unobsolete' and 'Reschedule everything'
A horrifically crude mock-up - http://i.imgur.com/D35i43I.png - the Ghostbusters icon is for 'Unobsoleting' and the restart icon is our usual one
Clicking on the Ghostbusters icon would reschedule any obsolete jobs for that arch on that build
Clicking on the restart icon would reschedule all jobs for that arch on that build
Updated by coolo over 8 years ago
I would say that this a too special job to put such a prominent icon on such a prominent page. What about providing a script to do such special mass retriggers?
Updated by RBrownSUSE over 8 years ago
It's not that 'special' - it happens repeatedly during the development of all SLE products and it needs to be doable by Release Managers - and we don't want to give them openQA shell access to run scripts themselves
Updated by coolo over 8 years ago
but how often is it done? once or twice every couple of weeks. how often do you see that button? every freaking day on every freaking job group.
And scripts don't need shell access - just as not everyong using osc rebuildpac requires shell access on OBS. Adam maintains a nice pair of python bindings
Updated by okurz over 8 years ago
I would go for the python bindings to create fast prototypes of what might be useful to have as controls in the web ui later on.
Updated by coolo over 8 years ago
BTW: Ludwig has a user story that is related to this. He wants to repost a set of ISO parameters to reschedule all jobs with the new templates.
Updated by oholecek about 8 years ago
- % Done changed from 0 to 50
Rescheduling of iso is done in issue#2776
The 'unobsoleting' thing seems to me can already be done by providing _NOOBSOLETEBUILD variable when scheduling new iso (apparently this has been already implemented). Technically this does does not do 'unobsoleting' rather it does not cancel running/scheduled tests of old iso when scheduling new one. What do you think?
Updated by okurz almost 8 years ago
Can't we simply disable the "obsoleting old builds" again and instead deprioritize them? It is also helpful for reviewers if the build they review is not obsoleted "under their hands" and keeping a steady history for scenarios.
Proposal:
- don't obsolete old jobs
- on new iso: if jobs for old build are in state scheduled, set priority-10
Updated by okurz almost 8 years ago
- Blocked by action #11072: Always finish important builds added
Updated by coolo almost 8 years ago
not everything is reviewed. we have plenty of staging DVDs that never see any reviewer for example
Updated by okurz almost 8 years ago
rbrown and coolo do not want to disable the old build obsoletion e.g. to reduce worker load and more regarding some use cases of RMs. However, he agrees that for reproduction purposes it would make sense not to abort the according job runs (we don't know how this could be done). Preferred would be to reproduce not on the productive instance but then we have the problem of how to make workers available for multiple parties (see #12194).
Also, we had the idea to tier tests more and do a prerequisite job in our "test pipeline", e.g. check if ISO is bootable and finish successfully and start all other jobs after that mini-test_suite.
Updated by okurz almost 8 years ago
So "finish testing a build" is considered to be complete with #11072, regarding "retest entire build", can someone post the easy CLI approach here? Is there really a GUI way necessary? If yes, how should it look like so that coolo is also ok with that (see #9760#note-3)? Maybe instruct it with an API call from some OBS dashboard?
Updated by okurz almost 8 years ago
- Status changed from In Progress to Resolved
seems like the "retest entire build" is not really needed often or we didn't need it for a long time. In most if not all cases we need to selectively retrigger jobs because there are probably always some jobs which work in every case. As the subtask is also done we can close this.