Snapshot rollback for tests that timed out asserting screen - UI defined rollback
As a test developer
I want to roll back the test module that timed out during the needle check
So that I can edit and create new needles when there is a timeout/failure.
This basically means that openQA should be able to pick up a running job, that has snapshots enabled, and allow the test developer to create new needles, and ensure that the test can be restarted from the same test module (Snapshot rollback).
When possible, the communication should be between the webUI and os-autoinst, without taking the worker into account.
10:41:08 coolo | as in wait for failure, edit the needles, upload the needles to os-autoinst (again: if anyway possible, leave out the worker), rollback the test module, run again
#2 Updated by coolo over 2 years ago
To put this into context: This is about job modules that need needle updates for check_screens. If in development mode, the job creates a snapshot per job module, so that when I recreate a needle, I get an option to restart the job from start or from the job module that had the needle.
#3 Updated by mkittler over 2 years ago
So that's basically 'setting the
SKIPTO variable' in the middle of the test. Just to check whether I understand this correctly:
1. Prerequisites (in which state should this rollback be possible)¶
- User has created a developer session
- Test is paused due to
- User has created a new needle via the needle editor
2. What we have so far (when https://github.com/os-autoinst/openQA/pull/1727 is merged)¶
- The test can only pause on
assert_screentimeout but not on
- The developer mode allows to resume a paused test where it has been interrupted.
- If it has been interrupted due to an
assert_screentimeout, needles are reloaded and the assertion is retried.
3. What we need to implement¶
- If in development mode, the job creates a snapshot per job module (by default?).
- The test can pause on
check_screentimeout (in addition to 2.1).
- When prerequisites are met, the web UI proposes to revert the test execution to a previous snapshot before resuming (in addition to 2.2 and 2.3).
- The web UI sends the command to revert via the liveviewhandler to os-autoinst. The worker is not involved.
- Status info and possible error messages related to the rollback are passed to the frontend via the liveviewhandler so the user can see what's going on.
- I assume that when we revert the VM to a previous snapshot, we must also revert the execution of the Perl test code to that point. Otherwise the state of the VM and the test code execution would be out-of-sync.
#4 Updated by mkittler over 2 years ago
- Status changed from New to Feedback
- Assignee set to mkittler
This ticket is mostly made of IRC messages. So I guess it doesn't make any sense to start with this without having feedback on my understanding of the feature.
However, in the meantime I could at least pick some of the points which would make sense to implement anyways (eg. "The test can pause on check_screen timeout").
#5 Updated by mkittler over 2 years ago
PR for making pause on
check_screen timeout from os-autoinst side: https://github.com/os-autoinst/os-autoinst/pull/1013
Note that this ticket still needs refinement.