Project

General

Profile

action #39566

Snapshot rollback for tests that timed out asserting screen - UI defined rollback

Added by szarate almost 3 years ago. Updated about 1 year ago.

Status:
New
Priority:
Low
Assignee:
-
Category:
Feature requests
Target version:
Start date:
2018-08-10
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

As a test developer
I want to roll back the test module that timed out during the needle check
So that I can edit and create new needles when there is a timeout/failure.

This basically means that openQA should be able to pick up a running job, that has snapshots enabled, and allow the test developer to create new needles, and ensure that the test can be restarted from the same test module (Snapshot rollback).

When possible, the communication should be between the webUI and os-autoinst, without taking the worker into account.

10:41:08        coolo | as in wait for failure, edit the needles, upload the needles to os-autoinst (again: if anyway possible, leave out the worker), rollback the test module, run again

History

#1 Updated by szarate almost 3 years ago

  • Description updated (diff)

#2 Updated by coolo almost 3 years ago

To put this into context: This is about job modules that need needle updates for check_screens. If in development mode, the job creates a snapshot per job module, so that when I recreate a needle, I get an option to restart the job from start or from the job module that had the needle.

#3 Updated by mkittler almost 3 years ago

So that's basically 'setting the SKIPTO variable' in the middle of the test. Just to check whether I understand this correctly:

1. Prerequisites (in which state should this rollback be possible)

  1. User has created a developer session
  2. Test is paused due to check_screen or assert_screen timeout
  3. User has created a new needle via the needle editor

2. What we have so far (when https://github.com/os-autoinst/openQA/pull/1727 is merged)

  1. The test can only pause on assert_screen timeout but not on check_screen timeout.
  2. The developer mode allows to resume a paused test where it has been interrupted.
  3. If it has been interrupted due to an assert_screen timeout, needles are reloaded and the assertion is retried.

3. What we need to implement

  1. If in development mode, the job creates a snapshot per job module (by default?).
  2. The test can pause on check_screen timeout (in addition to 2.1).
  3. When prerequisites are met, the web UI proposes to revert the test execution to a previous snapshot before resuming (in addition to 2.2 and 2.3).
  4. The web UI sends the command to revert via the liveviewhandler to os-autoinst. The worker is not involved.
  5. Status info and possible error messages related to the rollback are passed to the frontend via the liveviewhandler so the user can see what's going on.
  6. I assume that when we revert the VM to a previous snapshot, we must also revert the execution of the Perl test code to that point. Otherwise the state of the VM and the test code execution would be out-of-sync.

#4 Updated by mkittler almost 3 years ago

  • Status changed from New to Feedback
  • Assignee set to mkittler

This ticket is mostly made of IRC messages. So I guess it doesn't make any sense to start with this without having feedback on my understanding of the feature.

However, in the meantime I could at least pick some of the points which would make sense to implement anyways (eg. "The test can pause on check_screen timeout").

#5 Updated by mkittler almost 3 years ago

PR for making pause on check_screen timeout from os-autoinst side: https://github.com/os-autoinst/os-autoinst/pull/1013

Note that this ticket still needs refinement.

#6 Updated by coolo almost 3 years ago

  • Status changed from Feedback to New
  • Target version changed from Current Sprint to Ready

We will look at breaking this down for future planning meeting

#7 Updated by coolo almost 3 years ago

  • Assignee deleted (mkittler)

#8 Updated by mkittler over 2 years ago

  • Target version deleted (Ready)

Note that I won't pick that ticket unless it is clarified and preferably broken down. So I move it out of ready till that is done.

#9 Updated by okurz about 2 years ago

  • Category set to Feature requests

#10 Updated by okurz about 1 year ago

  • Priority changed from Normal to Low
  • Target version set to future

Also available in: Atom PDF