QA » openQA Project

1. Currently in openqa.suse.de has 192069 jobs and query "select count(*) from jobs where machine in (select name from machines where backend='qemu');" gives us that 166332 are using qemu backend which is 86% of all jobs. So it is not just one of others it is most used one
2. I am not asking to create some totally new functionality which will require a lot of effort to implement and support. No matter what you said about limited support among backends we already having functionality of creating snapshots and rolling back in case of failure , so this ticket is low hanging fruit which will not harm for 14% of our jobs but will give new abilities to other 86%

It fosters writing backend specific tests - this is harm

there is always some price which you should pay. Price for NOT having this feature for me that I lock down my tests with BTRFS so in case of any issues with BTRFS my tests will fail.
Can you state what kind of harm of using qemu specific feature with tests which running only in qemu ?
Also I didn't get how the fact that openQA will provide some new functionality is enforcing people to obligatory use it ? It would be still up to tests writer to use it or not

https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/4283

On backends which do not support snapshots, milestones could be handled in a different way. For LTP we will probably reset the machine when a test freezes it. Possibly we could introduce some callbacks for milestones which different backends, tests and architectures can override. On bare metal, getting the machine back into a previous state might require resetting the machine and rerunning some modules. Getting the block storage back into a previous state will depend on a lot of factors so it will be complicated. We could use BTRFS snapshots where they are available, but it requires dynamically running a module to handle it. The worst case will require a full reinstallation.

What I see could be done even though I think it is a bit different to what asmorodskyi wanted: Where we revert to a snapshot after a failed module on backends that support snapshots we could extend that to revert on demand also based on a test variable or even a test module flag.

So what I see as necessary steps to implement that:

• Find the location of snapshot reverting, probably https://github.com/os-autoinst/os-autoinst/blob/master/autotest.pm#L339
• Extend tests we have for snapshot handling, e.g. t/18-qemu.t
• Extend it to also revert based on test variables, e.g. pseudo code change

- revert_to_last_good if ($module_failed && snapshots_supported());
+ revert_to_last_good if ($module_failed && (snapshots_supported() || get_var('ROLLBACK_AFTER_EACH_MODULE') || $test_module{force_reset}));
}

sub handle_failed_module {
-    revert_to_last_good if snapshots_supported();
+    revert_to_last_good if snapshots_supported();
}

os-autoinst change merged and deployed to osd

I know it is a pain but for now we need to wait for o3 deployment. We might want to consider investing more time in an automated deployment infrastructure for os-autoinst changes which are less risky than openQA+database. Especiallly after o3 workers changed to transactional-update servers.