QA » openQA Project » openQA Tests

pcervinka wrote:

1. Universal function for package installation¶

Example in test code:

$testapi::distri->get_package_manager()->install_package("ltp")

As you can see, test code just contained universal installation command, which was distro independent.

This could be simplified to just install_package("ltp"); and all the $testapi::distri->get_package_manager() boilerplate should be hidden inside the install_package() helper function.

2. Existing trup_call function needs root-console¶

It means that code with $self->select_serial_terminal doesn't work, console change needs to be done.

You can find this pattern in the code:

select_console 'root-console';
trup_call("--continue pkg $cmd", timeout => 2000);

Is there a way to fix that? trup_call should be console independent.

Yes, this can be fixed, although it'd be easier to fix (or even write correctly from the start) if the console API design were sane.

3. Support for transactional command in script_run¶

We can run commands on the transaction system in the snapshot via transactional-update -c -d --quiet run.
For example, we install something, want to just check something , but we can't use normal script_run, because it will run on booted snapshot.

Would be worth to have something like script_run("COMMAND", transactional => [is_transactional| any other condition | 1]) which would wrap command in case we want transactional approach?

This should be handled by trup_call(). When that function is called on a non-transactional system, it should fall back to plain script_run().

MDoucha wrote:

pcervinka wrote:

1. Universal function for package installation¶

Example in test code:

$testapi::distri->get_package_manager()->install_package("ltp")

As you can see, test code just contained universal installation command, which was distro independent.

This could be simplified to just install_package("ltp"); and all the $testapi::distri->get_package_manager() boilerplate should be hidden inside the install_package() helper function.

What I had thought about this, is to use module flags, and the pre_run hooks, and do the installation of packages there, so that test modules could declare in a hash which packages it expects to have installed, and through an install wrapper use directly the abstract factory: $testapi::distri->packages->install(qw(ltp vim)), and if needed use a facade for further needs...

2. Existing trup_call function needs root-console¶

It means that code with $self->select_serial_terminal doesn't work, console change needs to be done.

You can find this pattern in the code:

select_console 'root-console';
trup_call("--continue pkg $cmd", timeout => 2000);

Is there a way to fix that? trup_call should be console independent.

Yes, this can be fixed, although it'd be easier to fix (or even write correctly from the start) if the console API design were sane.

Which console api in this case?

3. Support for transactional command in script_run¶

We can run commands on the transaction system in the snapshot via transactional-update -c -d --quiet run.
For example, we install something, want to just check something , but we can't use normal script_run, because it will run on booted snapshot.

Would be worth to have something like script_run("COMMAND", transactional => [is_transactional| any other condition | 1]) which would wrap command in case we want transactional approach?

This should be handled by trup_call(). When that function is called on a non-transactional system, it should fall back to plain script_run().

hmm... @Petr, here you mean any particular call to script_run, or any? for the earlier I'd go with what Martin suggests, for the latter, we might want to think a bit further... of course we could make script_run redefined in the test distribution, to be aware of a system being transactional or not, and whether we want it quiet or not for a given test module...

szarate wrote:

MDoucha wrote:

Yes, this can be fixed, although it'd be easier to fix (or even write correctly from the start) if the console API design were sane.

Which console api in this case?

All of it, from the public testapi functions all the way down to the backends.

MDoucha wrote:

szarate wrote:

MDoucha wrote:

Yes, this can be fixed, although it'd be easier to fix (or even write correctly from the start) if the console API design were sane.

Which console api in this case?

All of it, from the public testapi functions all the way down to the backends.

@okurz are you aware of any plans or ideas which could help?

szarate wrote:

MDoucha wrote:

pcervinka wrote:

3. Support for transactional command in script_run¶

We can run commands on the transaction system in the snapshot via transactional-update -c -d --quiet run.
For example, we install something, want to just check something , but we can't use normal script_run, because it will run on booted snapshot.

Would be worth to have something like script_run("COMMAND", transactional => [is_transactional| any other condition | 1]) which would wrap command in case we want transactional approach?

This should be handled by trup_call(). When that function is called on a non-transactional system, it should fall back to plain script_run().

hmm... @Petr, here you mean any particular call to script_run, or any? for the earlier I'd go with what Martin suggests, for the latter, we might want to think a bit further... of course we could make script_run redefined in the test distribution, to be aware of a system being transactional or not, and whether we want it quiet or not for a given test module...

This particular case, i sorted for short term, but there could be better generic solution. This one just uses script_output, idea is the same:
https://github.com/czerw/os-autoinst-distri-opensuse/blob/b53828660d486122b2bbc298ab3a791c50e95cbf/lib/bootloader_setup.pm#L144
https://github.com/czerw/os-autoinst-distri-opensuse/blob/b53828660d486122b2bbc298ab3a791c50e95cbf/lib/bootloader_setup.pm#L163

It could be any call, but maybe, i just want to generalize specific case, which is not need for wider use.

pcervinka wrote:

@okurz are you aware of any plans or ideas which could help?

I do not know about any further plans or ideas regarding this going beyond this ticket itself.

pcervinka wrote:

1. Universal function for package installation¶

Related PR: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/15683

Some of the Thoughts I shared with Petr, 2 weeks ago:

thinking for instance in how the transnational update module will work, one can have a pre run module that looks for a packages method which should return a list of packages that the system under test should install, when it is found, it is passed directly to the install_packages() function, which defined in opensusedistribution, should allow for managing different types of installers, via InstallerFactory pattern (or maybe some other), the important part, is to be able to ask $distri->action($packages)

on test teardown, system is either rolled back in cases where btrfs is being used and it’s possible, or restored to a previous snapshot (ideally) when it’s a transactional system

it’s also posible that for isotovideo, a new step might be needed to achieve that, so that basetest::setup and basetest::teardown methods exist, and are capable somehow, of producing junit compatible output, this with the idea of having openqa test runs, to integrate with other systems, and to be able. to have standard interface that allows the reviewer to have better info ob the test that have been executed

questions arise when looking into the state of needles or what to do with them, but it should not be a huge problem in the end

the flag $basetest::packages has to be a hash, or a dictionary that allows the caller to know what the test objects are going to be

Petr's suggestion is to: Consider installing everything from the beginning before the tests are executed

so in the end, we’re defining:

• setup
• teardown
• objects

This also would help us to avoid post_fail_hooks that have mixed roles, such as stopping services and collecting logs

With this, as a further step one should be able to easily write either a Perl::Tidy rule or a Perl::Critic check, that allows for verification of such things, like: no package install if testobjects are defined

We can continue this work on a new ticket, in the qe-core backlog, with the help of the rest of the teams

Above idea looks fine, but we should make some follow up for details, as other ideas are already poping up https://github.com/pablo-herranz/os-autoinst-distri-opensuse/blob/9ea1f81305648b0fad9dfdab70f80dd46417737e/lib/Wrappers/packages.pm

I like the idea of this ticket, then we can avoid those if-elses everywhere.
However, we need to tweak check_reboot_changes as it will fail if the package is already installed as it expects a change in the new snapshot created by transactional-update. I guess it's easy to change that behaviour.
There is another thing to avoid this, which is using process_reboot after pkg installation, but it will reboot always regardless if the package is installed or not.

Was updating maintenance part for kernel update recently and there is one outcome. transactional-update patch don't indicate return code for a need of another patch
round if there is package manager update. It just prints it in output, but this is not comfortable as with zypper (which has multiple error codes). I created https://bugzilla.suse.com/show_bug.cgi?id=1216504 for further discussion. @jlausuch im just wondering, why it was not discovered earlier? For example why function fully_patch_system was not adapted already for transactional systems, as it is function used across all maintenance tests. Adaptation is now done in https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/18012.

pcervinka wrote in #note-19:

Was updating maintenance part for kernel update recently and there is one outcome. transactional-update patch don't indicate return code for a need of another patch
round if there is package manager update. It just prints it in output, but this is not comfortable as with zypper (which has multiple error codes). I created https://bugzilla.suse.com/show_bug.cgi?id=1216504 for further discussion. @jlausuch im just wondering, why it was not discovered earlier? For example why function fully_patch_system was not adapted already for transactional systems, as it is function used across all maintenance tests. Adaptation is now done in https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/18012.

Not sure... The way we have always installed updates (aggregates and incidents) has been like this:
https://openqa.suse.de/tests/12746599#step/install_updates/341
e.g.

zypper -n --no-gpg-checks ar -f  -n 'TEST_0' http://download.suse.de/ibs/SUSE:/Maintenance:/29290/SUSE_Updates_SLE-Micro_5.5_x86_64/ 'TEST_0'
zypper -n ref

...
zypper -n --no-gpg-checks ar -f  -n 'TEST_25' http://download.suse.de/ibs/SUSE:/Maintenance:/31370/SUSE_Updates_SLE-Micro_5.5_x86_64/ 'TEST_25'
zypper -n ref

transactional-update -n up

Is there test I could use to test the wrapper, I tried live patching https://dzedro.suse.cz/tests/146#step/qa_test_klp/17
Ignoring the missing packages, I guess live patching with transactional update is not even possible, because you have to do reboot to apply changes.
I'm looking for some, but transactional tests are separated and shared tests which I checked don't do install.

edit: found tests/kernel/update_kernel.pm from PR in comments

cypr ? https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/18204

jlausuch wrote in #note-21:

transactional-update -n up

I believe, we should use patch, instead of up to validate particular update.

Also patch in transactional-update doesn't properly work in case, when is a need for additional restart after package manager update. zypper handles this case, but not via transactional-update. See https://bugzilla.suse.com/show_bug.cgi?id=1216504

This ticket was set to High priority but was not updated within the SLO period. Please consider picking up this ticket or just set the ticket to the next lower priority.

Can this be resolved ?
Is there anything to add/improve, then create subtask ?