QA » openQA Project » openQA Tests

Discussed with asmorodskyi and dheidler and also taking feedback in https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6392 into account. We agreed to the following points:

• Console tests by default should use whatever is the most reliable and fast approach to check correct executions of commands. A serial terminal seems to be most promising
• The current way how "virtio"-terminal or "serial"-terminal based tests are visualized in openQA should be improved before we continue porting more tests
• We should use a visualization of serial terminal output like in https://openqa.suse.de/tests/2282638#step/abort01/1 -> #44138
• Especially the prompt sign in the "expected output" checking is not really adding valuable information and should be supressed -> #45143
• Test modules should be written in a clean and simple way without any 'if/else'-branches on machines or architectures
• We should use proper consoles from os-autoinst and not write wrappers of wrappers as consoles should provide already enough abstraction -> #45146

I have the feeling the whole mess started with https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/2916 which was introducing the 'root-ssh' console. For me the distinction to the 'root-console' is not clear. I wonder if they should be equivalent, e.g. for IPMI, so that we could keep simply select_console 'root-console' in test modules and we would just select what works most reliable. But I guess 'root-console' allows to do needle checks and 'root-ssh' does not because we do not control it over VNC. The introduction of the formerly mentioned PR triggered tickets like #39665 where we need to call use_ssh_serial_console just to get "tests working again".

FYI I'm planning to use serial console for svirt backend in select_serial_terminal() (#36457, merged PR https://github.com/os-autoinst/os-autoinst/pull/1048 needs some additional work).

pvorel that's good news. Can we somehow achieve to replace the calls to 'select_serial_terminal' with the according 'select_console' calls? Right now it looks a bit like differing approaches to have on one hand calls like select_console 'root-console' and such and then these "wrapper-functions".

Note: tests/console/consoletest_setup.pm is using check_console_font(), which requires using assert_screen(), so it cannot use select_serial_terminal().

I decided to start to replace select_console "root-console" with $self->select_serial_terminal, which also bring IPMI support. See:
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6484

As one of these involved in virtio and serial console and creator of select_serial_terminal() offer to help with review. Not assigning to it due lack of time, but might do part of it in the future.

See for the reason of tracker change: http://mailman.suse.de/mailman/private/qa-sle/2020-October/002722.html