QA » openQA Project » openQA Tests

Discussed with asmorodskyi and dheidler and also taking feedback in https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6392 into account. We agreed to the following points:

• Console tests by default should use whatever is the most reliable and fast approach to check correct executions of commands. A serial terminal seems to be most promising
• The current way how "virtio"-terminal or "serial"-terminal based tests are visualized in openQA should be improved before we continue porting more tests
• We should use a visualization of serial terminal output like in https://openqa.suse.de/tests/2282638#step/abort01/1 -> #44138
• Especially the prompt sign in the "expected output" checking is not really adding valuable information and should be supressed -> #45143
• Test modules should be written in a clean and simple way without any 'if/else'-branches on machines or architectures
• We should use proper consoles from os-autoinst and not write wrappers of wrappers as consoles should provide already enough abstraction -> #45146

I have the feeling the whole mess started with https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/2916 which was introducing the 'root-ssh' console. For me the distinction to the 'root-console' is not clear. I wonder if they should be equivalent, e.g. for IPMI, so that we could keep simply select_console 'root-console' in test modules and we would just select what works most reliable. But I guess 'root-console' allows to do needle checks and 'root-ssh' does not because we do not control it over VNC. The introduction of the formerly mentioned PR triggered tickets like #39665 where we need to call use_ssh_serial_console just to get "tests working again".

FYI I'm planning to use serial console for svirt backend in select_serial_terminal() (#36457, merged PR https://github.com/os-autoinst/os-autoinst/pull/1048 needs some additional work).

pvorel that's good news. Can we somehow achieve to replace the calls to 'select_serial_terminal' with the according 'select_console' calls? Right now it looks a bit like differing approaches to have on one hand calls like select_console 'root-console' and such and then these "wrapper-functions".

Note: tests/console/consoletest_setup.pm is using check_console_font(), which requires using assert_screen(), so it cannot use select_serial_terminal().

I decided to start to replace select_console "root-console" with $self->select_serial_terminal, which also bring IPMI support. See:
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6484

As one of these involved in virtio and serial console and creator of select_serial_terminal() offer to help with review. Not assigning to it due lack of time, but might do part of it in the future.

See for the reason of tracker change: http://mailman.suse.de/mailman/private/qa-sle/2020-October/002722.html

This ticket was set to Normal priority but was not updated within the SLO period. Please consider picking up this ticket or just set the ticket to the next lower priority.