Project

General

Profile

action #44843

[functional][u][epic] Cleanup the use of serial-/virtio-/ssh-consoles in our tests (was: use $self->select_serial_terminal instead of checking IPMI in every module)

Added by okurz almost 2 years ago. Updated 5 months ago.

Status:
Workable
Priority:
Normal
Category:
Enhancement to existing tests
Target version:
Start date:
2018-12-13
Due date:
% Done:

50%

Estimated time:
(Total: 0.00 h)
Difficulty:

Description

Motivation

https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6032#discussion_r239122846
or: Reduce if/else on architecture or backend just so select whatever console is the most reliable one

Acceptance criteria

  • AC1: No more if/else on architecture or backend just for console selection in test modules

Suggestions

  • git grep 'BACKEND.*ipmi.*select_console' for a start, or look for 'root-ssh' or use_ssh_serial_console

Subtasks

openQA Project - action #45143: Suppress the prompt sign in the "expected output" of serial consoles (or virtio-console)Rejectedokurz

action #45146: [functional][u] Replace wrappers of wrappers for consoles with proper 'select_console' callsNewSLindoMansilla


Related issues

Related to openQA Project - action #36457: Implement serial terminal console for svirt backendResolved2018-05-23

Related to openQA Tests - action #13914: [functional][u][ipmi] wait_serial does not get expected output because ipmi console connection is closed Blocked2016-09-27

Related to openQA Tests - action #44888: [qam][ppc64le] test fails in sshd - fails in select_serial_terminalRejected2018-12-07

Related to openQA Project - action #12398: hide echoing to serial port in testapi calls from output in screenshotsNew2016-06-19

Related to openQA Tests - action #38819: [tools][functional][u][epic] Refactor use of backendsNew2019-02-17

Related to openQA Tests - action #44138: [functional][u][userspace] tests in QAM which seem to be the equivalent look so much easier to review, crosscheckResolved2018-11-21

Related to openQA Tests - action #45842: [all] select_serial_terminal on qemu can fail due to global VIRTIO_CONSOLE=1New2019-01-09

Blocks openQA Tests - action #34699: [functional][u][ipmi] access to serial log during installationBlocked2018-04-10

History

#1 Updated by dheidler almost 2 years ago

  • Status changed from Workable to In Progress
  • Assignee set to dheidler

#2 Updated by dheidler almost 2 years ago

  • Status changed from In Progress to Feedback

#3 Updated by dheidler almost 2 years ago

  • Assignee changed from dheidler to okurz

#4 Updated by okurz almost 2 years ago

Discussed with asmorodskyi and dheidler and also taking feedback in https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6392 into account. We agreed to the following points:

  • Console tests by default should use whatever is the most reliable and fast approach to check correct executions of commands. A serial terminal seems to be most promising
  • The current way how "virtio"-terminal or "serial"-terminal based tests are visualized in openQA should be improved before we continue porting more tests
  • We should use a visualization of serial terminal output like in https://openqa.suse.de/tests/2282638#step/abort01/1 -> #44138
  • Especially the prompt sign in the "expected output" checking is not really adding valuable information and should be supressed -> #45143
  • Test modules should be written in a clean and simple way without any 'if/else'-branches on machines or architectures
  • We should use proper consoles from os-autoinst and not write wrappers of wrappers as consoles should provide already enough abstraction -> #45146

I have the feeling the whole mess started with https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/2916 which was introducing the 'root-ssh' console. For me the distinction to the 'root-console' is not clear. I wonder if they should be equivalent, e.g. for IPMI, so that we could keep simply select_console 'root-console' in test modules and we would just select what works most reliable. But I guess 'root-console' allows to do needle checks and 'root-ssh' does not because we do not control it over VNC. The introduction of the formerly mentioned PR triggered tickets like #39665 where we need to call use_ssh_serial_console just to get "tests working again".

#6 Updated by okurz almost 2 years ago

  • Related to action #36457: Implement serial terminal console for svirt backend added

#7 Updated by okurz almost 2 years ago

  • Related to action #13914: [functional][u][ipmi] wait_serial does not get expected output because ipmi console connection is closed added

#8 Updated by okurz almost 2 years ago

  • Related to action #44888: [qam][ppc64le] test fails in sshd - fails in select_serial_terminal added

#9 Updated by okurz almost 2 years ago

  • Related to action #34699: [functional][u][ipmi] access to serial log during installation added

#10 Updated by okurz almost 2 years ago

  • Related to action #12398: hide echoing to serial port in testapi calls from output in screenshots added

#11 Updated by okurz almost 2 years ago

  • Subject changed from [functional][u] use $self->select_serial_terminal instead of checking IPMI in every module to [functional][u][epic] Cleanup the use of serial-/virtio-/ssh-consoles in our tests (was: use $self->select_serial_terminal instead of checking IPMI in every module)
  • Description updated (diff)
  • Status changed from Feedback to Blocked
  • Target version changed from Milestone 22 to future
  • Difficulty deleted (easy)

#12 Updated by okurz almost 2 years ago

  • Related to action #38819: [tools][functional][u][epic] Refactor use of backends added

#13 Updated by okurz almost 2 years ago

  • Related to action #44138: [functional][u][userspace] tests in QAM which seem to be the equivalent look so much easier to review, crosscheck added

#14 Updated by pvorel almost 2 years ago

FYI I'm planning to use serial console for svirt backend in select_serial_terminal() (#36457, merged PR https://github.com/os-autoinst/os-autoinst/pull/1048 needs some additional work).

#15 Updated by okurz almost 2 years ago

pvorel that's good news. Can we somehow achieve to replace the calls to 'select_serial_terminal' with the according 'select_console' calls? Right now it looks a bit like differing approaches to have on one hand calls like select_console 'root-console' and such and then these "wrapper-functions".

#16 Updated by pvorel over 1 year ago

Note: tests/console/consoletest_setup.pm is using check_console_font(), which requires using assert_screen(), so it cannot use select_serial_terminal().

I decided to start to replace select_console "root-console" with $self->select_serial_terminal, which also bring IPMI support. See:
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6484

#17 Updated by pvorel over 1 year ago

  • Related to action #45842: [all] select_serial_terminal on qemu can fail due to global VIRTIO_CONSOLE=1 added

#18 Updated by okurz over 1 year ago

  • Assignee changed from okurz to mgriessmeier

Move to new QSF-u PO after I moved to the "tools"-team. I mainly checked the subject line so in individual instances you might not agree to take it over completely into QSF-u. Feel free to discuss with me or reassign to me or someone else in this case. Thanks.

#19 Updated by pvorel over 1 year ago

As one of these involved in virtio and serial console and creator of select_serial_terminal() offer to help with review. Not assigning to it due lack of time, but might do part of it in the future.

#20 Updated by SLindoMansilla 5 months ago

  • Status changed from Blocked to Workable
  • Assignee changed from mgriessmeier to SLindoMansilla

#21 Updated by SLindoMansilla 5 months ago

  • Related to deleted (action #34699: [functional][u][ipmi] access to serial log during installation)

#22 Updated by SLindoMansilla 5 months ago

  • Blocks action #34699: [functional][u][ipmi] access to serial log during installation added

Also available in: Atom PDF