QA » openQA Project

The job doesn't fail - it crashes. In which module it crashed is generally speaking not a good hint. We can't label installation_overview as failed module IMO. We shouldn't incomplete, but fail in this scenario.

The backend most likely is not supposed to crash here but report a failure to select_console - and such die in testapi

[2019-11-10T09:33:56.307 CET] [debug] <<< testapi::check_screen(mustmatch='ssh-open', timeout=1)
[2019-11-10T09:33:56.794 CET] [debug] no match: 2.5s, best candidate: installation_overview-ssh-open-20170911 (0.00)
[2019-11-10T09:33:57.801 CET] [debug] no match: 1.5s, best candidate: ssh-open_firewall_disabled-20190221 (0.29)
[2019-11-10T09:33:59.344 CET] [debug] >>> testapi::_handle_found_needle: found ssh-open-20190207, similarity 1.00 @ 401/542
[2019-11-10T09:33:59.344 CET] [debug] /var/lib/openqa/cache/openqa.suse.de/tests/sle/tests/installation/installation_overview.pm:83 called testapi::select_console
[2019-11-10T09:33:59.344 CET] [debug] <<< testapi::select_console(testapi_console='install-shell')
/usr/lib/os-autoinst/consoles/vnc_base.pm:58:{
  'hostname' => 'localhost',
  'port' => 58291,
  'ikvm' => 0
}
[2019-11-10T09:34:00.445 CET] [debug] Connected to Xvnc - PID 30160
icewm PID is 30182
xterm PID is 30184
[2019-11-10T09:34:01.526 CET] [debug] led state 0 0 0 -261
[2019-11-10T09:34:01.537 CET] [debug] /var/lib/openqa/cache/openqa.suse.de/tests/sle/tests/installation/installation_overview.pm:83 called testapi::select_console
[2019-11-10T09:34:01.537 CET] [debug] <<< testapi::select_console(testapi_console='root-ssh')
/usr/lib/os-autoinst/consoles/vnc_base.pm:58:{
  'ikvm' => 0,
  'hostname' => 'localhost',
  'port' => 49909
}
[2019-11-10T09:34:02.658 CET] [debug] Connected to Xvnc - PID 30187
icewm PID is 30207
xterm PID is 30209
[2019-11-10T09:34:03.690 CET] [debug] Could not connect to root@scooter-1.qa.suse.de, Retrying after some seconds...
[2019-11-10T09:34:13.691 CET] [debug] Could not connect to root@scooter-1.qa.suse.de, Retrying after some seconds...
[2019-11-10T09:34:23.692 CET] [debug] Could not connect to root@scooter-1.qa.suse.de, Retrying after some seconds...
[2019-11-10T09:34:33.693 CET] [debug] Could not connect to root@scooter-1.qa.suse.de, Retrying after some seconds...
[2019-11-10T09:34:43.694 CET] [debug] Could not connect to root@scooter-1.qa.suse.de, Retrying after some seconds...
[2019-11-10T09:34:53.699 CET] [debug] Backend process died, backend errors are reported below in the following lines:
Error connecting to <root@scooter-1.qa.suse.de>: Connection refused
[2019-11-10T09:34:53.748 CET] [debug] IPMI: Chassis Power Control: Down/Off
[2019-11-10T09:34:53.748 CET] [debug] flushing frames
[2019-11-10T09:34:53.822 CET] [debug] sending magic and exit
[2019-11-10T09:34:53.822 CET] [debug] received magic close
[2019-11-10T09:34:53.823 CET] [debug] THERE IS NOTHING TO READ 15 4 3
[2019-11-10T09:34:53.823 CET] [debug] killing command server 28319 because test execution ended

I agree that a failed select_console should not fail the job. This is what #59267 is about. This ticket is about the regression that the incomplete which is triggered in a specific module is not notifying about the module which it did in before.

#45062 describes that this did work in the past.

Whether the test module is marked as failed (and e.g. shown as failed module in the various tables) depends on os-autoinst writing the test module result accordingly. Considering the inconsistent error handling in os-autoinst it is likely not very consistent about marking the test module as failed. So it might work in some cases like in #45062 but not generally.

Note that openQA and the result (e.g. incomplete) has not much influence here. Next & Previous and other lists will even show failed modules for passed jobs (if you manage to get a passed job with failed modules into the system).