Project

General

Profile

Actions

action #153766

open

[core][sporadic] Handle wild agetty better in tests/network/setup_multimachine.pm

Added by dheidler 10 months ago. Updated 6 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Bugs in existing tests
Target version:
-
Start date:
2024-01-17
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

https://openqa.opensuse.org/tests/3874464
https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/159#issuecomment-1894506514

This happens around 2 in 100 multi machine jobs so or 1 in 100 individual jobs.

A wild agetty appeared:

[2024-01-16T11:45:41.007648-05:00] [debug] [pid:10982] <<< testapi::type_string(string="(echo qQf4r; bash -eox pipefail /tmp/scriptqQf4r.sh ; echo SCRIPT_FINISHEDqQf4r-\$?-) | tee /dev/ttyS0\n", max_interval=250,
 wait_screen_change=0, wait_still_screen=0, timeout=30, similarity_level=47)
[2024-01-16T11:45:45.176979-05:00] [debug] [pid:10982] tests/network/setup_multimachine.pm:45 called mm_network::setup_static_mm_network -> lib/mm_network.pm:228 called mm_network::configure_static_dns -> lib/mm_
network.pm:130 called testapi::script_output
[2024-01-16T11:45:45.178288-05:00] [debug] [pid:10982] <<< testapi::wait_serial(regexp="SCRIPT_FINISHEDqQf4r-\\d+-", buffer_size=undef, quiet=undef, expect_not_found=0, record_output=1, no_regex=0, timeout=90)
[2024-01-16T11:45:47.235577-05:00] [debug] [pid:10982] >>> testapi::wait_serial: SCRIPT_FINISHEDqQf4r-\d+-: ok
[2024-01-16T11:45:47.235785-05:00] [debug] [pid:10982] tests/network/setup_multimachine.pm:45 called mm_network::setup_static_mm_network -> lib/mm_network.pm:228 called mm_network::configure_static_dns -> lib/mm_
network.pm:132 called testapi::assert_script_run
[2024-01-16T11:45:47.235910-05:00] [debug] [pid:10982] <<< testapi::assert_script_run(cmd="nmcli connection modify 'Welcome to openSUSE Tumbleweed 20240115 - Kernel 6.6.11-1-default (ttyS0).\n\nens4: 10.0.2.102 f
e80::5054:ff:fe12:2\n\n\nsusetest login: ens4' ipv4.dns '10.0.2.3'", quiet=undef, fail_message="", timeout=90)
[2024-01-16T11:45:47.236029-05:00] [debug] [pid:10982] tests/network/setup_multimachine.pm:45 called mm_network::setup_static_mm_network -> lib/mm_network.pm:228 called mm_network::configure_static_dns -> lib/mm_
network.pm:132 called testapi::assert_script_run
[2024-01-16T11:45:47.236129-05:00] [debug] [pid:10982] <<< testapi::type_string(string="nmcli connection modify 'Welcome to openSUSE Tumbleweed 20240115 - Kernel 6.6.11-1-default (ttyS0).\n\nens4: 10.0.2.102 fe80
::5054:ff:fe12:2\n\n\nsusetest login: ens4' ipv4.dns '10.0.2.3'", max_interval=250, wait_screen_change=0, wait_still_screen=0, timeout=30, similarity_level=47)
[2024-01-16T11:45:53.755719-05:00] [debug] [pid:10982] tests/network/setup_multimachine.pm:45 called mm_network::setup_static_mm_network -> lib/mm_network.pm:228 called mm_network::configure_static_dns -> lib/mm_
network.pm:132 called testapi::assert_script_run
[2024-01-16T11:45:53.758424-05:00] [debug] [pid:10982] <<< testapi::type_string(string="; echo Q94Z9-\$?- > /dev/ttyS0\n", max_interval=250, wait_screen_change=0, wait_still_screen=0, timeout=30, similarity_level
=47)
[2024-01-16T11:45:54.885176-05:00] [debug] [pid:10982] tests/network/setup_multimachine.pm:45 called mm_network::setup_static_mm_network -> lib/mm_network.pm:228 called mm_network::configure_static_dns -> lib/mm_
network.pm:132 called testapi::assert_script_run
[2024-01-16T11:45:54.885778-05:00] [debug] [pid:10982] <<< testapi::wait_serial(regexp=qr/Q94Z9-\d+-/u, quiet=undef, buffer_size=undef, expect_not_found=0, timeout=90, no_regex=0, record_output=undef)
[2024-01-16T11:45:56.941627-05:00] [debug] [pid:10982] >>> testapi::wait_serial: (?^u:Q94Z9-\d+-): ok
[2024-01-16T11:45:56.998284-05:00] [info] [pid:10982] ::: basetest::runtest: # Test died: command 'nmcli connection modify 'Welcome to openSUSE Tumbleweed 20240115 - Kernel 6.6.11-1-default (ttyS0).

  ens4: 10.0.2.102 fe80::5054:ff:fe12:2


  susetest login: ens4' ipv4.dns '10.0.2.3'' failed at /usr/lib/os-autoinst/testapi.pm line 926.
        testapi::assert_script_run("nmcli connection modify 'Welcome to openSUSE Tumbleweed 20240"...) called at opensuse/lib/mm_network.pm line 132
        mm_network::configure_static_dns(HASH(0x563622f61c98), "is_nm", 1) called at opensuse/lib/mm_network.pm line 228
        mm_network::setup_static_mm_network("10.0.2.102/24") called at opensuse/tests/network/setup_multimachine.pm line 45
        setup_multimachine::run(setup_multimachine=HASH(0x563626081bd0)) called at /usr/lib/os-autoinst/basetest.pm line 352
        eval {...} called at /usr/lib/os-autoinst/basetest.pm line 346
        basetest::runtest(setup_multimachine=HASH(0x563626081bd0)) called at /usr/lib/os-autoinst/autotest.pm line 415
        eval {...} called at /usr/lib/os-autoinst/autotest.pm line 415
        autotest::runalltests() called at /usr/lib/os-autoinst/autotest.pm line 272
        eval {...} called at /usr/lib/os-autoinst/autotest.pm line 272
        autotest::run_all() called at /usr/lib/os-autoinst/autotest.pm line 323
        autotest::__ANON__(Mojo::IOLoop::ReadWriteProcess=HASH(0x563627209df0)) called at /usr/lib/perl5/vendor_perl/5.38.2/Mojo/IOLoop/ReadWriteProcess.pm line 329
        eval {...} called at /usr/lib/perl5/vendor_perl/5.38.2/Mojo/IOLoop/ReadWriteProcess.pm line 329
        Mojo::IOLoop::ReadWriteProcess::_fork(Mojo::IOLoop::ReadWriteProcess=HASH(0x563627209df0), CODE(0x56362720a060)) called at /usr/lib/perl5/vendor_perl/5.38.2/Mojo/IOLoop/ReadWriteProcess.pm line 492
        Mojo::IOLoop::ReadWriteProcess::start(Mojo::IOLoop::ReadWriteProcess=HASH(0x563627209df0)) called at /usr/lib/os-autoinst/autotest.pm line 325
        autotest::start_process() called at /usr/lib/os-autoinst/OpenQA/Isotovideo/Runner.pm line 94
        OpenQA::Isotovideo::Runner::start_autotest(OpenQA::Isotovideo::Runner=HASH(0x563621c6b1d0)) called at /usr/bin/isotovideo line 192
        eval {...} called at /usr/bin/isotovideo line 181

Related issues 2 (0 open2 closed)

Related to openQA Project - action #138302: Ensure automated openQA tests verify that os-autoinst-setup-multi-machine sets up valid networking size:MResolveddheidler2023-07-192024-01-19

Actions
Related to openQA Project - action #155170: [openqa-in-openqa] [sporadic] test fails in test_running: parallel_failed size:MResolvedybonatakis2024-02-082024-02-29

Actions
Actions #1

Updated by dheidler 10 months ago

  • Related to action #138302: Ensure automated openQA tests verify that os-autoinst-setup-multi-machine sets up valid networking size:M added
Actions #2

Updated by dheidler 10 months ago

  • Description updated (diff)
Actions #3

Updated by dheidler 10 months ago

  • Description updated (diff)
Actions #4

Updated by okurz 10 months ago

  • Project changed from openQA Project to openQA Tests
  • Subject changed from Handle wild agetty better in tests/network/setup_multimachine.pm to [core][sporadic] Handle wild agetty better in tests/network/setup_multimachine.pm
  • Category changed from Regressions/Crashes to Bugs in existing tests
  • Priority changed from Normal to High

@qe-core Fixing this ticket would help us in the tools team to have stable openQA-in-openQA tests

Actions #5

Updated by dheidler 10 months ago

Suggestion: Try to use schedule a lot of https://openqa.opensuse.org/tests/3873052 to reproduce it.

Actions #6

Updated by tinita 9 months ago

  • Related to action #155170: [openqa-in-openqa] [sporadic] test fails in test_running: parallel_failed size:M added
Actions #7

Updated by tinita 8 months ago

This ticket was set to High priority but was not updated within the SLO period. Please consider picking up this ticket or just set the ticket to the next lower priority.

Actions #8

Updated by slo-gin 6 months ago

  • Priority changed from High to Normal

The ticket will be set to the next lower priority Normal

Actions

Also available in: Atom PDF