action #91848: [qe-core] test fails in journalctl - mkdir timed out consistently on aarch64, no useful logs - openQA Tests - openSUSE Project Management Tool

Actions

Copy link

action #91848

closed

[qe-core] test fails in journalctl - mkdir timed out consistently on aarch64, no useful logs

Added by szarate almost 3 years ago. Updated 3 months ago.

Status:

Resolved

Priority:

Normal

Assignee:

mloviska

Category:

Bugs in existing tests

Target version:

Start date:

2021-04-27

Due date:

% Done:

Estimated time:

Difficulty:

Description

Observation¶

openQA test in scenario sle-15-SP3-Online-aarch64-extra_tests_textmode@aarch64 fails in
journalctl

Test suite description¶

Maintainer: QE Core, asmorodskyi,dheidler. Mainly console extratest

Reproducible¶

Fails since (at least) Build 174.1

Expected result¶

Last good: 172.3 (or more recent)

Further details¶

Always latest result in this scenario: latest

Suggestions¶

call $self->SUPER::post_fail_hook; to get more logs, including dmesg and systemd logs
Manually verify why the mkdir command is taking longer than expected.

Actions

Copy link

Updated by mloviska almost 3 years ago

Status changed from Workable to In Progress
Assignee set to mloviska

# umount /dev/md1054

==> PASSED


Test Breakdown:
---------------

# cd /root
mdadm: stopped /dev/md1054
# losetup -d /dev/loop41
# losetup -d /dev/loop42
# losetup -d /dev/loop43
# rm -rf /var/tmp/mdadm_test /var/tmp/mdadm.sh.conf

==> all tests PASSED

IdKcN-0-
# curl --form upload=@mdadm.log --form upname=mdadm-mdadm.log http://10.0.2.2:20143/VMq4kxuE9h6KRlZp/uploadlog/mdadm.log; echo BvG22-$?-
OK: mdadm-mdadm.log
BvG22-0-
# cd
# clear
mkdir -p /etc/systemd/journald.conf.d/; echo ncUEu-$?-
zypper -n in systemd-container | cat; ( exit ${PIPESTATUS[0]} ); echo t3kSR-$?-
Refreshing service 'Basesystem_Module_15_SP3_aarch64'.
Refreshing service 'Desktop_Applications_Module_15_SP3_aarch64'.
Refreshing service 'Development_Tools_Module_15_SP3_aarch64'.
Refreshing service 'SUSE_Linux_Enterprise_Server_15_SP3_aarch64'.
Refreshing service 'Server_Applications_Module_15_SP3_aarch64'.
Loading repository data...
Reading installed packages...
Resolving package dependencies...

The following NEW package is going to be installed:
  systemd-container

It gets created by some predecessor test case, I will update the code. Thanks Santy :)

Actions

Copy link

Updated by mloviska almost 3 years ago

mloviska wrote:

# umount /dev/md1054

==> PASSED


Test Breakdown:
---------------

# cd /root
mdadm: stopped /dev/md1054
# losetup -d /dev/loop41
# losetup -d /dev/loop42
# losetup -d /dev/loop43
# rm -rf /var/tmp/mdadm_test /var/tmp/mdadm.sh.conf

==> all tests PASSED

IdKcN-0-
# curl --form upload=@mdadm.log --form upname=mdadm-mdadm.log http://10.0.2.2:20143/VMq4kxuE9h6KRlZp/uploadlog/mdadm.log; echo BvG22-$?-
OK: mdadm-mdadm.log
BvG22-0-
# cd
# clear
mkdir -p /etc/systemd/journald.conf.d/; echo ncUEu-$?-
zypper -n in systemd-container | cat; ( exit ${PIPESTATUS[0]} ); echo t3kSR-$?-
Refreshing service 'Basesystem_Module_15_SP3_aarch64'.
Refreshing service 'Desktop_Applications_Module_15_SP3_aarch64'.
Refreshing service 'Development_Tools_Module_15_SP3_aarch64'.
Refreshing service 'SUSE_Linux_Enterprise_Server_15_SP3_aarch64'.
Refreshing service 'Server_Applications_Module_15_SP3_aarch64'.
Loading repository data...
Reading installed packages...
Resolving package dependencies...

The following NEW package is going to be installed:
  systemd-container

It gets created by some predecessor test case, I will update the code. Thanks Santy :)

No, it is not the case.
The prompt is not ready https://openqa.suse.de/tests/5904707#step/journalctl/2. Seems like a specific issue in osd. https://openqa.opensuse.org/tests/1693580#step/journalctl/2

Actions

Copy link

Updated by mloviska almost 3 years ago

Seems like the same issue in a different test case occurred:
https://openqa.suse.de/tests/5924972#step/check_default_network_manager/3

Actions

Copy link

Updated by mloviska almost 3 years ago

I have executed 5 runs (sle-15-SP3-Online-aarch64-Build181.1-extra_tests_textmode_no_samba@aarch64) with EXCLUDE_MODULES=samba_adcli this module was added lately and all additional test runs have passed.
Not sure what happens, but it seems like integration samba_adcli + journalctl does not work well together at the moment.

Ooops, I have accidentally scheduled 2 more runs with the original test name. So that makes it 7.:D Sorry!

Hello @martinsmac, would you mind to check samba_adcli integration with journalctl and mdadm, please ? Seems like the serial terminal is not showing prompt/not ready after exiting samba_adcli. Funny that mdadm does not fail, but journalctl executes assert_script_run at this point therefore it causes a test case failure.