Project

General

Profile

action #91848

[qe-core] test fails in journalctl - mkdir timed out consistently on aarch64, no useful logs

Added by szarate 3 months ago. Updated 3 months ago.

Status:
In Progress
Priority:
Normal
Assignee:
Category:
Bugs in existing tests
Target version:
-
Start date:
2021-04-27
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Observation

openQA test in scenario sle-15-SP3-Online-aarch64-extra_tests_textmode@aarch64 fails in
journalctl

Test suite description

Maintainer: QE Core, asmorodskyi,dheidler. Mainly console extratest

Reproducible

Fails since (at least) Build 174.1

Expected result

Last good: 172.3 (or more recent)

Further details

Always latest result in this scenario: latest

Suggestions

  1. call $self->SUPER::post_fail_hook; to get more logs, including dmesg and systemd logs
  2. Manually verify why the mkdir command is taking longer than expected.

History

#1 Updated by mloviska 3 months ago

  • Status changed from Workable to In Progress
  • Assignee set to mloviska
# umount /dev/md1054

==> PASSED


Test Breakdown:
---------------

# cd /root
mdadm: stopped /dev/md1054
# losetup -d /dev/loop41
# losetup -d /dev/loop42
# losetup -d /dev/loop43
# rm -rf /var/tmp/mdadm_test /var/tmp/mdadm.sh.conf

==> all tests PASSED

IdKcN-0-
# curl --form upload=@mdadm.log --form upname=mdadm-mdadm.log http://10.0.2.2:20143/VMq4kxuE9h6KRlZp/uploadlog/mdadm.log; echo BvG22-$?-
OK: mdadm-mdadm.log
BvG22-0-
# cd
# clear
mkdir -p /etc/systemd/journald.conf.d/; echo ncUEu-$?-
zypper -n in systemd-container | cat; ( exit ${PIPESTATUS[0]} ); echo t3kSR-$?-
Refreshing service 'Basesystem_Module_15_SP3_aarch64'.
Refreshing service 'Desktop_Applications_Module_15_SP3_aarch64'.
Refreshing service 'Development_Tools_Module_15_SP3_aarch64'.
Refreshing service 'SUSE_Linux_Enterprise_Server_15_SP3_aarch64'.
Refreshing service 'Server_Applications_Module_15_SP3_aarch64'.
Loading repository data...
Reading installed packages...
Resolving package dependencies...

The following NEW package is going to be installed:
  systemd-container

It gets created by some predecessor test case, I will update the code. Thanks Santy :)

#2 Updated by mloviska 3 months ago

mloviska wrote:

# umount /dev/md1054

==> PASSED


Test Breakdown:
---------------

# cd /root
mdadm: stopped /dev/md1054
# losetup -d /dev/loop41
# losetup -d /dev/loop42
# losetup -d /dev/loop43
# rm -rf /var/tmp/mdadm_test /var/tmp/mdadm.sh.conf

==> all tests PASSED

IdKcN-0-
# curl --form upload=@mdadm.log --form upname=mdadm-mdadm.log http://10.0.2.2:20143/VMq4kxuE9h6KRlZp/uploadlog/mdadm.log; echo BvG22-$?-
OK: mdadm-mdadm.log
BvG22-0-
# cd
# clear
mkdir -p /etc/systemd/journald.conf.d/; echo ncUEu-$?-
zypper -n in systemd-container | cat; ( exit ${PIPESTATUS[0]} ); echo t3kSR-$?-
Refreshing service 'Basesystem_Module_15_SP3_aarch64'.
Refreshing service 'Desktop_Applications_Module_15_SP3_aarch64'.
Refreshing service 'Development_Tools_Module_15_SP3_aarch64'.
Refreshing service 'SUSE_Linux_Enterprise_Server_15_SP3_aarch64'.
Refreshing service 'Server_Applications_Module_15_SP3_aarch64'.
Loading repository data...
Reading installed packages...
Resolving package dependencies...

The following NEW package is going to be installed:
  systemd-container

It gets created by some predecessor test case, I will update the code. Thanks Santy :)

No, it is not the case.
The prompt is not ready https://openqa.suse.de/tests/5904707#step/journalctl/2. Seems like a specific issue in osd. https://openqa.opensuse.org/tests/1693580#step/journalctl/2

#3 Updated by mloviska 3 months ago

Seems like the same issue in a different test case occurred:
https://openqa.suse.de/tests/5924972#step/check_default_network_manager/3

#4 Updated by mloviska 3 months ago

I have executed 5 runs (sle-15-SP3-Online-aarch64-Build181.1-extra_tests_textmode_no_samba@aarch64) with EXCLUDE_MODULES=samba_adcli this module was added lately and all additional test runs have passed.
Not sure what happens, but it seems like integration samba_adcli + journalctl does not work well together at the moment.

Ooops, I have accidentally scheduled 2 more runs with the original test name. So that makes it 7.:D Sorry!

Hello martinsmac, would you mind to check samba_adcli integration with journalctl and mdadm, please ? Seems like the serial terminal is not showing prompt/not ready after exiting samba_adcli. Funny that mdadm does not fail, but journalctl executes assert_script_run at this point therefore it causes a test case failure.

#6 Updated by szarate 3 months ago

Now we're here: https://openqa.suse.de/tests/5939498 so, something else is causing the journal to get stuck...

Also available in: Atom PDF