Project

General

Profile

Actions

action #124898

closed

[qe-core] test fails in btrfsmaintenance

Added by mgrifalconi about 1 year ago. Updated about 1 year ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Bugs in existing tests
Target version:
-
Start date:
2023-02-21
Due date:
% Done:

100%

Estimated time:
Difficulty:

Description

Observation

openQA test in scenario sle-15-SP1-Server-DVD-Updates-x86_64-mau-filesystem@64bit fails in
btrfsmaintenance

Test suite description

Testsuite maintained at https://gitlab.suse.de/qa-maintenance/qam-openqa-yml. Run filesystem tests against aggregated test repo

Reproducible

Fails since (at least) Build nfs_qa_run

Expected result

Last good: nfs_qa_run (or more recent)

Further details

Always latest result in this scenario: latest

Actions #1

Updated by mgrifalconi about 1 year ago

seems to happen only on 15-sp1 and seems not caused by an update request: https://openqa.suse.de/tests/10546031#comments

Actions #2

Updated by ph03nix about 1 year ago

  • Status changed from New to In Progress
  • Assignee set to ph03nix
Actions #3

Updated by ph03nix about 1 year ago

Issue can be reproduced in reduced test run: https://duck-norris.qe.suse.de/tests/11944#step/btrfsmaintenance/20

Actions #4

Updated by ph03nix about 1 year ago

Incident difference to last good:

+   27801:290244   systemd-presets-common-SUSE
+   27849:290236   php7
+   27868:290296   poppler,poppler-qt5

I'm assuming https://smelt.suse.de/incident/27801/ is causing those issues.

Actions #5

Updated by ph03nix about 1 year ago

From the journal log (see https://duck-norris.qe.suse.de/tests/11945#downloads)

EDIT: This is likely coming from openQA (sysrq: SysRq : Show State - https://github.com/torvalds/linux/blob/master/Documentation/admin-guide/sysrq.rst)

Feb 21 08:30:58.682873 susetest root[2095]: ### Beginning of show_tasks_in_blocked_state
Feb 21 08:30:59.848488 susetest kernel: sysrq: SysRq : Show State
Feb 21 08:30:59.848626 susetest kernel:   task                        PC stack   pid father
Feb 21 08:30:59.848722 susetest kernel: systemd         S    0     1      0 0x00000000
Feb 21 08:30:59.848781 susetest kernel: Call Trace:
Feb 21 08:30:59.848838 susetest kernel:  __schedule+0x27f/0x840
Feb 21 08:30:59.848951 susetest kernel:  schedule+0x28/0x80
Feb 21 08:30:59.849017 susetest kernel:  schedule_hrtimeout_range_clock+0x168/0x180
Feb 21 08:30:59.849073 susetest kernel:  ? ep_scan_ready_list.constprop.18+0x1ea/0x1f0
Feb 21 08:30:59.849129 susetest kernel:  ep_poll+0x26e/0x370
Feb 21 08:30:59.849186 susetest kernel:  ? wake_up_q+0x80/0x80
Feb 21 08:30:59.849244 susetest kernel:  SyS_epoll_pwait+0x173/0x1f0
Feb 21 08:30:59.849300 susetest kernel:  ? _copy_to_user+0x22/0x30
Feb 21 08:30:59.849357 susetest kernel:  ? SyS_clock_gettime+0x88/0xb0
Feb 21 08:30:59.849416 susetest kernel:  do_syscall_64+0x74/0x150
Feb 21 08:30:59.849474 susetest kernel:  entry_SYSCALL_64_after_hwframe+0x76/0xdb
Feb 21 08:30:59.849530 susetest kernel: RIP: 0033:0x7f81609b0e10
Feb 21 08:30:59.849586 susetest kernel: RSP: 002b:00007ffe4035d218 EFLAGS: 00000246 ORIG_RAX: 0000000000000119
Feb 21 08:30:59.849651 susetest kernel: RAX: ffffffffffffffda RBX: 000055baf7da5aa0 RCX: 00007f81609b0e10
Feb 21 08:30:59.849708 susetest kernel: RDX: 000000000000002f RSI: 00007ffe4035d220 RDI: 0000000000000004
Feb 21 08:30:59.849764 susetest kernel: RBP: 00007ffe4035d560 R08: 0000000000000000 R09: 0000000000000008
Feb 21 08:30:59.849821 susetest kernel: R10: ffffffffffffffff R11: 0000000000000246 R12: 0000000000000001
Feb 21 08:30:59.849877 susetest kernel: R13: ffffffffffffffff R14: 00007ffe4035d220 R15: 0000000000000001
Feb 21 08:30:59.849933 susetest kernel: kthreadd        S    0     2      0 0x00000000
Feb 21 08:30:59.849990 susetest kernel: Call Trace:

Something's fishy.

Actions #6

Updated by ph03nix about 1 year ago

Issue is introduced by one of the MU: https://duck-norris.qe.suse.de/tests/11958#step/btrfsmaintenance/17 (test run without any test issues)

Clone the same test run only with 27801: https://duck-norris.qe.suse.de/tests/11963#dependencies


I don't fully understand the issue and how to reproduce it in a 15-SP1 VM. Still investigating.

Actions #7

Updated by ph03nix about 1 year ago

Failure with 27801: https://duck-norris.qe.suse.de/tests/11963#step/btrfsmaintenance/20

So, this is a positive match, that 27801 cases this test failure. But I still need to understand and isolate the actual issue.

Actions #8

Updated by ph03nix about 1 year ago

  • % Done changed from 0 to 80

Conclusion: The issue is that the btrfsmaintenance-refresh.service is not enabled after installation. The issue can't be reproduced in a VM by installing the update. It only occurs when the patch is applied during the installation.

Actions #9

Updated by ph03nix about 1 year ago

  • % Done changed from 80 to 90

Issue reported in #team-lsg-qe-openqa-review

Actions #10

Updated by ph03nix about 1 year ago

Actions #12

Updated by ph03nix about 1 year ago

Merged. Last topic I want to work on is to figure, where this is scheduled for SLES 15 and if not, why it's not there.

Actions #13

Updated by ph03nix about 1 year ago

Re-add btrfsmaintenance on SLES15-SP2-SP4

Actions #14

Updated by ph03nix about 1 year ago

Now looking into the Product-QA test runs.

Actions #15

Updated by ph03nix about 1 year ago

Product-QA test runs where it should be

Actions #16

Updated by ph03nix about 1 year ago

  • Status changed from In Progress to Resolved
  • % Done changed from 90 to 100

Since https://openqa.suse.de/tests/10563593 contains btrfsmaintenance, this ticket is resolved.

Actions

Also available in: Atom PDF