Project

General

Profile

Actions
Copy link

action #63610

closed

[kernel][ltp][spvm] - test fails in pkey01 on spvm

Added by pcervinka almost 5 years ago. Updated about 4 years ago.

Status:
Resolved
Priority:
High
Assignee:
pcervinka
Category:
Bugs in existing tests
Target version:
QE Kernel - QE Kernel Done
Start date:
2020-02-19
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Observation

openQA test in scenario sle-15-SP2-Online-ppc64le-ltp_syscalls_spvm@ppc64le-spvm fails in
pkey01

Test suite description

Maintainer: pcervinka, pvorel

Reproducible

Fails since (at least) Build 139.1

Expected result

Last good: (unknown) (or more recent)

Further details

Always latest result in this scenario: latest

Summary:
passed 0
failed 0
skipped 0
warnings 0

TEST pkey01 COMPLETE >>> 2

kQ8Ut-0-

2020-02-14T11:16:10.473 CET] [debug] ||| starting pkey01 tests/kernel/run_ltp.pm
[2020-02-14T11:16:10.480 CET] [debug] /var/lib/openqa/cache/openqa.suse.de/tests/sle/tests/kernel/run_ltp.pm:309 called testapi::type_string
[2020-02-14T11:16:10.480 CET] [debug] <<< testapi::type_string(string="(pkey01; echo \"### TEST pkey01 COMPLETE >>> \$?\") | tee /dev/sshserial\n", max_interval=250, wait_screen_changes=0, wait_still_screen=0, timeout=30, similarity_level=47)

Summary:
passed 0
failed 0
skipped 0
warnings 0

TEST pkey01 COMPLETE >>> 2

[2020-02-14T11:16:13.260 CET] [debug] /var/lib/openqa/cache/openqa.suse.de/tests/sle/tests/kernel/run_ltp.pm:311 called testapi::wait_serial
[2020-02-14T11:16:13.260 CET] [debug] <<< testapi::wait_serial(no_regex=0, regexp=qr/### TEST pkey01 COMPLETE >>> \d+/u, quiet=undef, timeout=900, expect_not_found=0, buffer_size=undef, record_output=1)
[2020-02-14T11:16:14.379 CET] [debug] >>> testapi::wait_serial: (?u:### TEST pkey01 COMPLETE >>> \d+): ok
[2020-02-14T11:16:14.380 CET] [debug] /var/lib/openqa/cache/openqa.suse.de/tests/sle/tests/kernel/run_ltp.pm:337 called testapi::script_run
[2020-02-14T11:16:14.380 CET] [debug] <<< testapi::script_run(cmd="vmstat -w", timeout=undef, quiet=undef, output="")
[2020-02-14T11:16:14.380 CET] [debug] /var/lib/openqa/cache/openqa.suse.de/tests/sle/tests/kernel/run_ltp.pm:337 called testapi::script_run
[2020-02-14T11:16:14.380 CET] [debug] <<< testapi::type_string(string="vmstat -w", max_interval=250, wait_screen_changes=0, wait_still_screen=0, timeout=30, similarity_level=47)
[2020-02-14T11:16:14.684 CET] [debug] /var/lib/openqa/cache/openqa.suse.de/tests/sle/tests/kernel/run_ltp.pm:337 called testapi::script_run
[2020-02-14T11:16:14.684 CET] [debug] <<< testapi::type_string(string="; echo kQ8Ut-\$?- > /dev/sshserial\n", max_interval=250, wait_screen_changes=0, wait_still_screen=0, timeout=30, similarity_level=47)
kQ8Ut-0-
[2020-02-14T11:16:15.914 CET] [debug] /var/lib/openqa/cache/openqa.suse.de/tests/sle/tests/kernel/run_ltp.pm:337 called testapi::script_run
[2020-02-14T11:16:15.915 CET] [debug] <<< testapi::wait_serial(no_regex=0, regexp=qr/kQ8Ut-\d+-/, quiet=undef, timeout=30, expect_not_found=0, buffer_size=undef, record_output=undef)
[2020-02-14T11:16:17.028 CET] [debug] >>> testapi::wait_serial: (?:kQ8Ut-\d+-): ok
[2020-02-14T11:16:17.037 CET] [debug] ||| finished pkey01 kernel at 2020-02-14 10:16:17 (7 s)

pkey01 test basically does nothing in 6 minutes with no output.

Manually tested on vugava-3 and it works fine, so it is not a bug.

Related issues 1 (0 open — 1 closed)

Related to openQA Tests - action #63943: [kernel][ltp] test fails in pkey01 - Setup hugepages before test executionResolvedpcervinka2020-02-28

Actions
Actions
Copy link
 #2

Updated by pcervinka over 4 years ago

  • Status changed from New to In Progress
  • Priority changed from Normal to High
  • Target version changed from 454 to 445
Actions
Copy link
 #3

Updated by pcervinka over 4 years ago

Looks like hugepages are not available.

You can specify it with option -z
COMMAND: /opt/ltp/bin/ltp-pan -e -S -a 8690 -n 8690 -p -f /tmp/ltp-e6swym63uL/alltests -l /opt/ltp/results/LTP_RUN_ON-2020_02_21-03h_13m_21s.log -C /opt/ltp/output/LTP_RUN_ON-2020_02_21-03h_13m_21s.failed -T /opt/ltp/output/LTP_RUN_ON-2020_02_21-03h_13m_21s.tconf
INFO: Restricted to pkey
LOG File: /opt/ltp/results/LTP_RUN_ON-2020_02_21-03h_13m_21s.log
FAILED COMMAND File: /opt/ltp/output/LTP_RUN_ON-2020_02_21-03h_13m_21s.failed
TCONF COMMAND File: /opt/ltp/output/LTP_RUN_ON-2020_02_21-03h_13m_21s.tconf
Running tests.......
<<>>
tag=pkey01 stime=1582272807
cmdline="pkey01"
contacts=""
analysis=exit
<<>>
tst_test.c:1217: INFO: Timeout per run is 0h 05m 00s
incrementing stop
pkey01.c:69: BROK: nr_hugepages = 0, but expect 1

Actions
Copy link
 #4

Updated by pcervinka over 4 years ago

grenache-2:/boot # ls /sys/kernel/mm/hugepages/
hugepages-16384kB hugepages-16777216kB
grenache-2:/boot # ls /proc/sys/vm/nr_hugepages
/proc/sys/vm/nr_hugepages
grenache-2:/boot # cat /proc/sys/vm/nr_hugepages
0

Actions
Copy link
 #5

Updated by pcervinka over 4 years ago

Few minutes later on grenache and started to work....
pkey01.c:187: PASS: Child ended by SIGSEGV as expected
pkey01.c:191: INFO: Remove PKEY_DISABLE_WRITE from the buffer
pkey01.c:205: PASS: Read & Write buffer success, buffer[0] = 97

Summary:
passed 96
failed 0
skipped 0
warnings 0
<<>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=2
<<>>
INFO: ltp-pan reported all tests PASS
LTP Version: 20200120

grenache-2:/opt/ltp #
grenache-2:/opt/ltp #
grenache-2:/opt/ltp # cat /proc/sys/vm/nr_hugepages
0
grenache-2:/opt/ltp # ls /proc/sys/vm/nr_hugepages
/proc/sys/vm/nr_hugepages
grenache-2:/opt/ltp #

Actions
Copy link
 #6

Updated by pcervinka over 4 years ago

And another run fails...
<<>>
tst_test.c:1217: INFO: Timeout per run is 0h 05m 00s
incrementing stop
pkey01.c:69: BROK: nr_hugepages = 0, but expect 1

Actions
Copy link
 #7

Updated by pcervinka over 4 years ago

It randomly fails (on PowerVM only) in pkey01.c:

64      int val;
65      SAFE_FILE_PRINTF(PATH_VM_NRHPS, "%d", 1);
66      SAFE_FILE_SCANF(PATH_VM_NRHPS, "%d", &val);
67      if (val != 1)
68          tst_brk(TBROK, "nr_hugepages = %d, but expect %d",
69                  val, 1);

Line 65 writes 1 to /proc/sys/vm/nr_hugepages and line 66 reads from /proc/sys/vm/nr_hugepages.
If content of /proc/sys/vm/nr_hugepages is not 1, it fails.

Actions
Copy link
 #8

Updated by pcervinka over 4 years ago

  • Status changed from In Progress to Feedback

Should be fixed in https://patchwork.ozlabs.org/project/ltp/list/?series=157441.
Patch will be merged soon to LTP, so let's wait for next results.

Actions
Copy link
 #9

Updated by pvorel over 4 years ago

Fix merged. I tested it on pinotage.arch.suse.de, but it worked there before, so we need to wait for next build.

Actions
Copy link
 #10

Updated by pcervinka over 4 years ago

Pinotage is not spvm/PowerVM system, and vugava-* in orthos is Power9, i was able to randomly reproduce it only on osd grenache worker, which is Power8 spvm backend.

Actions
Copy link
 #11

Updated by pcervinka over 4 years ago

  • Status changed from Feedback to Resolved
Actions
Copy link
 #12

Updated by metan over 4 years ago

  • Target version changed from 445 to 457
Actions
Copy link
 #13

Updated by pcervinka over 4 years ago

  • Related to action #63943: [kernel][ltp] test fails in pkey01 - Setup hugepages before test execution added
Actions
Copy link
 #14

Updated by pcervinka about 4 years ago

  • Target version changed from 457 to QE Kernel Done
Actions
Copy link

Also available in: Atom PDF