Project

General

Profile

Actions

action #54329

closed

24-worker-overall.t test assumes cpu_opmode capability will always be present, but it isn't

Added by AdamWill almost 5 years ago. Updated over 4 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Target version:
-
Start date:
2019-07-16
Due date:
% Done:

0%

Estimated time:

Description

There's a subtest in 24-worker-overall.t:

my $capabilities      = $worker->capabilities;
my @capabilities_keys = sort keys %$capabilities;
is_deeply(
    \@capabilities_keys,
    [
        qw(
          cpu_arch cpu_modelname cpu_opmode host instance isotovideo_interface_version
          mem_max websocket_api_version worker_class
          )
    ],
    'capabilities contain expected information'
) or diag explain \@capabilities_keys;

that's assuming there will always be a cpu_opmode key in the capabilities hash. But...there won't. Here's the code that creates it, from lib/OpenQA/Worker.pm:

    open(my $LSCPU, "-|", "LC_ALL=C lscpu");
    for my $line (<$LSCPU>) {
        chomp $line;
        if ($line =~ m/Model name:\s+(.+)$/) {
            $caps->{cpu_modelname} = $1;
        }
        if ($line =~ m/Architecture:\s+(.+)$/) {
            $caps->{cpu_arch} = $1;
        }
        if ($line =~ m/CPU op-mode\(s\):\s+(.+)$/) {
            $caps->{cpu_opmode} = $1;
        }

so, we only get cpu_opmode if the lscpu output has "CPU op-mode(s): [something]" in it. But this is not always true. For a specific example, here's a ppc64 system I have access to:

[adamwill@openqa-ppc64le-01 ~][PROD]$ lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 80
On-line CPU(s) list: 0,8,16,24,32,40,48,56,64,72
Off-line CPU(s) list: 1-7,9-15,17-23,25-31,33-39,41-47,49-55,57-63,65-71,73-79
Thread(s) per core: 1
Core(s) per socket: 10
Socket(s): 1
NUMA node(s): 1
Model: 2.0 (pvr 004d 0200)
Model name: POWER8 (raw), altivec supported
CPU max MHz: 3491.0000
CPU min MHz: 2061.0000
L1d cache: 64K
L1i cache: 32K
L2 cache: 512K
L3 cache: 8192K
NUMA node0 CPU(s): 0,8,16,24,32,40,48,56,64,72

This is a problem for Fedora because noarch packages (like openQA) can be built on any builder. If the package happens to get built on an ppc64 builder, this test fails and the build fails.

Not sure if the better fix for this is to assign cpu_opmode some empty value if no line is found, somehow infer a correct value, or just correct the test (and any other code that makes the same assumption?) to not assume it will be present.

Actions #1

Updated by AdamWill almost 5 years ago

Does cpu_opmode have any point at all? AFAICS nothing actually uses it, in openQA or os-autoinst.

Actions #2

Updated by mkittler almost 5 years ago

  • Status changed from New to In Progress
  • Assignee set to mkittler

I'm going to fix it by simply removing the check from the tests. As you're already saying it is not used anyways.

I was also wondering about this when restructuring the worker code recently. But I didn't remove it because I didn't want to touch too much of the actual functionality at this point.

Actions #3

Updated by mkittler almost 5 years ago

I suppose it has been added as "little extra" because we get this information from lscpu for free and it might be useful at some point. So I'll keep it but remove it from the check for required keys.

Actions #4

Updated by mkittler almost 5 years ago

  • Status changed from In Progress to Resolved

should be fixed on master

Actions #5

Updated by AdamWill almost 5 years ago

Per https://koji.fedoraproject.org/koji/taskinfo?taskID=36579421 and https://kojipkgs.fedoraproject.org//work/tasks/9421/36579421/build.log , we seem to have the same issue with cpu_modelname on s390x - that was an openQA package build that happened to hit an s390x build host, it fails with:

BUILDSTDERR:     #   Failed test 'capabilities contain expected information'
BUILDSTDERR:     #   at ./t/24-worker-overall.t line 89.
BUILDSTDERR:     #     Structures begin differing at:
BUILDSTDERR:     #          $got->[1] = 'host'
BUILDSTDERR:     #     $expected->[1] = 'cpu_modelname'
BUILDSTDERR:     # {
BUILDSTDERR:     #   'cpu_arch' => 's390x',
BUILDSTDERR:     #   'host' => 'buildvm-s390x-11.s390.fedoraproject.org',
BUILDSTDERR:     #   'instance' => 1,
BUILDSTDERR:     #   'isotovideo_interface_version' => '14',
BUILDSTDERR:     #   'mem_max' => 8044,
BUILDSTDERR:     #   'websocket_api_version' => 1,
BUILDSTDERR:     #   'worker_class' => 'qemu_i386,qemu_x86_64'
BUILDSTDERR:     # }

Here's how the lscpu output looks on one of those:

Architecture:        s390x
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Big Endian
CPU(s):              4
On-line CPU(s) list: 0-3
Thread(s) per core:  1
Core(s) per socket:  1
Socket(s) per book:  1
Book(s) per drawer:  1
Drawer(s):           4
NUMA node(s):        1
Vendor ID:           IBM/S390
Machine type:        2964
CPU dynamic MHz:     5000
CPU static MHz:      5000
BogoMIPS:            3033.00
Hypervisor:          KVM/Linux
Hypervisor vendor:   KVM
Virtualization type: full
Dispatching mode:    horizontal
L1d cache:           128K
L1i cache:           96K
L2d cache:           2048K
L2i cache:           2048K
L3 cache:            65536K
L4 cache:            491520K
NUMA node0 CPU(s):   0-3
Flags:               esan3 zarch stfle msa ldisp eimm dfp edat etf3eh highgprs te vx sie
Actions #6

Updated by mkittler over 4 years ago

cpu_modelname is excluded on the latest master.

Actions

Also available in: Atom PDF