Project

General

Profile

Actions
Copy link

action #109638

closed

coordination #103323: [epic] BCI testing

[bci-tests] Add hability to execute/skip tests depending on the arch

Added by jlausuch about 2 years ago. Updated about 2 years ago.

Status:
Resolved
Priority:
High
Assignee:
-
Target version:
-
Start date:
2022-04-07
Due date:
% Done:

0%

Estimated time:

Description

Some of the tests are not properly working on all the architectures and that's completely fine. Supporting 100% of the cases is a non-sense.

We need a feature to skip a test for certain architectures in https://gitlab.suse.de/dancermak/bci-tests.

Related issues 1 (0 open — 1 closed)

Related to Containers - action #107857: Enable and fix BCI tests on aarch64 (except dotnet family)Resolvedph03nix2022-03-03

Actions
Actions
Copy link
 #1

Updated by jlausuch about 2 years ago

  • Related to action #107857: Enable and fix BCI tests on aarch64 (except dotnet family) added
Actions
Copy link
 #2

Updated by ph03nix about 2 years ago

Reson to do so:

6 test runs for NodeJS all failing with Error: Timeout of 2000ms exceeded in the express test repo - See the 4.51_nodejs-16-image build:

To be sure that this is a timeout due to missing computational resources, I performed the following tests:

  • Use the registry.suse.com/bci/nodejs:16 container on my RPi VM and run the express test: PASSING
  • Use the registry.suse.com/bci/nodejs:16 container on my emulated aarch64 VM and run the express test: FAIL (timeout)

The test procedure was

git clone https://github.com/expressjs/express.git
cd express
npm config set shrinkwrap false && npm rm --silent --save-dev connect-redis && npm run test-ci && npm run lint

To me it looks very much like a timeout due to a too tight timeout because our aarch64 openQA workers are not fast enough.

Actions
Copy link
 #3

Updated by ph03nix about 2 years ago

The error happens when running npm run test-ci:

  1) Router
       should not stack overflow with many registered routes:
     Error: Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/opt/express/test/Router.js)
      at /opt/express/test/Router.js:73:11
      at Layer.handle [as handle_request] (lib/router/layer.js:35:103)
      at next (lib/router/route.js:36:802)
      at Route.dispatch (lib/router/route.js:34:677)
      at Layer.handle [as handle_request] (lib/router/layer.js:35:103)
      at /opt/express/lib/router/index.js:82:415
      at Function.process_params (lib/router/index.js:93:170)
      at next (lib/router/index.js:82:31)
      at Function.handle (lib/router/index.js:67:194)
      at Context.<anonymous> (test/Router.js:76:12)
      at processImmediate (node:internal/timers:464:21)

It points nicely to test/Router.js:73:11

Full log: http://pasta.qam.suse.de/VxYTqXFr

Actions
Copy link
 #4

Updated by ph03nix about 2 years ago

After fixing npm on aarch64, I think this ticket is no longer required?

Actions
Copy link
 #5

Updated by jlausuch about 2 years ago

  • Status changed from Workable to Resolved

Not applicable any more, but anyway, the feature is available in BCI-tests:
https://github.com/SUSE/BCI-tests/blob/main/tests/test_python.py#L173
Example for tensorflow (only on x86_64):

@pytest.mark.skipif(
    # skip test if architecture is not x86.
    LOCALHOST.system_info.arch != "x86_64",
    reason="Tensorflow python library tested on x86_64",
)
Actions
Copy link

Also available in: Atom PDF