Project

General

Profile

Actions
Copy link

action #136136

closed

[qe-core] jobs scheduled for currently not available worker31 "ovs-server+client"

Added by okurz 10 months ago. Updated 9 months ago.

Status:
Resolved
Priority:
High
Assignee:
dzedro
Category:
Bugs in existing tests
Target version:
-
Start date:
2023-09-20
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Observation

https://openqa.suse.de/tests/12196999# "ovs-server" is scheduled for WORKER_CLASS combination "qemu_x86_64,tap,worker31" which is currently not fulfillable because the specific machine is not available as an openQA worker. The worker specific pinning needs to be removed.

Related issues 1 (0 open — 1 closed)

Related to openQA Tests - action #135818: [kernel] minimal reproducer for many multi-machine test failures in "ovs-client+ovs-server" test scenario when tests are run across different workersResolvedpcervinka2023-08-15

Actions
Actions
Copy link
 #1

Updated by okurz 10 months ago

  • Subject changed from jobs scheduled for currently not available worker31 "ovs-server+client" to [qe-core] jobs scheduled for currently not available worker31 "ovs-server+client"
Actions
Copy link
 #2

Updated by okurz 10 months ago

  • Related to action #135818: [kernel] minimal reproducer for many multi-machine test failures in "ovs-client+ovs-server" test scenario when tests are run across different workers added
Actions
Copy link
 #3

Updated by okurz 10 months ago

  • Priority changed from Urgent to High

To handle the currently scheduled stuck jobs I did

for i in $(openqa-cli api --osd jobs state=scheduled | jq -r '.jobs | .[] | select(.settings.WORKER_CLASS|match("worker31")) | .id'); do openqa-cli api --osd -X post jobs/$i/comments text="poo#136136" && openqa-clone-job --skip-chained-deps --within-instance https://openqa.suse.de/tests/$i WORKER_CLASS="qemu_x86_64,tap"; done

yielding

{"id":1033271}
Cloning parents of sle-15-SP6-Online-x86_64-Build20.1-ovs-server@64bit
Cloning children of sle-15-SP6-Online-x86_64-Build20.1-ovs-server@64bit
Cloning parents of sle-15-SP6-Online-x86_64-Build20.1-ovs-client@64bit
2 jobs have been created:
 - sle-15-SP6-Online-x86_64-Build20.1-ovs-server@64bit -> https://openqa.suse.de/tests/12207581
 - sle-15-SP6-Online-x86_64-Build20.1-ovs-client@64bit -> https://openqa.suse.de/tests/12207582
{"id":1033272}
Cloning parents of sle-15-SP6-Online-x86_64-Build20.1-ovs-client@64bit
Cloning parents of sle-15-SP6-Online-x86_64-Build20.1-ovs-server@64bit
Cloning children of sle-15-SP6-Online-x86_64-Build20.1-ovs-server@64bit
2 jobs have been created:
 - sle-15-SP6-Online-x86_64-Build20.1-ovs-server@64bit -> https://openqa.suse.de/tests/12207584
 - sle-15-SP6-Online-x86_64-Build20.1-ovs-client@64bit -> https://openqa.suse.de/tests/12207583
{"id":1033273}
Cloning parents of sle-15-SP3-Server-DVD-Updates-x86_64-Build20230919-1-ovs-server@64bit
Cloning children of sle-15-SP3-Server-DVD-Updates-x86_64-Build20230919-1-ovs-server@64bit
Cloning parents of sle-15-SP3-Server-DVD-Updates-x86_64-Build20230919-1-ovs-client@64bit
2 jobs have been created:
 - sle-15-SP3-Server-DVD-Updates-x86_64-Build20230919-1-ovs-server@64bit -> https://openqa.suse.de/tests/12207585
 - sle-15-SP3-Server-DVD-Updates-x86_64-Build20230919-1-ovs-client@64bit -> https://openqa.suse.de/tests/12207586
{"id":1033274}
Cloning parents of sle-15-SP3-Server-DVD-Updates-x86_64-Build20230919-1-ovs-client@64bit
Cloning parents of sle-15-SP3-Server-DVD-Updates-x86_64-Build20230919-1-ovs-server@64bit
Cloning children of sle-15-SP3-Server-DVD-Updates-x86_64-Build20230919-1-ovs-server@64bit
2 jobs have been created:
 - sle-15-SP3-Server-DVD-Updates-x86_64-Build20230919-1-ovs-server@64bit -> https://openqa.suse.de/tests/12207587
 - sle-15-SP3-Server-DVD-Updates-x86_64-Build20230919-1-ovs-client@64bit -> https://openqa.suse.de/tests/12207588
{"id":1033275}
Cloning parents of sle-15-SP4-Server-DVD-Updates-x86_64-Build20230919-1-ovs-server@64bit
Cloning children of sle-15-SP4-Server-DVD-Updates-x86_64-Build20230919-1-ovs-server@64bit
Cloning parents of sle-15-SP4-Server-DVD-Updates-x86_64-Build20230919-1-ovs-client@64bit
2 jobs have been created:
 - sle-15-SP4-Server-DVD-Updates-x86_64-Build20230919-1-ovs-server@64bit -> https://openqa.suse.de/tests/12207590
 - sle-15-SP4-Server-DVD-Updates-x86_64-Build20230919-1-ovs-client@64bit -> https://openqa.suse.de/tests/12207589
{"id":1033276}
Cloning parents of sle-15-SP4-Server-DVD-Updates-x86_64-Build20230919-1-ovs-client@64bit
Cloning parents of sle-15-SP4-Server-DVD-Updates-x86_64-Build20230919-1-ovs-server@64bit
Cloning children of sle-15-SP4-Server-DVD-Updates-x86_64-Build20230919-1-ovs-server@64bit
2 jobs have been created:
 - sle-15-SP4-Server-DVD-Updates-x86_64-Build20230919-1-ovs-server@64bit -> https://openqa.suse.de/tests/12207592
 - sle-15-SP4-Server-DVD-Updates-x86_64-Build20230919-1-ovs-client@64bit -> https://openqa.suse.de/tests/12207591
{"id":1033277}
Cloning parents of sle-15-SP5-Server-DVD-Updates-x86_64-Build20230919-1-ovs-server@64bit
Cloning children of sle-15-SP5-Server-DVD-Updates-x86_64-Build20230919-1-ovs-server@64bit
Cloning parents of sle-15-SP5-Server-DVD-Updates-x86_64-Build20230919-1-ovs-client@64bit
2 jobs have been created:
 - sle-15-SP5-Server-DVD-Updates-x86_64-Build20230919-1-ovs-server@64bit -> https://openqa.suse.de/tests/12207594
 - sle-15-SP5-Server-DVD-Updates-x86_64-Build20230919-1-ovs-client@64bit -> https://openqa.suse.de/tests/12207593
{"id":1033278}
Cloning parents of sle-15-SP5-Server-DVD-Updates-x86_64-Build20230919-1-ovs-client@64bit
Cloning parents of sle-15-SP5-Server-DVD-Updates-x86_64-Build20230919-1-ovs-server@64bit
Cloning children of sle-15-SP5-Server-DVD-Updates-x86_64-Build20230919-1-ovs-server@64bit
2 jobs have been created:
 - sle-15-SP5-Server-DVD-Updates-x86_64-Build20230919-1-ovs-server@64bit -> https://openqa.suse.de/tests/12207602
 - sle-15-SP5-Server-DVD-Updates-x86_64-Build20230919-1-ovs-client@64bit -> https://openqa.suse.de/tests/12207601
{"id":1033279}
Cloning parents of sle-15-SP2-Server-DVD-Updates-x86_64-Build20230919-1-ovs-client@64bit
1 job has been created:
 - sle-15-SP2-Server-DVD-Updates-x86_64-Build20230919-1-ovs-client@64bit -> https://openqa.suse.de/tests/12207609
Actions
Copy link
 #4

Updated by okurz 10 months ago

Still found scheduled jobs blocking SLE maintenance releases, e.g. https://openqa.suse.de/tests/12228008

Actions
Copy link
 #5

Updated by okurz 10 months ago

I have now removed the worker class setting for worker31 in https://openqa.suse.de/admin/test_suites

For all currently stuck jobs from https://openqa.suse.de/tests?match=ovs-server I have executed

for i in 12238298 12237916 12237682 12237487; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.suse.de/tests/$i WORKER_CLASS=qemu_x86_64,tap; done

and labeled and cancelled the according old jobs

Actions
Copy link
 #6

Updated by dzedro 10 months ago

  • Status changed from New to Feedback
Actions
Copy link
 #7

Updated by dzedro 10 months ago

There were multiple issues due to which test failed, will see next days how does the test behave.

Actions
Copy link
 #8

Updated by dzedro 10 months ago

  • Assignee set to dzedro
Actions
Copy link
 #9

Updated by dzedro 9 months ago

  • Status changed from Feedback to Resolved
Actions
Copy link

Also available in: Atom PDF