Project

General

Profile

Download (164 KB)

action #165198 ยป 05-scheduler-full.t

tinita, 2024-08-13 15:15

 
# OPENQA_BASEDIR: /home/squamata/project/test-results/fullstack/scheduler
# OPENQA_CONFIG: /home/squamata/project/test-results/fullstack/scheduler/config
[INFO] using database schema "tmp_gPaEEnKKHPydK4z5"
# Used ports: {}
# Starting WebSocket service. Port: 9527
# Bogus: 0 | No wait: 1
# Starting WebUI service. Port: 9526
# PID of openqa-websocket: 1526
# Embedding scheduler within ws server subprocess
# PID of openqa-webapi: 1527
[info] [pid:1527] setting database search path to tmp_gPaEEnKKHPydK4z5 when registering Minion plugin
[info] Listening at "http://127.0.0.1:9527"
Web application available at http://127.0.0.1:9527
# using tests and needles from /home/squamata/os-autoinst/t/data/tests
ok 1 - results directory created under /home/squamata/project/test-results/fullstack/scheduler/openqa/testresults
# Subtest: Scheduler worker job allocation
# try to allocate to previous worker (supposed to fail)
ok 1 - no jobs allocated for no active workers
# starting two workers
# Starting standard worker. Instance: 1 for host http://localhost:9526
# Starting standard worker. Instance: 2 for host http://localhost:9526
# Waiting for worker with ID 3
[info] [pid:1531] worker 2:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 41
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/2
[error] [pid:1531] The average load 49.71 is exceeding the configured threshold of 40.
[info] [pid:1531] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1531] Registering with openQA http://localhost:9526
[info] [pid:1530] worker 1:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 41
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: qemu_i386,qemu_x86_64
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/1
[error] [pid:1530] The average load 49.71 is exceeding the configured threshold of 40.
[info] [pid:1530] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1530] Registering with openQA http://localhost:9526
[info] [pid:1531] Establishing ws connection via ws://localhost:9526/api/v1/ws/3
[info] [pid:1531] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 3
[warn] [pid:1531] The average load 49.71 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 290.21 s
[debug] [pid:1526] Updating seen of worker 3 from worker_status (broken)
[info] [pid:1530] Establishing ws connection via ws://localhost:9526/api/v1/ws/4
[info] [pid:1530] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 4
[warn] [pid:1530] The average load 49.71 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 130.63 s
[debug] [pid:1526] Updating seen of worker 4 from worker_status (broken)
# Waiting for worker with ID 4
# assigning one job to each worker
not ok 2 - jobs dispatched to different workers
not ok 3 - each of the two jobs allocated to one of the workers
# [
# {},
# {}
# ]
ok 4 - no more jobs need to be allocated
[info] [pid:1530] Received signal TERM
[debug] [pid:1530] Informing http://localhost:9526 that we are going offline
[info] [pid:1526] Worker 4 websocket connection closed - 1006
[info] [pid:1531] Received signal TERM
[debug] [pid:1531] Informing http://localhost:9526 that we are going offline
[info] [pid:1526] Worker 3 websocket connection closed - 1006
1..4
not ok 2 - Scheduler worker job allocation
# Subtest: re-scheduling and incompletion of jobs when worker rejects jobs or goes offline
[DEBUG] Duplicating jobs: {
99981 => {
chained_children => [],
chained_parents => [],
directly_chained_children => [],
directly_chained_parents => [],
is_parent_or_initial_job => 1,
ok => 0,
parallel_children => [],
parallel_parents => [],
state => "cancelled",
},
}
[DEBUG] Job 99981 duplicated as 99982
ok 1 - no jobs can be allocated to previous workers
# Starting broken worker. Instance: 3 for host http://localhost:9526
# Waiting for worker with ID 5
# PID of openqa-worker: 1546
[info] [pid:1546] worker 3:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/3
[error] [pid:1546] out of order
[info] [pid:1546] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1546] Registering with openQA http://localhost:9526
[info] [pid:1546] Establishing ws connection via ws://localhost:9526/api/v1/ws/5
[info] [pid:1546] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 5
[warn] [pid:1546] out of order - checking again for web UI 'http://localhost:9526' in 0.1 s
[debug] [pid:1526] Updating seen of worker 5 from worker_status (broken)
[warn] [pid:1546] out of order - checking again for web UI 'http://localhost:9526' in 0.1 s
[info] [pid:1526] Received worker 5 status too close to the last update, websocket server possibly overloaded or worker misconfigured
[debug] [pid:1526] Updating seen of worker 5 from worker_status (broken)
[warn] [pid:1546] out of order - checking again for web UI 'http://localhost:9526' in 0.1 s
[info] [pid:1526] Received worker 5 status too close to the last update, websocket server possibly overloaded or worker misconfigured
[debug] [pid:1526] Updating seen of worker 5 from worker_status (broken)
[warn] [pid:1546] out of order - checking again for web UI 'http://localhost:9526' in 0.1 s
[info] [pid:1526] Received worker 5 status too close to the last update, websocket server possibly overloaded or worker misconfigured
[debug] [pid:1526] Updating seen of worker 5 from worker_status (broken)
ok 2 - scheduler does not consider broken worker for allocating job
[info] [pid:1526] Worker 5 websocket connection closed - 1006
# Starting rejective worker. Instance: 3 for host http://localhost:9526
# Waiting for worker with ID 5
# PID of openqa-worker: 1548
[info] [pid:1548] worker 3:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/3
[error] [pid:1548] The average load 49.78 is exceeding the configured threshold of 40.
[info] [pid:1548] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1548] Registering with openQA http://localhost:9526
[info] [pid:1548] Establishing ws connection via ws://localhost:9526/api/v1/ws/5
[info] [pid:1548] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 5
[warn] [pid:1548] The average load 49.78 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 0.1 s
[debug] [pid:1526] Updating seen of worker 5 from worker_status (broken)
[debug] [pid:1548] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};
[warn] [pid:1548] The average load 49.78 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 0.1 s
[info] [pid:1526] Received worker 5 status too close to the last update, websocket server possibly overloaded or worker misconfigured
[debug] [pid:1526] Updating seen of worker 5 from worker_status (broken)
[debug] [pid:1548] Received ws message: $VAR1 = {
'seen' => 1,
'type' => 'info'
};
[warn] [pid:1548] The average load 49.78 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 0.1 s
[info] [pid:1526] Received worker 5 status too close to the last update, websocket server possibly overloaded or worker misconfigured
[debug] [pid:1526] Updating seen of worker 5 from worker_status (broken)
[debug] [pid:1548] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};
[warn] [pid:1548] The average load 49.78 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 0.1 s
[info] [pid:1526] Received worker 5 status too close to the last update, websocket server possibly overloaded or worker misconfigured
[debug] [pid:1526] Updating seen of worker 5 from worker_status (broken)
[debug] [pid:1548] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};
# waiting for job to be assigned and set back to re-scheduled
# scheduler could not yet assign to rejective worker, try: 1
# scheduler could not yet assign to rejective worker, try: 2
not ok 3 - one job allocated
ok 4 - assigned job set back to scheduled if worker reports back again but has abandoned the job
[info] [pid:1526] Worker 5 websocket connection closed - 1006
# Starting unstable worker. Instance: 3 for host http://localhost:9526
# PID of openqa-worker-unstable: 1550
# Waiting for worker with ID 5
[info] [pid:1550] worker 3:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/3
[error] [pid:1550] The average load 49.78 is exceeding the configured threshold of 40.
[info] [pid:1550] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1550] Registering with openQA http://localhost:9526
[info] [pid:1550] Establishing ws connection via ws://localhost:9526/api/v1/ws/5
[info] [pid:1550] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 5
[warn] [pid:1550] The average load 49.78 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 0.1 s
[debug] [pid:1526] Updating seen of worker 5 from worker_status (broken)
[warn] [pid:1550] The average load 49.78 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 0.1 s
[info] [pid:1526] Received worker 5 status too close to the last update, websocket server possibly overloaded or worker misconfigured
[debug] [pid:1526] Updating seen of worker 5 from worker_status (broken)
[warn] [pid:1550] The average load 49.78 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 0.1 s
[info] [pid:1526] Received worker 5 status too close to the last update, websocket server possibly overloaded or worker misconfigured
[debug] [pid:1526] Updating seen of worker 5 from worker_status (broken)
[warn] [pid:1550] The average load 49.78 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 0.1 s
[info] [pid:1526] Received worker 5 status too close to the last update, websocket server possibly overloaded or worker misconfigured
[debug] [pid:1526] Updating seen of worker 5 from worker_status (broken)
# scheduler could not yet assign to broken worker, try: 1
# scheduler could not yet assign to broken worker, try: 2
not ok 5 - one job allocated
[info] [pid:1526] Worker 5 websocket connection closed - 1006
# Starting unstable worker. Instance: 3 for host http://localhost:9526
# Waiting for worker with ID 5
# PID of openqa-worker-unstable: 1552
# Waiting for 'job 99982 is incompleted' to become available (timeout: 20)
[info] [pid:1552] worker 3:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/3
[error] [pid:1552] The average load 49.78 is exceeding the configured threshold of 40.
[info] [pid:1552] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1552] Registering with openQA http://localhost:9526
[debug] [pid:1527] Duplicating jobs: {
99982 => {
chained_children => [],
chained_parents => [],
directly_chained_children => [],
directly_chained_parents => [],
is_parent_or_initial_job => 1,
ok => 0,
parallel_children => [],
parallel_parents => [],
state => "running",
},
}
[debug] [pid:1527] Job 99982 duplicated as 99983
[debug] [pid:1527] _carry_over_candidate(99982): _failure_reason=GOOD
[info] [pid:1552] Establishing ws connection via ws://localhost:9526/api/v1/ws/5
[info] [pid:1552] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 5
[warn] [pid:1552] The average load 49.78 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 0.1 s
[debug] [pid:1526] Updating seen of worker 5 from worker_status (broken)
ok 6 - running job set to done if its worker re-connects claiming not to work on it anymore
ok 7 - running job incompleted if its worker re-connects claiming not to work on it anymore
ok 8 - reason is set
[info] [pid:1526] Worker 5 websocket connection closed - 1006
1..8
not ok 3 - re-scheduling and incompletion of jobs when worker rejects jobs or goes offline
# Subtest: Simulation of heavy unstable load
[DEBUG] Duplicating jobs: {
99963 => {
chained_children => [],
chained_parents => [],
directly_chained_children => [],
directly_chained_parents => [],
is_parent_or_initial_job => 1,
ok => 0,
parallel_children => [],
parallel_parents => [],
state => "running",
},
}
[DEBUG] Job 99963 duplicated as 99984
[DEBUG] Duplicating jobs: {
99961 => {
chained_children => [],
chained_parents => [],
directly_chained_children => [],
directly_chained_parents => [],
is_parent_or_initial_job => 1,
ok => 0,
parallel_children => [],
parallel_parents => [],
state => "running",
},
}
[DEBUG] Job 99961 duplicated as 99985
[DEBUG] Duplicating jobs: {
99947 => {
chained_children => [],
chained_parents => [],
directly_chained_children => [],
directly_chained_parents => [],
is_parent_or_initial_job => 1,
ok => 1,
parallel_children => [],
parallel_parents => [],
state => "done",
},
}
[DEBUG] Job 99947 duplicated as 99986
[DEBUG] Duplicating jobs: {
99946 => {
chained_children => [],
chained_parents => [],
directly_chained_children => [],
directly_chained_parents => [],
is_parent_or_initial_job => 1,
ok => 1,
parallel_children => [],
parallel_parents => [],
state => "done",
},
}
[DEBUG] Job 99946 duplicated as 99987
[DEBUG] Duplicating jobs: {
99940 => {
chained_children => [],
chained_parents => [],
directly_chained_children => [],
directly_chained_parents => [],
is_parent_or_initial_job => 1,
ok => 0,
parallel_children => [],
parallel_parents => [],
state => "done",
},
}
[DEBUG] Job 99940 duplicated as 99988
[DEBUG] Duplicating jobs: {
99939 => {
chained_children => [],
chained_parents => [],
directly_chained_children => [],
directly_chained_parents => [],
is_parent_or_initial_job => 1,
ok => 1,
parallel_children => [],
parallel_parents => [],
state => "done",
},
}
[DEBUG] Job 99939 duplicated as 99989
[DEBUG] Duplicating jobs: {
99938 => {
chained_children => [],
chained_parents => [],
directly_chained_children => [],
directly_chained_parents => [],
is_parent_or_initial_job => 1,
ok => 0,
parallel_children => [],
parallel_parents => [],
state => "done",
},
}
[DEBUG] Job 99938 duplicated as 99990
[DEBUG] Duplicating jobs: {
99937 => {
chained_children => [],
chained_parents => [],
directly_chained_children => [],
directly_chained_parents => [],
is_parent_or_initial_job => 1,
ok => 1,
parallel_children => [],
parallel_parents => [],
state => "done",
},
}
[DEBUG] Job 99937 duplicated as 99991
[DEBUG] Duplicating jobs: {
99936 => {
chained_children => [],
chained_parents => [],
directly_chained_children => [],
directly_chained_parents => [],
is_parent_or_initial_job => 1,
ok => 1,
parallel_children => [],
parallel_parents => [],
state => "done",
},
}
[DEBUG] Job 99936 duplicated as 99992
[DEBUG] Duplicating jobs: {
99926 => {
chained_children => [],
chained_parents => [],
directly_chained_children => [],
directly_chained_parents => [],
is_parent_or_initial_job => 1,
ok => 0,
parallel_children => [],
parallel_parents => [],
state => "done",
},
}
[DEBUG] Job 99926 duplicated as 99993
[DEBUG] Duplicating jobs: {
99764 => {
chained_children => [],
chained_parents => [],
directly_chained_children => [],
directly_chained_parents => [],
is_parent_or_initial_job => 1,
ok => 1,
parallel_children => [],
parallel_parents => [],
state => "done",
},
}
[DEBUG] Job 99764 duplicated as 99994
[DEBUG] Duplicating jobs: {
80000 => {
chained_children => [],
chained_parents => [],
directly_chained_children => [],
directly_chained_parents => [],
is_parent_or_initial_job => 1,
ok => 1,
parallel_children => [],
parallel_parents => [],
state => "done",
},
}
[DEBUG] Job 80000 duplicated as 99995
# Starting unresponsive worker. Instance: 1 for host http://localhost:9526
# Starting unresponsive worker. Instance: 2 for host http://localhost:9526
# PID of openqa-worker: 1555
# Starting unresponsive worker. Instance: 3 for host http://localhost:9526
# PID of openqa-worker: 1556
# PID of openqa-worker: 1557
# Starting unresponsive worker. Instance: 4 for host http://localhost:9526
[info] [pid:1555] worker 1:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: qemu_i386,qemu_x86_64
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/1
[error] [pid:1555] The average load 49.78 is exceeding the configured threshold of 40.
[info] [pid:1555] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1555] Registering with openQA http://localhost:9526
[info] [pid:1556] worker 2:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/2
[error] [pid:1556] The average load 49.78 is exceeding the configured threshold of 40.
[info] [pid:1556] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1556] Registering with openQA http://localhost:9526
# PID of openqa-worker: 1559
# Starting unresponsive worker. Instance: 5 for host http://localhost:9526
[info] [pid:1557] worker 3:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/3
[error] [pid:1557] The average load 49.78 is exceeding the configured threshold of 40.
[info] [pid:1557] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1557] Registering with openQA http://localhost:9526
# PID of openqa-worker: 1561
# Starting unresponsive worker. Instance: 6 for host http://localhost:9526
[info] [pid:1559] worker 4:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/4
[error] [pid:1559] The average load 49.78 is exceeding the configured threshold of 40.
[info] [pid:1559] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1559] Registering with openQA http://localhost:9526
# Starting unresponsive worker. Instance: 7 for host http://localhost:9526
# PID of openqa-worker: 1563
[info] [pid:1561] worker 5:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/5
[error] [pid:1561] The average load 49.78 is exceeding the configured threshold of 40.
[info] [pid:1561] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1561] Registering with openQA http://localhost:9526
# PID of openqa-worker: 1565
# Starting unresponsive worker. Instance: 8 for host http://localhost:9526
# Starting unresponsive worker. Instance: 9 for host http://localhost:9526
# PID of openqa-worker: 1567
[info] [pid:1563] worker 6:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/6
[error] [pid:1563] The average load 49.78 is exceeding the configured threshold of 40.
[info] [pid:1563] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1563] Registering with openQA http://localhost:9526
[info] [pid:1565] worker 7:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/7
# PID of openqa-worker: 1568
# Starting unresponsive worker. Instance: 10 for host http://localhost:9526
[error] [pid:1565] The average load 49.78 is exceeding the configured threshold of 40.
[info] [pid:1565] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1565] Registering with openQA http://localhost:9526
[info] [pid:1567] worker 8:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/8
# PID of openqa-worker: 1571
# Starting unresponsive worker. Instance: 11 for host http://localhost:9526
[error] [pid:1567] The average load 49.78 is exceeding the configured threshold of 40.
[info] [pid:1567] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1567] Registering with openQA http://localhost:9526
[info] [pid:1568] worker 9:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/9
[error] [pid:1568] The average load 49.78 is exceeding the configured threshold of 40.
[info] [pid:1568] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1568] Registering with openQA http://localhost:9526
[info] [pid:1571] worker 10:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/10
[error] [pid:1571] The average load 49.78 is exceeding the configured threshold of 40.
[info] [pid:1571] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
# PID of openqa-worker: 1573
[info] [pid:1571] Registering with openQA http://localhost:9526
# Starting unresponsive worker. Instance: 12 for host http://localhost:9526
# Starting unresponsive worker. Instance: 13 for host http://localhost:9526
# PID of openqa-worker: 1576
# Starting unresponsive worker. Instance: 14 for host http://localhost:9526
# PID of openqa-worker: 1577
[info] [pid:1573] worker 11:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/11
[error] [pid:1573] The average load 49.78 is exceeding the configured threshold of 40.
[info] [pid:1573] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1573] Registering with openQA http://localhost:9526
[info] [pid:1577] worker 13:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/13
[info] [pid:1576] worker 12:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/12
[error] [pid:1577] The average load 49.78 is exceeding the configured threshold of 40.
[error] [pid:1576] The average load 49.78 is exceeding the configured threshold of 40.
[info] [pid:1577] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1576] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1577] Registering with openQA http://localhost:9526
[info] [pid:1576] Registering with openQA http://localhost:9526
# Starting unresponsive worker. Instance: 15 for host http://localhost:9526
# PID of openqa-worker: 1578
# Starting unresponsive worker. Instance: 16 for host http://localhost:9526
# PID of openqa-worker: 1582
# Starting unresponsive worker. Instance: 17 for host http://localhost:9526
# PID of openqa-worker: 1583
[info] [pid:1578] worker 14:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/14
[error] [pid:1578] The average load 49.78 is exceeding the configured threshold of 40.
[info] [pid:1578] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1556] Establishing ws connection via ws://localhost:9526/api/v1/ws/3
[info] [pid:1578] Registering with openQA http://localhost:9526
# PID of openqa-worker: 1584
# Starting unresponsive worker. Instance: 18 for host http://localhost:9526
[info] [pid:1583] worker 16:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/16
[error] [pid:1583] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1583] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1583] Registering with openQA http://localhost:9526
[info] [pid:1582] worker 15:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/15
[error] [pid:1582] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1582] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1582] Registering with openQA http://localhost:9526
# PID of openqa-worker: 1586
# Starting unresponsive worker. Instance: 19 for host http://localhost:9526
[info] [pid:1584] worker 17:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/17
# Starting unresponsive worker. Instance: 20 for host http://localhost:9526
[error] [pid:1584] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1584] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1584] Registering with openQA http://localhost:9526
# PID of openqa-worker: 1589
[info] [pid:1586] worker 18:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/18
[error] [pid:1586] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1586] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1586] Registering with openQA http://localhost:9526
[info] [pid:1589] worker 19:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/19
[error] [pid:1589] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1589] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1589] Registering with openQA http://localhost:9526
# PID of openqa-worker: 1592
# Starting unresponsive worker. Instance: 21 for host http://localhost:9526
[info] [pid:1592] worker 20:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/20
[error] [pid:1592] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1592] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1592] Registering with openQA http://localhost:9526
# Starting unresponsive worker. Instance: 22 for host http://localhost:9526
# PID of openqa-worker: 1594
# PID of openqa-worker: 1596
# Starting unresponsive worker. Instance: 23 for host http://localhost:9526
[info] [pid:1594] worker 21:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/21
[error] [pid:1594] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1594] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1594] Registering with openQA http://localhost:9526
# PID of openqa-worker: 1597
# Starting unresponsive worker. Instance: 24 for host http://localhost:9526
[info] [pid:1568] Establishing ws connection via ws://localhost:9526/api/v1/ws/6
[info] [pid:1596] worker 22:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/22
[error] [pid:1596] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1596] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
# Starting unresponsive worker. Instance: 25 for host http://localhost:9526
# PID of openqa-worker: 1599
[info] [pid:1596] Registering with openQA http://localhost:9526
[info] [pid:1597] worker 23:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/23
# PID of openqa-worker: 1600
# Starting unresponsive worker. Instance: 26 for host http://localhost:9526
[error] [pid:1597] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1597] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1597] Registering with openQA http://localhost:9526
[info] [pid:1599] worker 24:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/24
# PID of openqa-worker: 1602
# Starting unresponsive worker. Instance: 27 for host http://localhost:9526
[error] [pid:1599] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1599] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1599] Registering with openQA http://localhost:9526
[info] [pid:1600] worker 25:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/25
# Starting unresponsive worker. Instance: 28 for host http://localhost:9526
# PID of openqa-worker: 1604
[error] [pid:1600] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1600] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1600] Registering with openQA http://localhost:9526
# PID of openqa-worker: 1606
# Starting unresponsive worker. Instance: 29 for host http://localhost:9526
[info] [pid:1602] worker 26:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/26
[error] [pid:1602] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1602] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1604] worker 27:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/27
[info] [pid:1602] Registering with openQA http://localhost:9526
[error] [pid:1604] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1604] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1604] Registering with openQA http://localhost:9526
# Starting unresponsive worker. Instance: 30 for host http://localhost:9526
# PID of openqa-worker: 1607
# Starting unresponsive worker. Instance: 31 for host http://localhost:9526
# PID of openqa-worker: 1611
[info] [pid:1606] worker 28:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/28
[error] [pid:1606] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1606] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1606] Registering with openQA http://localhost:9526
# PID of openqa-worker: 1612
# Starting unresponsive worker. Instance: 32 for host http://localhost:9526
[info] [pid:1607] worker 29:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/29
[error] [pid:1607] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1607] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
# PID of openqa-worker: 1614
# Starting unresponsive worker. Instance: 33 for host http://localhost:9526
[info] [pid:1611] worker 30:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/30
[info] [pid:1607] Registering with openQA http://localhost:9526
[error] [pid:1611] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1611] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1611] Registering with openQA http://localhost:9526
# PID of openqa-worker: 1615
# Starting unresponsive worker. Instance: 34 for host http://localhost:9526
[info] [pid:1612] worker 31:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/31
[error] [pid:1612] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1612] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1612] Registering with openQA http://localhost:9526
# Starting unresponsive worker. Instance: 35 for host http://localhost:9526
# PID of openqa-worker: 1618
[info] [pid:1615] worker 33:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/33
# Starting unresponsive worker. Instance: 36 for host http://localhost:9526
# PID of openqa-worker: 1619
[info] [pid:1614] worker 32:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/32
[error] [pid:1615] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1615] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1615] Registering with openQA http://localhost:9526
[error] [pid:1614] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1614] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1614] Registering with openQA http://localhost:9526
# PID of openqa-worker: 1621
# Starting unresponsive worker. Instance: 37 for host http://localhost:9526
[info] [pid:1619] worker 35:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/35
[info] [pid:1618] worker 34:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/34
[error] [pid:1619] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1565] Establishing ws connection via ws://localhost:9526/api/v1/ws/7
[info] [pid:1619] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[error] [pid:1618] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1619] Registering with openQA http://localhost:9526
[info] [pid:1618] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1618] Registering with openQA http://localhost:9526
# PID of openqa-worker: 1624
# Starting unresponsive worker. Instance: 38 for host http://localhost:9526
# PID of openqa-worker: 1627
# Starting unresponsive worker. Instance: 39 for host http://localhost:9526
# PID of openqa-worker: 1628
# Starting unresponsive worker. Instance: 40 for host http://localhost:9526
[info] [pid:1624] worker 37:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/37
[error] [pid:1624] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1624] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1624] Registering with openQA http://localhost:9526
[info] [pid:1621] worker 36:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/36
[error] [pid:1621] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1621] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1627] worker 38:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/38
[info] [pid:1621] Registering with openQA http://localhost:9526
[error] [pid:1627] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1627] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1627] Registering with openQA http://localhost:9526
[info] [pid:1628] worker 39:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/39
# Starting unresponsive worker. Instance: 41 for host http://localhost:9526
# PID of openqa-worker: 1630
[error] [pid:1628] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1628] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1628] Registering with openQA http://localhost:9526
# Starting unresponsive worker. Instance: 42 for host http://localhost:9526
# PID of openqa-worker: 1633
# Starting unresponsive worker. Instance: 43 for host http://localhost:9526
# PID of openqa-worker: 1635
# PID of openqa-worker: 1636
# Starting unresponsive worker. Instance: 44 for host http://localhost:9526
[info] [pid:1630] worker 40:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/40
[error] [pid:1630] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1630] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1630] Registering with openQA http://localhost:9526
[info] [pid:1633] worker 41:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/41
[error] [pid:1633] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1633] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1633] Registering with openQA http://localhost:9526
# Starting unresponsive worker. Instance: 45 for host http://localhost:9526
# PID of openqa-worker: 1637
[info] [pid:1635] worker 42:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/42
[error] [pid:1635] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1636] worker 43:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/43
[info] [pid:1635] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1635] Registering with openQA http://localhost:9526
[error] [pid:1636] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1636] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1636] Registering with openQA http://localhost:9526
# PID of openqa-worker: 1641
[info] [pid:1637] worker 44:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/44
[error] [pid:1637] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1637] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1637] Registering with openQA http://localhost:9526
# Starting unresponsive worker. Instance: 46 for host http://localhost:9526
# PID of openqa-worker: 1644
# Starting unresponsive worker. Instance: 47 for host http://localhost:9526
[info] [pid:1641] worker 45:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/45
[error] [pid:1641] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1641] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1571] Establishing ws connection via ws://localhost:9526/api/v1/ws/8
[info] [pid:1641] Registering with openQA http://localhost:9526
# Starting unresponsive worker. Instance: 48 for host http://localhost:9526
# PID of openqa-worker: 1645
[info] [pid:1644] worker 46:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/46
# PID of openqa-worker: 1647
[error] [pid:1644] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1644] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
# Starting unresponsive worker. Instance: 49 for host http://localhost:9526
[info] [pid:1644] Registering with openQA http://localhost:9526
# PID of openqa-worker: 1649
# Starting unresponsive worker. Instance: 50 for host http://localhost:9526
# PID of openqa-worker: 1650
# Waiting for worker with ID 3
[info] [pid:1647] worker 48:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/48
[error] [pid:1647] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1647] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1647] Registering with openQA http://localhost:9526
# Waiting for worker with ID 4
[info] [pid:1645] worker 47:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/47
[info] [pid:1649] worker 49:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/49
[error] [pid:1649] The average load 49.81 is exceeding the configured threshold of 40.
[error] [pid:1645] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1645] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1649] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1649] Registering with openQA http://localhost:9526
[info] [pid:1645] Registering with openQA http://localhost:9526
[info] [pid:1650] worker 50:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/50
[error] [pid:1650] The average load 49.81 is exceeding the configured threshold of 40.
[info] [pid:1650] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1650] Registering with openQA http://localhost:9526
[info] [pid:1555] Establishing ws connection via ws://localhost:9526/api/v1/ws/4
[info] [pid:1559] Establishing ws connection via ws://localhost:9526/api/v1/ws/9
[info] [pid:1561] Establishing ws connection via ws://localhost:9526/api/v1/ws/10
[info] [pid:1557] Establishing ws connection via ws://localhost:9526/api/v1/ws/5
[info] [pid:1563] Establishing ws connection via ws://localhost:9526/api/v1/ws/11
[info] [pid:1567] Establishing ws connection via ws://localhost:9526/api/v1/ws/12
[info] [pid:1571] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 8
[warn] [pid:1571] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[info] [pid:1563] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 11
[warn] [pid:1563] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[debug] [pid:1526] Updating seen of worker 8 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 11 from worker_status (broken)
[debug] [pid:1571] Received ws message: $VAR1 = {
'seen' => 1,
'type' => 'info'
};
[debug] [pid:1563] Received ws message: $VAR1 = {
'seen' => 1,
'type' => 'info'
};
[info] [pid:1644] Establishing ws connection via ws://localhost:9526/api/v1/ws/13
# Waiting for worker with ID 5
# Waiting for worker with ID 6
# Waiting for worker with ID 7
# Waiting for worker with ID 8
# Waiting for worker with ID 9
# Waiting for worker with ID 10
# Waiting for worker with ID 11
# Waiting for worker with ID 12
# Waiting for worker with ID 13
# Waiting for worker with ID 14
# Waiting for worker with ID 15
[info] [pid:1624] Establishing ws connection via ws://localhost:9526/api/v1/ws/14
[info] [pid:1635] Establishing ws connection via ws://localhost:9526/api/v1/ws/15
[info] [pid:1606] Establishing ws connection via ws://localhost:9526/api/v1/ws/16
[info] [pid:1589] Establishing ws connection via ws://localhost:9526/api/v1/ws/17
[info] [pid:1619] Establishing ws connection via ws://localhost:9526/api/v1/ws/18
[info] [pid:1576] Establishing ws connection via ws://localhost:9526/api/v1/ws/19
[info] [pid:1630] Establishing ws connection via ws://localhost:9526/api/v1/ws/20
[info] [pid:1637] Establishing ws connection via ws://localhost:9526/api/v1/ws/21
[info] [pid:1567] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 12
[warn] [pid:1567] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[info] [pid:1559] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 9
[warn] [pid:1559] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[debug] [pid:1526] Updating seen of worker 12 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 9 from worker_status (broken)
[debug] [pid:1567] Received ws message: $VAR1 = {
'seen' => 1,
'type' => 'info'
};
[debug] [pid:1559] Received ws message: $VAR1 = {
'seen' => 1,
'type' => 'info'
};
[info] [pid:1641] Establishing ws connection via ws://localhost:9526/api/v1/ws/22
# Waiting for worker with ID 16
# Waiting for worker with ID 17
# Waiting for worker with ID 18
# Waiting for worker with ID 19
# Waiting for worker with ID 20
# Waiting for worker with ID 21
# Waiting for worker with ID 22
# Waiting for worker with ID 23
# Waiting for worker with ID 24
[info] [pid:1647] Establishing ws connection via ws://localhost:9526/api/v1/ws/23
[info] [pid:1583] Establishing ws connection via ws://localhost:9526/api/v1/ws/24
[info] [pid:1614] Establishing ws connection via ws://localhost:9526/api/v1/ws/25
[info] [pid:1612] Establishing ws connection via ws://localhost:9526/api/v1/ws/26
[info] [pid:1586] Establishing ws connection via ws://localhost:9526/api/v1/ws/27
[info] [pid:1636] Establishing ws connection via ws://localhost:9526/api/v1/ws/28
[info] [pid:1627] Establishing ws connection via ws://localhost:9526/api/v1/ws/29
# Waiting for worker with ID 25
# Waiting for worker with ID 26
# Waiting for worker with ID 27
# Waiting for worker with ID 28
# Waiting for worker with ID 29
# Waiting for worker with ID 30
# Waiting for worker with ID 31
[info] [pid:1650] Establishing ws connection via ws://localhost:9526/api/v1/ws/30
[info] [pid:1600] Establishing ws connection via ws://localhost:9526/api/v1/ws/31
[info] [pid:1555] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 4
[warn] [pid:1555] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[debug] [pid:1526] Updating seen of worker 4 from worker_status (broken)
[debug] [pid:1555] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};
[info] [pid:1596] Establishing ws connection via ws://localhost:9526/api/v1/ws/32
[info] [pid:1633] Establishing ws connection via ws://localhost:9526/api/v1/ws/33
[info] [pid:1592] Establishing ws connection via ws://localhost:9526/api/v1/ws/34
[info] [pid:1628] Establishing ws connection via ws://localhost:9526/api/v1/ws/35
[info] [pid:1568] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 6
[warn] [pid:1568] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[debug] [pid:1526] Updating seen of worker 6 from worker_status (broken)
[debug] [pid:1568] Received ws message: $VAR1 = {
'seen' => 1,
'type' => 'info'
};
[info] [pid:1584] Establishing ws connection via ws://localhost:9526/api/v1/ws/36
[info] [pid:1561] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 10
[warn] [pid:1561] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[debug] [pid:1526] Updating seen of worker 10 from worker_status (broken)
[debug] [pid:1561] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};
[info] [pid:1599] Establishing ws connection via ws://localhost:9526/api/v1/ws/37
[info] [pid:1557] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 5
[warn] [pid:1557] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[debug] [pid:1526] Updating seen of worker 5 from worker_status (broken)
[debug] [pid:1557] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};
# Waiting for worker with ID 32
[info] [pid:1594] Establishing ws connection via ws://localhost:9526/api/v1/ws/38
# Waiting for worker with ID 33
# Waiting for worker with ID 34
# Waiting for worker with ID 35
# Waiting for worker with ID 36
# Waiting for worker with ID 37
# Waiting for worker with ID 38
# Waiting for worker with ID 39
# Waiting for worker with ID 40
[info] [pid:1582] Establishing ws connection via ws://localhost:9526/api/v1/ws/39
[info] [pid:1602] Establishing ws connection via ws://localhost:9526/api/v1/ws/40
[info] [pid:1645] Establishing ws connection via ws://localhost:9526/api/v1/ws/41
[info] [pid:1615] Establishing ws connection via ws://localhost:9526/api/v1/ws/42
[info] [pid:1578] Establishing ws connection via ws://localhost:9526/api/v1/ws/43
[info] [pid:1597] Establishing ws connection via ws://localhost:9526/api/v1/ws/44
[info] [pid:1573] Establishing ws connection via ws://localhost:9526/api/v1/ws/45
[info] [pid:1556] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 3
[warn] [pid:1556] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[debug] [pid:1526] Updating seen of worker 3 from worker_status (broken)
[debug] [pid:1556] Received ws message: $VAR1 = {
'seen' => 1,
'type' => 'info'
};
# Waiting for worker with ID 41
# Waiting for worker with ID 42
# Waiting for worker with ID 43
# Waiting for worker with ID 44
[info] [pid:1577] Establishing ws connection via ws://localhost:9526/api/v1/ws/46
# Waiting for worker with ID 45
# Waiting for worker with ID 46
# Waiting for worker with ID 47
[info] [pid:1607] Establishing ws connection via ws://localhost:9526/api/v1/ws/47
[info] [pid:1604] Establishing ws connection via ws://localhost:9526/api/v1/ws/48
[info] [pid:1565] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 7
[warn] [pid:1565] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[debug] [pid:1526] Updating seen of worker 7 from worker_status (broken)
[debug] [pid:1565] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};
[info] [pid:1621] Establishing ws connection via ws://localhost:9526/api/v1/ws/49
[info] [pid:1649] Establishing ws connection via ws://localhost:9526/api/v1/ws/50
[info] [pid:1611] Establishing ws connection via ws://localhost:9526/api/v1/ws/51
[info] [pid:1618] Establishing ws connection via ws://localhost:9526/api/v1/ws/52
[info] [pid:1599] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 37
[warn] [pid:1599] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[info] [pid:1592] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 34
[warn] [pid:1592] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[debug] [pid:1526] Updating seen of worker 37 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 34 from worker_status (broken)
[info] [pid:1577] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 46
[warn] [pid:1577] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[info] [pid:1627] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 29
[warn] [pid:1627] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[debug] [pid:1599] Received ws message: $VAR1 = {
'seen' => 1,
'type' => 'info'
};
[info] [pid:1573] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 45
[warn] [pid:1573] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[info] [pid:1594] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 38
[warn] [pid:1594] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[info] [pid:1584] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 36
[warn] [pid:1584] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[info] [pid:1606] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 16
[warn] [pid:1606] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[info] [pid:1614] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 25
[warn] [pid:1614] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[info] [pid:1576] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 19
[warn] [pid:1576] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[info] [pid:1578] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 43
[warn] [pid:1578] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
# Waiting for worker with ID 48
# Waiting for worker with ID 49
# Waiting for worker with ID 50
[info] [pid:1596] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 32
[warn] [pid:1596] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
# Waiting for worker with ID 51
[info] [pid:1618] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 52
[warn] [pid:1618] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
# Waiting for worker with ID 52
[debug] [pid:1592] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};
[info] [pid:1604] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 48
[warn] [pid:1604] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[debug] [pid:1526] Updating seen of worker 46 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 29 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 45 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 38 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 36 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 16 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 25 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 19 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 43 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 32 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 52 from worker_status (broken)
[info] [pid:1582] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 39
[warn] [pid:1582] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[debug] [pid:1614] Received ws message: $VAR1 = {
'seen' => 1,
'type' => 'info'
};
[info] [pid:1637] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 21
[warn] [pid:1637] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[info] [pid:1621] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 49
[warn] [pid:1621] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[info] [pid:1649] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 50
[warn] [pid:1649] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[DEBUG] Scheduling: Free workers: 38/52; Scheduled jobs: 15
[info] [pid:1602] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 40
[warn] [pid:1602] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[debug] [pid:1584] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};
[info] [pid:1586] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 27
[DEBUG] Need to schedule 1 parallel jobs for job 99984 (with priority 35)
[warn] [pid:1586] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[DEBUG] Need to schedule 1 parallel jobs for job 99985 (with priority 35)
[DEBUG] Need to schedule 1 parallel jobs for job 99986 (with priority 35)
[DEBUG] Need to schedule 1 parallel jobs for job 99987 (with priority 35)
[DEBUG] Need to schedule 1 parallel jobs for job 99991 (with priority 35)
[DEBUG] Need to schedule 1 parallel jobs for job 99994 (with priority 35)
[DEBUG] Need to schedule 1 parallel jobs for job 99988 (with priority 36)
[DEBUG] Need to schedule 1 parallel jobs for job 99989 (with priority 36)
[DEBUG] Need to schedule 1 parallel jobs for job 99990 (with priority 36)
[DEBUG] Need to schedule 1 parallel jobs for job 99992 (with priority 36)
[DEBUG] limit reached, scheduling no additional jobs (max_running_jobs=-1, free workers=38, running=2, allocated=10)
[debug] [pid:1627] Received ws message: $VAR1 = {
'seen' => 1,
'type' => 'info'
};
[debug] [pid:1577] Received ws message: $VAR1 = {
'seen' => 1,
'type' => 'info'
};
[info] [pid:1647] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 23
[warn] [pid:1647] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[debug] [pid:1618] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};
[info] [pid:1619] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 18
[warn] [pid:1619] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[info] [pid:1633] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 33
[warn] [pid:1633] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[info] [pid:1600] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 31
[warn] [pid:1600] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[DEBUG] Assigned job '99984' to worker ID '35'
[info] [pid:1624] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 14
[warn] [pid:1624] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[DEBUG] [Job#99984] Prepare for being processed by worker 35
[info] [pid:1650] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 30
[warn] [pid:1650] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[debug] [pid:1606] Received ws message: $VAR1 = {
'seen' => 1,
'type' => 'info'
};

[debug] [pid:1576] Received ws message: $VAR1 = {
'seen' => 1,
'type' => 'info'
};
[info] [pid:1583] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 24
[warn] [pid:1583] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[info] [pid:1597] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 44
[warn] [pid:1597] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[info] [pid:1641] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 22
[warn] [pid:1641] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[info] [pid:1636] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 28
[warn] [pid:1636] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[info] [pid:1615] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 42
[warn] [pid:1615] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[debug] [pid:1594] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};
[info] [pid:1611] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 51
[warn] [pid:1611] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[info] [pid:1628] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 35
[warn] [pid:1628] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[debug] [pid:1573] Received ws message: $VAR1 = {
'seen' => 1,
'type' => 'info'
};
[info] [pid:1607] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 47
[warn] [pid:1607] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[info] [pid:1635] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 15
[warn] [pid:1635] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[debug] [pid:1526] Updating seen of worker 48 from worker_status (broken)
[debug] [pid:1596] Received ws message: $VAR1 = {
'seen' => 1,
'type' => 'info'
};
[info] [pid:1645] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 41
[warn] [pid:1645] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[info] [pid:1630] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 20
[warn] [pid:1630] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[info] [pid:1589] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 17
[warn] [pid:1589] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[info] [pid:1644] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 13
[warn] [pid:1644] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[debug] [pid:1578] Received ws message: $VAR1 = {
'seen' => 1,
'type' => 'info'
};
[info] [pid:1612] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 26
[warn] [pid:1612] The average load 49.81 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 181.57 s
[debug] [pid:1526] Updating seen of worker 39 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 21 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 49 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 50 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 40 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 27 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 23 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 18 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 33 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 31 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 14 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 30 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 24 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 44 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 22 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 28 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 42 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 51 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 35 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 47 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 15 from worker_status (broken)
[debug] [pid:1604] Received ws message: $VAR1 = {
'seen' => 1,
'type' => 'info'
};
[debug] [pid:1526] Updating seen of worker 41 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 20 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 17 from worker_status (broken)
[debug] [pid:1526] Updating seen of worker 13 from worker_status (broken)
[debug] [pid:1645] Received ws message: $VAR1 = {
'seen' => 1,
'type' => 'info'
};
[debug] [pid:1630] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};

[debug] [pid:1589] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};

[debug] [pid:1644] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};
[debug] [pid:1526] Updating seen of worker 26 from worker_status (broken)
[debug] [pid:1526] Started to send message to 35 for job(s) 99984
[debug] [pid:1607] Received ws message: $VAR1 = {
'seen' => 1,
'type' => 'info'
};
[debug] [pid:1635] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};

[debug] [pid:1641] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};

[debug] [pid:1597] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};

[debug] [pid:1636] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};

[debug] [pid:1611] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};

[debug] [pid:1615] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};
[debug] [pid:1628] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};

[DEBUG] Sent job(s) '99984' to worker '35'
[debug] [pid:1650] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};

[debug] [pid:1583] Received ws message: $VAR1 = {
'seen' => 1,
'type' => 'info'
};

[debug] [pid:1633] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};

[debug] [pid:1600] Received ws message: $VAR1 = {
'seen' => 1,
'type' => 'info'
};

[debug] [pid:1624] Received ws message: $VAR1 = {
'seen' => 1,
'type' => 'info'
};

[debug] [pid:1628] Received ws message: $VAR1 = {
'type' => 'grab_job',
'job' => {
'test' => 'kde',
'assets' => {
'iso' => [
'openSUSE-13.1-DVD-x86_64-Build0091-Media.iso'
]
},
'settings' => {
'DESKTOP' => 'kde',
'FLAVOR' => 'DVD',
'ISO' => 'openSUSE-13.1-DVD-x86_64-Build0091-Media.iso',
'DISTRI' => 'opensuse',
'TEST' => 'kde',
'ARCH' => 'x86_64',
'DVD' => '1',
'NAME' => '00099984-opensuse-13.1-DVD-x86_64-Build0091-kde@64bit',
'VERSION' => '13.1',
'ISO_MAXSIZE' => '4700372992',
'MACHINE' => '64bit',
'BUILD' => '0091',
'JOBTOKEN' => 'Z_6pUV4GtRv5V5Hl'
},
'assigned_worker_id' => 35,
'origin_id' => 99963,
't_finished' => undef,
'priority' => 35,
'clone_id' => undef,
'name' => 'opensuse-13.1-DVD-x86_64-Build0091-kde@64bit',
'state' => 'assigned',
't_started' => undef,
'result' => 'none',
'id' => 99984,
'blocked_by_id' => undef,
'group' => 'opensuse',
'group_id' => 1001
}
};

[debug] [pid:1647] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};

[debug] [pid:1619] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};

[debug] [pid:1582] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};

[debug] [pid:1621] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};

[debug] [pid:1637] Received ws message: $VAR1 = {
'type' => 'info',
'seen' => 1
};

[debug] [pid:1649] Received ws message: $VAR1 = {
'seen' => 1,
'type' => 'info'
};

[debug] [pid:1602] Received ws message: $VAR1 = {
'seen' => 1,
'type' => 'info'
};

[debug] [pid:1586] Received ws message: $VAR1 = {
'seen' => 1,
'type' => 'info'
};
[debug] [pid:1612] Received ws message: $VAR1 = {
'seen' => 1,
'type' => 'info'
};
[DEBUG] Assigned job '99990' to worker ID '44'
[DEBUG] [Job#99990] Prepare for being processed by worker 44
[debug] [pid:1526] Started to send message to 44 for job(s) 99990
[debug] [pid:1597] Received ws message: $VAR1 = {
'job' => {
'priority' => 36,
't_finished' => undef,
'origin_id' => 99938,
'assigned_worker_id' => 44,
'settings' => {
'DESKTOP' => 'kde',
'FOO' => 'foo/foo.txt',
'QEMUCPU' => 'qemu64',
'FLAVOR' => 'DVD',
'ISO' => 'openSUSE-Factory-DVD-x86_64-Build0048-Media.iso',
'DISTRI' => 'opensuse',
'TEST' => 'doc',
'ARCH' => 'x86_64',
'DVD' => '1',
'NAME' => '00099990-opensuse-Factory-DVD-x86_64-Build0048-doc@64bit',
'VERSION' => 'Factory',
'ISO_MAXSIZE' => '4700372992',
'MACHINE' => '64bit',
'BUILD' => '0048',
'JOBTOKEN' => 'cjwEdQscom0alorS'
},
'test' => 'doc',
'assets' => {
'iso' => [
'openSUSE-Factory-DVD-x86_64-Build0048-Media.iso'
]
},
'group' => 'opensuse',
'group_id' => 1001,
't_started' => undef,
'id' => 99990,
'blocked_by_id' => undef,
'result' => 'none',
'state' => 'assigned',
'name' => 'opensuse-Factory-DVD-x86_64-Build0048-doc@64bit',
'clone_id' => undef
},
'type' => 'grab_job'
};
[DEBUG] Sent job(s) '99990' to worker '44'
[DEBUG] Assigned job '99985' to worker ID '36'
[DEBUG] [Job#99985] Prepare for being processed by worker 36
[debug] [pid:1526] Started to send message to 36 for job(s) 99985
[debug] [pid:1584] Received ws message: $VAR1 = {
'job' => {
'priority' => 35,
't_finished' => undef,
'assigned_worker_id' => 36,
'test' => 'kde',
'settings' => {
'TEST' => 'kde',
'ARCH' => 'x86_64',
'FLAVOR' => 'NET',
'ISO' => 'openSUSE-13.1-DVD-x86_64-Build0091-Media.iso',
'DISTRI' => 'opensuse',
'DESKTOP' => 'kde',
'MACHINE' => '64bit',
'BUILD' => '0091',
'JOBTOKEN' => 'Z2lXZHgm_NXlh7Rz',
'ISO_MAXSIZE' => '4700372992',
'NAME' => '00099985-opensuse-13.1-NET-x86_64-Build0091-kde@64bit',
'VERSION' => '13.1',
'DVD' => '1'
},
'assets' => {
'iso' => [
'openSUSE-13.1-DVD-x86_64-Build0091-Media.iso'
]
},
'origin_id' => 99961,
'state' => 'assigned',
'name' => 'opensuse-13.1-NET-x86_64-Build0091-kde@64bit',
'group' => 'opensuse test',
'group_id' => 1002,
't_started' => undef,
'blocked_by_id' => undef,
'result' => 'none',
'id' => 99985,
'clone_id' => undef
},
'type' => 'grab_job'
};
[DEBUG] Sent job(s) '99985' to worker '36'
[DEBUG] Assigned job '99987' to worker ID '39'
[DEBUG] [Job#99987] Prepare for being processed by worker 39
[debug] [pid:1526] Started to send message to 39 for job(s) 99987
[DEBUG] Sent job(s) '99987' to worker '39'
[debug] [pid:1582] Received ws message: $VAR1 = {
'type' => 'grab_job',
'job' => {
'state' => 'assigned',
'name' => 'opensuse-13.1-DVD-i586-Build0091-textmode@32bit',
'group_id' => 1001,
'group' => 'opensuse',
'id' => 99987,
'blocked_by_id' => undef,
'result' => 'none',
't_started' => undef,
'clone_id' => undef,
'priority' => 35,
't_finished' => undef,
'assigned_worker_id' => 39,
'assets' => {
'hdd' => [
'openSUSE-13.1-x86_64.hda'
],
'iso' => [
'openSUSE-13.1-DVD-i586-Build0091-Media.iso'
]
},
'test' => 'textmode',
'settings' => {
'JOBTOKEN' => 'ciSJ9NLm68OWwJtM',
'ISO_MAXSIZE' => '4700372992',
'NAME' => '00099987-opensuse-13.1-DVD-i586-Build0091-textmode@32bit',
'ARCH' => 'i586',
'ISO' => 'openSUSE-13.1-DVD-i586-Build0091-Media.iso',
'QEMUCPU' => 'qemu32',
'HDD_1' => 'openSUSE-13.1-x86_64.hda',
'MACHINE' => '32bit',
'BUILD' => '0091',
'VERSION' => '13.1',
'DVD' => '1',
'VIDEOMODE' => 'text',
'TEST' => 'textmode',
'DISTRI' => 'opensuse',
'FLAVOR' => 'DVD',
'DESKTOP' => 'textmode'
},
'origin_id' => 99946
}
};
[DEBUG] Assigned job '99988' to worker ID '42'
[DEBUG] [Job#99988] Prepare for being processed by worker 42
[debug] [pid:1526] Started to send message to 42 for job(s) 99988
[debug] [pid:1615] Received ws message: $VAR1 = {
'job' => {
'priority' => 36,
't_finished' => undef,
'origin_id' => 99940,
'assigned_worker_id' => 42,
'test' => 'doc',
'settings' => {
'DESKTOP' => 'kde',
'QEMUCPU' => 'qemu64',
'DISTRI' => 'opensuse',
'ISO' => 'openSUSE-Factory-DVD-x86_64-Build0048-Media.iso',
'FLAVOR' => 'DVD',
'ARCH' => 'x86_64',
'TEST' => 'doc',
'DVD' => '1',
'VERSION' => 'Factory',
'NAME' => '00099988-opensuse-Factory-DVD-x86_64-Build0048@0815-doc@64bit',
'ISO_MAXSIZE' => '4700372992',
'JOBTOKEN' => 'lPmprX7nbH4R9sky',
'BUILD' => '0048@0815',
'MACHINE' => '64bit'
},
'assets' => {
'iso' => [
'openSUSE-Factory-DVD-x86_64-Build0048-Media.iso'
]
},
'group_id' => 1001,
'group' => 'opensuse',
'id' => 99988,
'result' => 'none',
'blocked_by_id' => undef,
't_started' => undef,
'state' => 'assigned',
'name' => 'opensuse-Factory-DVD-x86_64-Build0048@0815-doc@64bit',
'clone_id' => undef
},
'type' => 'grab_job'
};

[DEBUG] Sent job(s) '99988' to worker '42'
[DEBUG] Assigned job '99994' to worker ID '41'
[DEBUG] [Job#99994] Prepare for being processed by worker 41
[debug] [pid:1526] Started to send message to 41 for job(s) 99994
[debug] [pid:1645] Received ws message: $VAR1 = {
'type' => 'grab_job',
'job' => {
't_started' => undef,
't_finished' => undef,
'result' => 'none',
'blocked_by_id' => undef,
'id' => 99994,
'priority' => 35,
'group_id' => undef,
'name' => 'opensuse-13.1-DVD-x86_64-Build0091-console_tests@64bit',
'state' => 'assigned',
'origin_id' => 99764,
'settings' => {
'DESKTOP' => 'textmode',
'FLAVOR' => 'DVD',
'ISO' => 'openSUSE-13.1-DVD-x86_64-Build0091-Media.iso',
'DISTRI' => 'opensuse',
'TEST' => 'console tests',
'ARCH' => 'x86_64',
'DVD' => '1',
'NAME' => '00099994-opensuse-13.1-DVD-x86_64-Build0091-console_tests@64bit',
'VERSION' => '13.1',
'ISO_MAXSIZE' => '4700372992',
'BUILD' => '0091',
'MACHINE' => '64bit',
'JOBTOKEN' => 'vyAYiihYHE91jk4M'
},
'test' => 'console tests',
'clone_id' => undef,
'assets' => {
'iso' => [
'openSUSE-13.1-DVD-x86_64-Build0091-Media.iso'
]
},
'assigned_worker_id' => 41
}
};
[DEBUG] Sent job(s) '99994' to worker '41'
[DEBUG] Assigned job '99986' to worker ID '38'
[DEBUG] [Job#99986] Prepare for being processed by worker 38
[debug] [pid:1526] Started to send message to 38 for job(s) 99986
[DEBUG] Sent job(s) '99986' to worker '38'
[debug] [pid:1594] Received ws message: $VAR1 = {
'type' => 'grab_job',
'job' => {
'clone_id' => undef,
'name' => 'opensuse-13.1-DVD-i586-Build0092-textmode@32bit',
'state' => 'assigned',
't_started' => undef,
'blocked_by_id' => undef,
'result' => 'none',
'id' => 99986,
'group' => 'opensuse',
'group_id' => 1001,
'test' => 'textmode',
'settings' => {
'DESKTOP' => 'textmode',
'QEMUCPU' => 'qemu32',
'FLAVOR' => 'DVD',
'ISO' => 'openSUSE-13.1-DVD-i586-Build0092-Media.iso',
'DISTRI' => 'opensuse',
'TEST' => 'textmode',
'ARCH' => 'i586',
'VIDEOMODE' => 'text',
'DVD' => '1',
'NAME' => '00099986-opensuse-13.1-DVD-i586-Build0092-textmode@32bit',
'VERSION' => '13.1',
'ISO_MAXSIZE' => '4700372992',
'BUILD' => '0092',
'MACHINE' => '32bit',
'JOBTOKEN' => 'ZSWeipexs6dibQ6R'
},
'assets' => {
'iso' => [
'openSUSE-13.1-DVD-i586-Build0092-Media.iso'
]
},
'assigned_worker_id' => 38,
'origin_id' => 99947,
't_finished' => undef,
'priority' => 35
}
};
[DEBUG] Assigned job '99992' to worker ID '45'
[DEBUG] [Job#99992] Prepare for being processed by worker 45
[debug] [pid:1526] Started to send message to 45 for job(s) 99992
[DEBUG] Sent job(s) '99992' to worker '45'
[debug] [pid:1573] Received ws message: $VAR1 = {
'type' => 'grab_job',
'job' => {
'blocked_by_id' => undef,
'id' => 99992,
'result' => 'none',
't_started' => undef,
'group_id' => 1001,
'group' => 'opensuse',
'name' => 'opensuse-Factory-DVD-x86_64-Build0048-kde@64bit-uefi',
'state' => 'assigned',
'clone_id' => undef,
't_finished' => undef,
'priority' => 36,
'origin_id' => 99936,
'assets' => {
'iso' => [
'openSUSE-Factory-DVD-x86_64-Build0048-Media.iso'
],
'hdd' => [
'openSUSE-13.1-x86_64.hda'
]
},
'test' => 'kde',
'settings' => {
'MACHINE' => '64bit-uefi',
'BUILD' => '0048',
'JOBTOKEN' => '1XE3lY41QkvG7EJB',
'ISO_MAXSIZE' => '4700372992',
'NAME' => '00099992-opensuse-Factory-DVD-x86_64-Build0048-kde@64bit-uefi',
'VERSION' => 'Factory',
'DVD' => '1',
'TEST' => 'kde',
'ARCH' => 'x86_64',
'ISO' => 'openSUSE-Factory-DVD-x86_64-Build0048-Media.iso',
'FLAVOR' => 'DVD',
'DISTRI' => 'opensuse',
'QEMUCPU' => 'qemu64',
'DESKTOP' => 'kde',
'HDD_1' => 'openSUSE-13.1-x86_64.hda'
},
'assigned_worker_id' => 45
}
};
[DEBUG] Assigned job '99991' to worker ID '40'
[DEBUG] [Job#99991] Prepare for being processed by worker 40
[debug] [pid:1526] Started to send message to 40 for job(s) 99991
[DEBUG] Sent job(s) '99991' to worker '40'
[debug] [pid:1602] Received ws message: $VAR1 = {
'type' => 'grab_job',
'job' => {
'id' => 99991,
'blocked_by_id' => undef,
'result' => 'none',
't_started' => undef,
'group_id' => 1001,
'group' => 'opensuse',
'name' => 'opensuse-13.1-DVD-i586-Build0091-kde@32bit',
'state' => 'assigned',
'clone_id' => undef,
't_finished' => undef,
'priority' => 35,
'origin_id' => 99937,
'test' => 'kde',
'assets' => {
'iso' => [
'openSUSE-13.1-DVD-i586-Build0091-Media.iso'
]
},
'settings' => {
'DESKTOP' => 'kde',
'QEMUCPU' => 'qemu32',
'ISO' => 'openSUSE-13.1-DVD-i586-Build0091-Media.iso',
'FLAVOR' => 'DVD',
'DISTRI' => 'opensuse',
'TEST' => 'kde',
'ARCH' => 'i586',
'DVD' => '1',
'NAME' => '00099991-opensuse-13.1-DVD-i586-Build0091-kde@32bit',
'VERSION' => '13.1',
'ISO_MAXSIZE' => '4700372992',
'MACHINE' => '32bit',
'BUILD' => '0091',
'JOBTOKEN' => 'zGjGrSeaLeB_fHh1'
},
'assigned_worker_id' => 40
}
};
[DEBUG] Assigned job '99989' to worker ID '43'
[DEBUG] [Job#99989] Prepare for being processed by worker 43
[debug] [pid:1526] Started to send message to 43 for job(s) 99989
[debug] [pid:1578] Received ws message: $VAR1 = {
'type' => 'grab_job',
'job' => {
'state' => 'assigned',
'name' => 'opensuse-Factory-DVD-x86_64-Build0048-kde@64bit',
'group_id' => 1001,
'group' => 'opensuse',
'id' => 99989,
'blocked_by_id' => undef,
'result' => 'none',
't_started' => undef,
'clone_id' => undef,
'priority' => 36,
't_finished' => undef,
'assigned_worker_id' => 43,
'test' => 'kde',
'assets' => {
'hdd' => [
'openSUSE-13.1-x86_64.hda'
],
'iso' => [
'openSUSE-Factory-DVD-x86_64-Build0048-Media.iso'
]
},
'settings' => {
'HDD_1' => 'openSUSE-13.1-x86_64.hda',
'DESKTOP' => 'kde',
'QEMUCPU' => 'qemu64',
'DISTRI' => 'opensuse',
'ISO' => 'openSUSE-Factory-DVD-x86_64-Build0048-Media.iso',
'FLAVOR' => 'DVD',
'ARCH' => 'x86_64',
'TEST' => 'kde',
'DVD' => '1',
'VERSION' => 'Factory',
'NAME' => '00099989-opensuse-Factory-DVD-x86_64-Build0048-kde@64bit',
'ISO_MAXSIZE' => '4700372992',
'JOBTOKEN' => 'AGVTNaeaO33jWd46',
'MACHINE' => '64bit',
'BUILD' => '0048'
},
'origin_id' => 99939
}
};
[DEBUG] Sent job(s) '99989' to worker '43'
[DEBUG] Scheduler took 1.01963s to perform operations and allocated 10 jobs
[DEBUG] Allocated: { job => 99984, worker => 35 }
[DEBUG] Allocated: { job => 99990, worker => 44 }
[DEBUG] Allocated: { job => 99985, worker => 36 }
[DEBUG] Allocated: { job => 99987, worker => 39 }
[DEBUG] Allocated: { job => 99988, worker => 42 }
[DEBUG] Allocated: { job => 99994, worker => 41 }
[DEBUG] Allocated: { job => 99986, worker => 38 }
[DEBUG] Allocated: { job => 99992, worker => 45 }
[DEBUG] Allocated: { job => 99991, worker => 40 }
[DEBUG] Allocated: { job => 99989, worker => 43 }
ok 1 - Allocated maximum number of jobs that could have been allocated
ok 2 - Job (99984) allocated only once
ok 3 - Worker (35) used only once
ok 4 - Job (99990) allocated only once
ok 5 - Worker (44) used only once
ok 6 - Job (99985) allocated only once
ok 7 - Worker (36) used only once
ok 8 - Job (99987) allocated only once
ok 9 - Worker (39) used only once
ok 10 - Job (99988) allocated only once
ok 11 - Worker (42) used only once
ok 12 - Job (99994) allocated only once
ok 13 - Worker (41) used only once
ok 14 - Job (99986) allocated only once
ok 15 - Worker (38) used only once
ok 16 - Job (99992) allocated only once
ok 17 - Worker (45) used only once
ok 18 - Job (99991) allocated only once
ok 19 - Worker (40) used only once
ok 20 - Job (99989) allocated only once
ok 21 - Worker (43) used only once
ok 22 - Job(99984) back in scheduled state
ok 23 - Job(99985) back in scheduled state
ok 24 - Job(99986) back in scheduled state
ok 25 - Job(99987) back in scheduled state
ok 26 - Job(99988) back in scheduled state
ok 27 - Job(99989) back in scheduled state
ok 28 - Job(99990) back in scheduled state
ok 29 - Job(99991) back in scheduled state
ok 30 - Job(99992) back in scheduled state
ok 31 - Job(99993) back in scheduled state
ok 32 - Job(99994) back in scheduled state
ok 33 - Job(99995) back in scheduled state
[info] [pid:1526] Worker 4 websocket connection closed - 1006
[info] [pid:1526] Worker 3 websocket connection closed - 1006
[info] [pid:1526] Worker 5 websocket connection closed - 1006
[info] [pid:1526] Worker 9 websocket connection closed - 1006
[info] [pid:1526] Worker 10 websocket connection closed - 1006
[info] [pid:1526] Worker 11 websocket connection closed - 1006
[info] [pid:1526] Worker 7 websocket connection closed - 1006
[info] [pid:1526] Worker 12 websocket connection closed - 1006
[info] [pid:1526] Worker 6 websocket connection closed - 1006
[info] [pid:1526] Worker 8 websocket connection closed - 1006
[info] [pid:1526] Worker 45 websocket connection closed - 1006
[info] [pid:1526] Worker 19 websocket connection closed - 1006
[info] [pid:1526] Worker 46 websocket connection closed - 1006
[info] [pid:1526] Worker 43 websocket connection closed - 1006
[info] [pid:1526] Worker 39 websocket connection closed - 1006
[info] [pid:1526] Worker 24 websocket connection closed - 1006
[info] [pid:1526] Worker 36 websocket connection closed - 1006
[info] [pid:1526] Worker 27 websocket connection closed - 1006
[info] [pid:1526] Worker 17 websocket connection closed - 1006
[info] [pid:1526] Worker 34 websocket connection closed - 1006
[info] [pid:1526] Worker 38 websocket connection closed - 1006
[info] [pid:1526] Worker 32 websocket connection closed - 1006
[info] [pid:1526] Worker 44 websocket connection closed - 1006
[info] [pid:1526] Worker 37 websocket connection closed - 1006
[info] [pid:1526] Worker 31 websocket connection closed - 1006
[info] [pid:1526] Worker 40 websocket connection closed - 1006
[info] [pid:1526] Worker 48 websocket connection closed - 1006
[info] [pid:1526] Worker 16 websocket connection closed - 1006
[info] [pid:1526] Worker 47 websocket connection closed - 1006
[info] [pid:1526] Worker 51 websocket connection closed - 1006
[info] [pid:1526] Worker 26 websocket connection closed - 1006
[info] [pid:1526] Worker 25 websocket connection closed - 1006
[info] [pid:1526] Worker 42 websocket connection closed - 1006
[info] [pid:1526] Worker 52 websocket connection closed - 1006
[info] [pid:1526] Worker 18 websocket connection closed - 1006
[info] [pid:1526] Worker 49 websocket connection closed - 1006
[info] [pid:1526] Worker 14 websocket connection closed - 1006
[info] [pid:1526] Worker 29 websocket connection closed - 1006
[info] [pid:1526] Worker 35 websocket connection closed - 1006
[info] [pid:1526] Worker 20 websocket connection closed - 1006
[info] [pid:1526] Worker 33 websocket connection closed - 1006
[info] [pid:1526] Worker 15 websocket connection closed - 1006
[info] [pid:1526] Worker 28 websocket connection closed - 1006
[info] [pid:1526] Worker 21 websocket connection closed - 1006
[info] [pid:1526] Worker 22 websocket connection closed - 1006
[info] [pid:1526] Worker 13 websocket connection closed - 1006
[info] [pid:1526] Worker 41 websocket connection closed - 1006
[info] [pid:1526] Worker 23 websocket connection closed - 1006
[info] [pid:1526] Worker 50 websocket connection closed - 1006
[info] [pid:1526] Worker 30 websocket connection closed - 1006
# Starting unstable worker. Instance: 1 for host http://localhost:9526
# Starting unstable worker. Instance: 2 for host http://localhost:9526
# PID of openqa-worker-unstable: 1655
# PID of openqa-worker-unstable: 1656
# Starting unstable worker. Instance: 3 for host http://localhost:9526
# Starting unstable worker. Instance: 4 for host http://localhost:9526
# PID of openqa-worker-unstable: 1657
[info] [pid:1655] worker 1:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: qemu_i386,qemu_x86_64
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/1
[error] [pid:1655] The average load 49.75 is exceeding the configured threshold of 40.
[info] [pid:1655] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1655] Registering with openQA http://localhost:9526
# Starting unstable worker. Instance: 5 for host http://localhost:9526
# PID of openqa-worker-unstable: 1659
[info] [pid:1656] worker 2:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/2
[error] [pid:1656] The average load 49.75 is exceeding the configured threshold of 40.
[info] [pid:1656] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1656] Registering with openQA http://localhost:9526
# Starting unstable worker. Instance: 6 for host http://localhost:9526
# PID of openqa-worker-unstable: 1660
[info] [pid:1657] worker 3:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/3
# Starting unstable worker. Instance: 7 for host http://localhost:9526
# PID of openqa-worker-unstable: 1662
[error] [pid:1657] The average load 49.75 is exceeding the configured threshold of 40.
[info] [pid:1657] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1657] Registering with openQA http://localhost:9526
[info] [pid:1659] worker 4:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/4
[error] [pid:1659] The average load 49.75 is exceeding the configured threshold of 40.
[info] [pid:1659] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1659] Registering with openQA http://localhost:9526
# Starting unstable worker. Instance: 8 for host http://localhost:9526
# PID of openqa-worker-unstable: 1663
[info] [pid:1660] worker 5:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/5
[error] [pid:1660] The average load 49.75 is exceeding the configured threshold of 40.
[info] [pid:1660] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1660] Registering with openQA http://localhost:9526
# PID of openqa-worker-unstable: 1666
# Starting unstable worker. Instance: 9 for host http://localhost:9526
[info] [pid:1662] worker 6:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/6
[error] [pid:1662] The average load 49.75 is exceeding the configured threshold of 40.
[info] [pid:1662] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1662] Registering with openQA http://localhost:9526
# PID of openqa-worker-unstable: 1668
# Starting unstable worker. Instance: 10 for host http://localhost:9526
[info] [pid:1663] worker 7:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/7
[error] [pid:1663] The average load 49.75 is exceeding the configured threshold of 40.
[info] [pid:1663] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1663] Registering with openQA http://localhost:9526
[info] [pid:1666] worker 8:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/8
# Starting unstable worker. Instance: 11 for host http://localhost:9526
# PID of openqa-worker-unstable: 1670
[error] [pid:1666] The average load 49.75 is exceeding the configured threshold of 40.
[info] [pid:1666] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1666] Registering with openQA http://localhost:9526
[info] [pid:1668] worker 9:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/9
# PID of openqa-worker-unstable: 1671
# Starting unstable worker. Instance: 12 for host http://localhost:9526
[error] [pid:1668] The average load 49.75 is exceeding the configured threshold of 40.
[info] [pid:1668] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1668] Registering with openQA http://localhost:9526
[info] [pid:1670] worker 10:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/10
[error] [pid:1670] The average load 49.75 is exceeding the configured threshold of 40.
[info] [pid:1670] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1670] Registering with openQA http://localhost:9526
# PID of openqa-worker-unstable: 1675
# Starting unstable worker. Instance: 13 for host http://localhost:9526
[info] [pid:1671] worker 11:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/11
[error] [pid:1671] The average load 49.75 is exceeding the configured threshold of 40.
[info] [pid:1671] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1671] Registering with openQA http://localhost:9526
# PID of openqa-worker-unstable: 1677
# Starting unstable worker. Instance: 14 for host http://localhost:9526
[info] [pid:1675] worker 12:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/12
[error] [pid:1675] The average load 49.75 is exceeding the configured threshold of 40.
[info] [pid:1675] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1675] Registering with openQA http://localhost:9526
# Starting unstable worker. Instance: 15 for host http://localhost:9526
# PID of openqa-worker-unstable: 1679
# PID of openqa-worker-unstable: 1681
# Starting unstable worker. Instance: 16 for host http://localhost:9526
[info] [pid:1677] worker 13:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/13
[error] [pid:1677] The average load 49.75 is exceeding the configured threshold of 40.
[info] [pid:1677] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1677] Registering with openQA http://localhost:9526
[info] [pid:1679] worker 14:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/14
# Starting unstable worker. Instance: 17 for host http://localhost:9526
# PID of openqa-worker-unstable: 1682
[info] [pid:1681] worker 15:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/15
[error] [pid:1681] The average load 49.75 is exceeding the configured threshold of 40.
[info] [pid:1681] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[error] [pid:1679] The average load 49.75 is exceeding the configured threshold of 40.
[info] [pid:1679] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1681] Registering with openQA http://localhost:9526
[info] [pid:1679] Registering with openQA http://localhost:9526
# Starting unstable worker. Instance: 18 for host http://localhost:9526
# PID of openqa-worker-unstable: 1684
# PID of openqa-worker-unstable: 1687
# Starting unstable worker. Instance: 19 for host http://localhost:9526
[info] [pid:1682] worker 16:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/16
[error] [pid:1682] The average load 49.75 is exceeding the configured threshold of 40.
[info] [pid:1682] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1682] Registering with openQA http://localhost:9526
# Starting unstable worker. Instance: 20 for host http://localhost:9526
# PID of openqa-worker-unstable: 1688
[info] [pid:1684] worker 17:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/17
# PID of openqa-worker-unstable: 1689
# Starting unstable worker. Instance: 21 for host http://localhost:9526
[error] [pid:1684] The average load 49.75 is exceeding the configured threshold of 40.
[info] [pid:1684] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1684] Registering with openQA http://localhost:9526
# Starting unstable worker. Instance: 22 for host http://localhost:9526
# PID of openqa-worker-unstable: 1692
[info] [pid:1657] Establishing ws connection via ws://localhost:9526/api/v1/ws/5
[info] [pid:1687] worker 18:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/18
[info] [pid:1688] worker 19:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/19
[error] [pid:1688] The average load 49.75 is exceeding the configured threshold of 40.
[error] [pid:1687] The average load 49.75 is exceeding the configured threshold of 40.
[info] [pid:1688] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1687] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1688] Registering with openQA http://localhost:9526
[info] [pid:1687] Registering with openQA http://localhost:9526
# Starting unstable worker. Instance: 23 for host http://localhost:9526
# PID of openqa-worker-unstable: 1693
[info] [pid:1689] worker 20:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/20
[error] [pid:1689] The average load 49.75 is exceeding the configured threshold of 40.
# PID of openqa-worker-unstable: 1696
# Starting unstable worker. Instance: 24 for host http://localhost:9526
[info] [pid:1689] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1692] worker 21:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/21
[info] [pid:1689] Registering with openQA http://localhost:9526
[error] [pid:1692] The average load 49.75 is exceeding the configured threshold of 40.
[info] [pid:1692] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1692] Registering with openQA http://localhost:9526
# Starting unstable worker. Instance: 25 for host http://localhost:9526
# PID of openqa-worker-unstable: 1697
[info] [pid:1693] worker 22:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/22
[error] [pid:1693] The average load 49.75 is exceeding the configured threshold of 40.
[info] [pid:1693] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1693] Registering with openQA http://localhost:9526
# Starting unstable worker. Instance: 26 for host http://localhost:9526
# PID of openqa-worker-unstable: 1700
[info] [pid:1696] worker 23:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/23
[error] [pid:1696] The average load 49.75 is exceeding the configured threshold of 40.
[info] [pid:1696] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1696] Registering with openQA http://localhost:9526
# PID of openqa-worker-unstable: 1701
[info] [pid:1697] worker 24:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/24
[error] [pid:1697] The average load 49.75 is exceeding the configured threshold of 40.
[info] [pid:1697] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1697] Registering with openQA http://localhost:9526
# Starting unstable worker. Instance: 27 for host http://localhost:9526
# PID of openqa-worker-unstable: 1705
# Starting unstable worker. Instance: 28 for host http://localhost:9526
[info] [pid:1701] worker 26:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/26
[info] [pid:1700] worker 25:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/25
[error] [pid:1701] The average load 49.75 is exceeding the configured threshold of 40.
[error] [pid:1700] The average load 49.75 is exceeding the configured threshold of 40.
[info] [pid:1701] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1700] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1701] Registering with openQA http://localhost:9526
[info] [pid:1700] Registering with openQA http://localhost:9526
[info] [pid:1670] Establishing ws connection via ws://localhost:9526/api/v1/ws/8
[info] [pid:1705] worker 27:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/27
[error] [pid:1705] The average load 49.75 is exceeding the configured threshold of 40.
[info] [pid:1705] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1705] Registering with openQA http://localhost:9526
# PID of openqa-worker-unstable: 1707
# Starting unstable worker. Instance: 29 for host http://localhost:9526
# Starting unstable worker. Instance: 30 for host http://localhost:9526
# PID of openqa-worker-unstable: 1710
[info] [pid:1707] worker 28:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/28
[error] [pid:1707] The average load 49.75 is exceeding the configured threshold of 40.
# PID of openqa-worker-unstable: 1711
[info] [pid:1707] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1707] Registering with openQA http://localhost:9526
# Waiting for worker with ID 6
[info] [pid:1711] worker 30:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/30
[info] [pid:1710] worker 29:
- config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
- name used to register: ddc4e34409cb
- worker address (WORKER_HOSTNAME): 127.0.0.1
- isotovideo version: 0
- websocket API version: 1
- web UI hosts: http://localhost:9526
- class: ?
- no cleanup: no
- pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/29
[error] [pid:1711] The average load 49.75 is exceeding the configured threshold of 40.
[info] [pid:1711] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1711] Registering with openQA http://localhost:9526
[error] [pid:1710] The average load 49.75 is exceeding the configured threshold of 40.
[info] [pid:1710] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
[info] [pid:1710] Registering with openQA http://localhost:9526
[info] [pid:1660] Establishing ws connection via ws://localhost:9526/api/v1/ws/10
[info] [pid:1659] Establishing ws connection via ws://localhost:9526/api/v1/ws/9
[info] [pid:1666] Establishing ws connection via ws://localhost:9526/api/v1/ws/12
[info] [pid:1656] Establishing ws connection via ws://localhost:9526/api/v1/ws/3
[info] [pid:1675] Establishing ws connection via ws://localhost:9526/api/v1/ws/19
[info] [pid:1655] Establishing ws connection via ws://localhost:9526/api/v1/ws/4
[info] [pid:1662] Establishing ws connection via ws://localhost:9526/api/v1/ws/11
[info] [pid:1663] Establishing ws connection via ws://localhost:9526/api/v1/ws/7
[info] [pid:1668] Establishing ws connection via ws://localhost:9526/api/v1/ws/6
# Waiting for worker with ID 7
# Waiting for worker with ID 8
# Waiting for worker with ID 9
# Waiting for worker with ID 10
# Waiting for worker with ID 11
# Waiting for worker with ID 12
[debug] [pid:1527] Job 99992 reset to state scheduled
# Waiting for worker with ID 13
[info] [pid:1671] Establishing ws connection via ws://localhost:9526/api/v1/ws/45
[info] [pid:1687] Establishing ws connection via ws://localhost:9526/api/v1/ws/27
[info] [pid:1707] Establishing ws connection via ws://localhost:9526/api/v1/ws/16
[info] [pid:1693] Establishing ws connection via ws://localhost:9526/api/v1/ws/32
[info] [pid:1711] Establishing ws connection via ws://localhost:9526/api/v1/ws/51
[info] [pid:1697] Establishing ws connection via ws://localhost:9526/api/v1/ws/37
[info] [pid:1689] Establishing ws connection via ws://localhost:9526/api/v1/ws/34
[info] [pid:1688] Establishing ws connection via ws://localhost:9526/api/v1/ws/17
[debug] [pid:1527] Job 99989 reset to state scheduled
[info] [pid:1679] Establishing ws connection via ws://localhost:9526/api/v1/ws/43
[debug] [pid:1527] Job 99985 reset to state scheduled
[info] [pid:1684] Establishing ws connection via ws://localhost:9526/api/v1/ws/36
[debug] [pid:1527] Job 99987 reset to state scheduled
[info] [pid:1681] Establishing ws connection via ws://localhost:9526/api/v1/ws/39
[info] [pid:1700] Establishing ws connection via ws://localhost:9526/api/v1/ws/31
[info] [pid:1705] Establishing ws connection via ws://localhost:9526/api/v1/ws/48
[info] [pid:1682] Establishing ws connection via ws://localhost:9526/api/v1/ws/24
[info] [pid:1677] Establishing ws connection via ws://localhost:9526/api/v1/ws/46
[debug] [pid:1527] Job 99986 reset to state scheduled
[info] [pid:1692] Establishing ws connection via ws://localhost:9526/api/v1/ws/38
[debug] [pid:1527] Job 99991 reset to state scheduled
[info] [pid:1701] Establishing ws connection via ws://localhost:9526/api/v1/ws/40
[info] [pid:1710] Establishing ws connection via ws://localhost:9526/api/v1/ws/47
[debug] [pid:1527] Job 99990 reset to state scheduled
[info] [pid:1696] Establishing ws connection via ws://localhost:9526/api/v1/ws/44
# No worker with ID 13 active
# Waiting for worker with ID 14
# No worker with ID 14 active
# Waiting for worker with ID 15
# No worker with ID 15 active
# Waiting for worker with ID 16
# Waiting for worker with ID 17
# Waiting for worker with ID 18
# No worker with ID 18 active
[DEBUG] Scheduling: Free workers: 30/52; Scheduled jobs: 12
[DEBUG] Need to schedule 1 parallel jobs for job 99985 (with priority 35)
[DEBUG] Need to schedule 1 parallel jobs for job 99986 (with priority 35)
[DEBUG] Need to schedule 1 parallel jobs for job 99987 (with priority 35)
[DEBUG] Need to schedule 1 parallel jobs for job 99991 (with priority 35)
[DEBUG] Need to schedule 1 parallel jobs for job 99989 (with priority 36)
[DEBUG] Need to schedule 1 parallel jobs for job 99990 (with priority 36)
[DEBUG] Need to schedule 1 parallel jobs for job 99992 (with priority 36)
[DEBUG] Need to schedule 1 parallel jobs for job 99927 (with priority 45)
[DEBUG] Need to schedule 1 parallel jobs for job 99928 (with priority 46)
[DEBUG] Need to schedule 1 parallel jobs for job 99983 (with priority 50)
[DEBUG] limit reached, scheduling no additional jobs (max_running_jobs=-1, free workers=30, running=5, allocated=10)
[DEBUG] Assigned job '99990' to worker ID '36'
[DEBUG] [Job#99990] Prepare for being processed by worker 36
[DEBUG] Job 99990 reset to state scheduled
[DEBUG] Assigned job '99928' to worker ID '3'
[DEBUG] [Job#99928] Prepare for being processed by worker 3
[DEBUG] Job 99928 reset to state scheduled
[DEBUG] Assigned job '99987' to worker ID '19'
[DEBUG] [Job#99987] Prepare for being processed by worker 19
[DEBUG] Job 99987 reset to state scheduled
[DEBUG] Assigned job '99985' to worker ID '9'
[DEBUG] [Job#99985] Prepare for being processed by worker 9
[DEBUG] Job 99985 reset to state scheduled
[DEBUG] Assigned job '99927' to worker ID '7'
[DEBUG] [Job#99927] Prepare for being processed by worker 7
[DEBUG] Job 99927 reset to state scheduled
[DEBUG] Assigned job '99986' to worker ID '8'
[DEBUG] [Job#99986] Prepare for being processed by worker 8
[DEBUG] Job 99986 reset to state scheduled
[DEBUG] Assigned job '99992' to worker ID '10'
[DEBUG] [Job#99992] Prepare for being processed by worker 10
[DEBUG] Job 99992 reset to state scheduled
[DEBUG] Assigned job '99991' to worker ID '27'
[DEBUG] [Job#99991] Prepare for being processed by worker 27
[DEBUG] Job 99991 reset to state scheduled
[DEBUG] Assigned job '99983' to worker ID '12'
[DEBUG] [Job#99983] Prepare for being processed by worker 12
[DEBUG] Job 99983 reset to state scheduled
[DEBUG] Assigned job '99989' to worker ID '4'
[DEBUG] [Job#99989] Prepare for being processed by worker 4
[DEBUG] Job 99989 reset to state scheduled
[DEBUG] Scheduler took 10.57659s to perform operations and allocated 0 jobs
ok 34 - All failed allocation on second step - workers were killed
ok 35 - Job(99984) is still in scheduled state
ok 36 - Job(99985) is still in scheduled state
ok 37 - Job(99986) is still in scheduled state
ok 38 - Job(99987) is still in scheduled state
ok 39 - Job(99988) is still in scheduled state
ok 40 - Job(99989) is still in scheduled state
ok 41 - Job(99990) is still in scheduled state
ok 42 - Job(99991) is still in scheduled state
ok 43 - Job(99992) is still in scheduled state
ok 44 - Job(99993) is still in scheduled state
ok 45 - Job(99994) is still in scheduled state
ok 46 - Job(99995) is still in scheduled state
1..46
ok 4 - Simulation of heavy unstable load
# Subtest: Websocket server - close connection test
# Starting WebSocket service. Port: 9527
# Bogus: 1 | No wait: 0
# PID of openqa-websocket: 1719
# Starting standard worker. Instance: 2 for host http://localhost:9526
[info] Listening at "http://127.0.0.1:9527"
Web application available at http://127.0.0.1:9527
# worker out: [info] [pid:1720] worker 2:
# - config file: /home/squamata/project/test-results/fullstack/scheduler/config/workers.ini
# - name used to register: ddc4e34409cb
# - worker address (WORKER_HOSTNAME): 127.0.0.1
# - isotovideo version: 41
# - websocket API version: 1
# - web UI hosts: http://localhost:9526
# - class: ?
# - no cleanup: no
# - pool directory: /home/squamata/project/test-results/fullstack/scheduler/openqa/pool/2
# worker out: [error] [pid:1720] The average load 45.46 is exceeding the configured threshold of 40.
# worker out: [info] [pid:1720] Project dir for host http://localhost:9526 is /home/squamata/project/test-results/fullstack/scheduler/openqa/share
# worker out: [info] [pid:1720] Registering with openQA http://localhost:9526
# worker out: [info] [pid:1720] Establishing ws connection via ws://localhost:9526/api/v1/ws/3
# worker out: [info] [pid:1720] Registered and connected via websockets with openQA host http://localhost:9526 and worker ID 3
# [warn] [pid:1720] The average load 45.46 is exceeding the configured threshold of 40. - checking again for web UI 'http://localhost:9526' in 147.60 s
[warn] A message received from unknown worker connection
[debug] [pid:1719] A message received from unknown worker connection (terminating ws): {
"reason" => "The average load 45.46 is exceeding the configured threshold of 40.",
"status" => "broken",
"type" => "worker_status"
}
[error] [pid:1719] Worker not found for given connection during connection close
# worker out: [warn] [pid:1720] Websocket connection to http://localhost:9526/api/v1/ws/3 finished by remote side with code 1008, Connection terminated from WebSocket server - thought dead - trying again in 0 seconds
ok 1 - closed ws connection logged by worker
1..1
ok 5 - Websocket server - close connection test
ok 6 - no (unexpected) warnings (via done_testing)
1..6
# ### all processes: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
# squamata 8 4.8 0.0 1731912 51108 ? Ssl 13:47 0:29 /bin/circleci-agent _internal inner --innerConfig -
# squamata 303 0.0 0.0 9340 3968 pts/1 Ss+ 13:48 0:00 \_ /bin/bash tools/ci/run_unit_tests.sh fullstack-unstable
# squamata 696 0.0 0.0 6032 2688 pts/1 S+ 13:49 0:00 \_ make test-fullstack-unstable
# squamata 701 0.0 0.0 6032 2608 pts/1 S+ 13:49 0:00 \_ make test-with-database FULLSTACK=1 TIMEOUT_M=15 PROVE_ARGS=--harness TAP::Harness::JUnit --timer t/05-scheduler-full.t RETRY=5
# squamata 734 0.0 0.0 6032 2612 pts/1 S+ 13:49 0:00 \_ make test-unit-and-integration TEST_PG=DBI:Pg:dbname=openqa_test;host=/dev/shm/tpg
# squamata 739 0.0 0.0 9340 3508 pts/1 S+ 13:49 0:00 \_ /bin/sh -c export GLOBIGNORE="";\ export DEVEL_COVER_DB_FORMAT=JSON;\ export PERL5OPT="-mJSON::PP -MDevel::Cover=-select_re,'^/lib',+ignore_re,lib/perlcritic/Perl/Critic/Policy,-coverage,statement,-db,cover_db_fullstack-unstable, -It/lib -I/home/squamata/project/t/lib -I/home/squamata/project/external/os-autoinst-common/lib -MOpenQA::Test::PatchDeparse";\ RETRY=5 HOOK=./tools/delete-coverdb-folder timeout -s SIGINT -k 5 -v $((15 * 2 * (5 + 1) ))m tools/retry prove -l --harness TAP::Harness::JUnit --timer t/05-scheduler-full.t
# squamata 740 0.0 0.0 5716 1192 pts/1 S 13:49 0:00 \_ timeout -s SIGINT -k 5 -v 180m tools/retry prove -l --harness TAP::Harness::JUnit --timer t/05-scheduler-full.t
# squamata 741 0.0 0.0 9340 3760 pts/1 S 13:49 0:00 \_ /bin/sh -e tools/retry prove -l --harness TAP::Harness::JUnit --timer t/05-scheduler-full.t
# squamata 1521 0.4 0.0 46400 41760 pts/1 S 13:55 0:00 \_ /usr/bin/perl /usr/bin/prove -l --harness TAP::Harness::JUnit --timer t/05-scheduler-full.t
# squamata 1523 5.0 0.2 227600 200104 pts/1 S 13:55 0:07 \_ /usr/bin/perl t/05-scheduler-full.t
# squamata 1727 0.0 0.0 10320 3668 pts/1 R 13:57 0:00 \_ ps auxf
# squamata 1 0.0 0.0 1112 4 pts/0 Ss 13:47 0:00 /sbin/docker-init -- /bin/sh
# squamata 7 0.0 0.0 10128 4588 pts/0 S+ 13:47 0:00 /bin/sh
# squamata 723 0.0 0.0 188828 26400 ? Ss 13:49 0:00 /usr/lib/postgresql13/bin/postgres -D /dev/shm/tpg
# squamata 724 0.0 0.0 43184 5548 ? Ss 13:49 0:00 \_ postgres: logger
# squamata 726 0.0 0.0 189268 22352 ? Ss 13:49 0:00 \_ postgres: checkpointer
# squamata 727 0.0 0.0 188828 6500 ? Ss 13:49 0:00 \_ postgres: background writer
# squamata 728 0.0 0.0 188828 10572 ? Ss 13:49 0:00 \_ postgres: walwriter
# squamata 729 0.0 0.0 189384 9736 ? Ss 13:49 0:00 \_ postgres: autovacuum launcher
# squamata 730 0.0 0.0 43712 6644 ? Ss 13:49 0:00 \_ postgres: stats collector
# squamata 731 0.0 0.0 189260 7712 ? Ss 13:49 0:00 \_ postgres: logical replication launcher
# squamata 1525 0.2 0.0 193440 31624 ? Ss 13:55 0:00 \_ postgres: squamata openqa_test [local] idle
#
# ### processes in tree: PID TTY STAT TIME COMMAND
# 303 pts/1 Ss+ 0:00 /bin/bash tools/ci/run_unit_tests.sh fullstack-unstable
# 696 pts/1 S+ 0:00 \_ make test-fullstack-unstable
# 701 pts/1 S+ 0:00 \_ make test-with-database FULLSTACK=1 TIMEOUT_M=15 PROVE_ARGS=--harness TAP::Harness::JUnit --timer t/05-scheduler-full.t RETRY=5
# 734 pts/1 S+ 0:00 \_ make test-unit-and-integration TEST_PG=DBI:Pg:dbname=openqa_test;host=/dev/shm/tpg
# 739 pts/1 S+ 0:00 \_ /bin/sh -c export GLOBIGNORE="";\ export DEVEL_COVER_DB_FORMAT=JSON;\ export PERL5OPT="-mJSON::PP -MDevel::Cover=-select_re,'^/lib',+ignore_re,lib/perlcritic/Perl/Critic/Policy,-coverage,statement,-db,cover_db_fullstack-unstable, -It/lib -I/home/squamata/project/t/lib -I/home/squamata/project/external/os-autoinst-common/lib -MOpenQA::Test::PatchDeparse";\ RETRY=5 HOOK=./tools/delete-coverdb-folder timeout -s SIGINT -k 5 -v $((15 * 2 * (5 + 1) ))m tools/retry prove -l --harness TAP::Harness::JUnit --timer t/05-scheduler-full.t
# 740 pts/1 S 0:00 \_ timeout -s SIGINT -k 5 -v 180m tools/retry prove -l --harness TAP::Harness::JUnit --timer t/05-scheduler-full.t
# 741 pts/1 S 0:00 \_ /bin/sh -e tools/retry prove -l --harness TAP::Harness::JUnit --timer t/05-scheduler-full.t
# 1521 pts/1 S 0:00 \_ /usr/bin/perl /usr/bin/prove -l --harness TAP::Harness::JUnit --timer t/05-scheduler-full.t
# 1523 pts/1 S 0:07 \_ /usr/bin/perl t/05-scheduler-full.t
# 1728 pts/1 R 0:00 \_ ps Tf
#
    (1-1/1)