Project

General

Profile

Actions

action #23320

closed

openQA Tests - action #20378: [tools]Too many 502 on openqa

[tools][sprint 201709.2][sprint 201710.1][sprint 201710.2] move locks/mutexes/barriers/job restarts out of scheduler

Added by EDiGiacinto over 6 years ago. Updated over 6 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Feature requests
Target version:
Start date:
2017-10-11
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)

Description

With the new scheduler logic, we now use the dbus service to schedule: this operation could keep busy the dbus service for long time, delaying access to locks, mutexes, barriers and job restarts.
We should move those access to resources in another process (and hence makes sense to me to have a new "ResourceAllocator" service) before it could become a real problem


Subtasks 5 (0 open5 closed)

action #25930: [tools][sprint 201710.1][sprint 201710.2] Split fullstack tests in different build matrixResolvedEDiGiacinto2017-10-11

Actions
action #25936: [tools][sprint 201710.1][sprint 201710.2] Regression in OpenQA::ServerStartup - even if logfile is supplied, messages are printed in journalResolvedEDiGiacinto2017-10-11

Actions
action #25944: [tools][sprint 201710.1][sprint 201710.2] Make job update take into account of ASSIGNED and SETUP stateResolvedEDiGiacinto2017-10-11

Actions
action #25952: [tools][sprint 201710.1][sprint 201710.2] Reset Job state in Worker also when we loose current_host (due to persistent failures while contacting webserver)ResolvedEDiGiacinto2017-10-11

Actions
action #25954: [tools][sprint 201710.1][sprint 201710.2] Regression: no live streamResolvedEDiGiacinto2017-10-11

Actions

Related issues 1 (0 open1 closed)

Related to openQA Project - action #23378: [tools][sprint 201709.1] Enhancement and cleanup of "assigned" stateResolvedEDiGiacinto2017-08-14

Actions
Actions #1

Updated by EDiGiacinto over 6 years ago

  • Project changed from openQA Tests to openQA Project
Actions #2

Updated by okurz over 6 years ago

  • Related to action #23378: [tools][sprint 201709.1] Enhancement and cleanup of "assigned" state added
Actions #3

Updated by EDiGiacinto over 6 years ago

temporary workaround applied in production: https://github.com/os-autoinst/openQA/pull/1433
we may bump those values as well to reduce the websocket server load caused also by dispatching jobs over ws.

Actions #4

Updated by nicksinger over 6 years ago

  • Copied to action #23536: [tools] org.freedesktop.DBus.Error.NoReply: Did not receive a reply. appreas regularly in openQA logs added
Actions #5

Updated by nicksinger over 6 years ago

  • Copied to deleted (action #23536: [tools] org.freedesktop.DBus.Error.NoReply: Did not receive a reply. appreas regularly in openQA logs)
Actions #6

Updated by szarate over 6 years ago

  • Subject changed from [tools] move locks/mutexes/barriers out of scheduler to [tools][Sprint 1] move locks/mutexes/barriers out of scheduler
  • Category set to Feature requests
  • Assignee set to EDiGiacinto
  • Target version set to Milestone 10
Actions #7

Updated by szarate over 6 years ago

  • Subject changed from [tools][Sprint 1] move locks/mutexes/barriers out of scheduler to [tools][sprint 201709.2] move locks/mutexes/barriers out of scheduler
Actions #8

Updated by EDiGiacinto over 6 years ago

  • Status changed from New to In Progress
  • % Done changed from 0 to 70

Bits are already there, needs to be tested widely, since are changing also our build matrix. Will proceed to create a separate branch for the feature right after everything is good to go.

Actions #9

Updated by szarate over 6 years ago

  • Subject changed from [tools][sprint 201709.2] move locks/mutexes/barriers out of scheduler to [tools][sprint 201709.2][sprint 201710.1] move locks/mutexes/barriers out of scheduler
Actions #10

Updated by EDiGiacinto over 6 years ago

  • % Done changed from 70 to 90
Actions #11

Updated by EDiGiacinto over 6 years ago

  • Subject changed from [tools][sprint 201709.2][sprint 201710.1] move locks/mutexes/barriers out of scheduler to [tools][sprint 201709.2][sprint 201710.1][sprint 201710.2] move locks/mutexes/barriers out of scheduler
Actions #12

Updated by EDiGiacinto over 6 years ago

  • Subject changed from [tools][sprint 201709.2][sprint 201710.1][sprint 201710.2] move locks/mutexes/barriers out of scheduler to [tools][sprint 201709.2][sprint 201710.1][sprint 201710.2] move locks/mutexes/barriers/job restarts out of scheduler
Actions #13

Updated by EDiGiacinto over 6 years ago

  • Status changed from In Progress to Resolved

PR Merged

Actions

Also available in: Atom PDF