action #32725
closedcoordination #32851: [tools][EPIC] Scheduling redesign
[tools] Scheduler job_grab/filter_jobs refactoring
0%
Description
For the moment it would be sufficient to refactor job_grab, and pass over schemas objects instead of hashes and re-use them in the filtering phase, meanwhile refactoring properly those areas (e.g. extracting them) - better if we also optimize the queries.
Kinda of a hard task, since could potentially make things 'shaky'.
Updated by coolo about 6 years ago
- Target version set to Ready
Taken that we usually have < 2000 jobs scheduled, I think it's worth an experiment to grab all open jobs and schedule in memory. There is only one scheduler process, so IMO this should simplify a lot. Yes, hard task - but worth the time spent to make the whole scheduling understandable.
Updated by EDiGiacinto about 6 years ago
- Description updated (diff)
- Parent task set to #32851
Updated by szarate almost 6 years ago
- Start date changed from 2018-03-02 to 2018-05-05
due to changes in a related task
Updated by szarate almost 6 years ago
- Target version changed from Ready to Current Sprint
We agreed during the sprint planning to keep going with this: https://github.com/os-autoinst/openQA/pull/1718 ettore will take over coolo's branch.
Updated by EDiGiacinto over 5 years ago
PR: https://github.com/os-autoinst/openQA/pull/1729 ( missing acceptance testing now )
Updated by szarate over 5 years ago
- Assignee set to EDiGiacinto
I think there are still some rough edges here, right?
Updated by EDiGiacinto over 5 years ago
- Status changed from New to Feedback
- Assignee deleted (
EDiGiacinto)
Yes, but as i'm not the author of this set of changes, can't help much more from here - i adapted the tests we already had for the scheduler and did some adaptations to the issues i could notice, but i think there are still bugs to catch.
Latest related change to it is: https://github.com/os-autoinst/openQA/pull/1743
Updated by EDiGiacinto over 5 years ago
- Related to action #39560: Tests for blocked_by and loops inside of it added
Updated by EDiGiacinto over 5 years ago
- Related to action #39629: openQA Scheduler refactor fallout added
Updated by coolo over 5 years ago
- Status changed from Feedback to Resolved
The scheduler was refactored - we have some new problems (like starving multimachine jobs), but they are design decisions and need future fine tuning
Updated by coolo over 5 years ago
- Target version changed from Current Sprint to Done