action #32725
closed
coordination #32851: [tools][EPIC] Scheduling redesign
[tools] Scheduler job_grab/filter_jobs refactoring
Added by EDiGiacinto about 6 years ago.
Updated over 5 years ago.
Category:
Feature requests
Description
For the moment it would be sufficient to refactor job_grab, and pass over schemas objects instead of hashes and re-use them in the filtering phase, meanwhile refactoring properly those areas (e.g. extracting them) - better if we also optimize the queries.
Kinda of a hard task, since could potentially make things 'shaky'.
- Description updated (diff)
- Description updated (diff)
- Target version set to Ready
Taken that we usually have < 2000 jobs scheduled, I think it's worth an experiment to grab all open jobs and schedule in memory. There is only one scheduler process, so IMO this should simplify a lot. Yes, hard task - but worth the time spent to make the whole scheduling understandable.
- Description updated (diff)
- Parent task set to #32851
- Start date changed from 2018-03-02 to 2018-05-05
due to changes in a related task
- Target version changed from Ready to Current Sprint
- Assignee set to EDiGiacinto
I think there are still some rough edges here, right?
- Status changed from New to Feedback
- Assignee deleted (
EDiGiacinto)
Yes, but as i'm not the author of this set of changes, can't help much more from here - i adapted the tests we already had for the scheduler and did some adaptations to the issues i could notice, but i think there are still bugs to catch.
Latest related change to it is: https://github.com/os-autoinst/openQA/pull/1743
- Related to action #39560: Tests for blocked_by and loops inside of it added
- Related to action #39629: openQA Scheduler refactor fallout added
- Status changed from Feedback to Resolved
The scheduler was refactored - we have some new problems (like starving multimachine jobs), but they are design decisions and need future fine tuning
- Target version changed from Current Sprint to Done
Also available in: Atom
PDF