Improve time to schedule a build
|Target version:||Current Sprint|
currently scheduling a SP1@x86_64 build takes > 180s, which was up to now our apache limit.
This really should be reduced a lot. Like all the cancelling/deprio logic could be moved to a minion job - just as the calculcation of blocked_by state I guess.
I'm only wondering why we do the blocked_by calculation currently twice. One time directly after creating a job via
create_from_settings and then again for each job after dealing with cycles, wrong parents, ...
If the last recalculation is required because it adds blocked-by IDs which couldn't be assigned in the first place, wouldn't that allow the scheduler to pick jobs accidentally (in the small period of time between the job creation and the final blocked-by calculation)? It that what you mean by touchy? With the right isolation level for the transaction that problem shouldn't occur, actually. Even if I remove the blocked-by calculation within
create_from_settings and only do it in the end. (Everything is done in one transaction.)
Yet another job state to be sure we don't schedule jobs in an inconsistent state would make sense in general. Maybe
ADDED? Or we just disable the scheduler (somehow) while job creation via posting an ISO is ongoing?
Marius and me discussed this in detail - and I propose to make 'Scheduled Product' a high level DBIx class. And we create a new API that creates that and schedule a minion task to do the actual scheduling. So you can poll the status of the Scheduled Product - and it can be 'scheduling' or 'scheduled' and if it's scheduled you can query the errors it created.
This would make actually a nice addition as currently the scheduled products are extracted from audit log and the errors just appear in some log file. Plus it solves the timeout problem - clients simply poll if they care (or use the old API :)