action #129487
Updated by okurz 11 months ago
## Motivation
OSD suffers from too many jobs uploading too much stuff in parallel (stuff could be "module results" or logs, I don't know). To prevent such situation overloading the instance so that also visitors of the webUI run into unresponsiveness we should limit the number of uploads but not invalidate job results or cause incomplete results, just serialize and queue the uploading.
## Acceptance criteria
* **AC1:** Based on configuration setting concurrent job upload handling can be limited
* **AC2:** By default no limit (otherwise we would need "user feedback")
## Suggestions
* Marius assumes that mojolicious does not support that so will be very hard but can we use a semaphore or lock or "free upload ticket count" using the database?
* The webUI could deny uploads if "upload ticket count" is exceeded, the worker understands that response, backs off and waits
* "upload ticket count" could for a start be "all jobs in the running and uploading states" but as necessary every job going to uploading could increment a counter
* The worker code *already* has "back off and retry later" code, e.g. when the webUI is rebooting+restarting, so we can just rely on the same and the webUI sends back the same, I dunno, 503 response?
## Further details
* Keep very much related #129619 in mind
## Out of scope
* Don't care about telling the worker something specific assuming it would automatically retry uploading anyway, same as when the webUI host reboots