action #14926
closed
(missing docu) job data is cleaned up way too eagerly, e.g. for still open bugs
Added by okurz over 7 years ago.
Updated over 7 years ago.
Category:
Feature requests
Description
motivation¶
Take for example the bug bsc#990254 referencing an openQA job from July. The corresponding job run is https://openqa.suse.de/tests/482779 which is not available anymore. Unfortunately the job information page does not even tell if the link is invalid or has been valid but the job does not exist anymore. Even if the corresponding logfiles would be cleaned already the job would provide at least some information, at least it would show a link to "latest"
user story¶
As a user referencing job results externally (e.g. in bug reports) I want the linked jobs to stay around for way longer to not loose at least the job details
acceptance criteria¶
- AC1: A job used in bug reports is not cleaned as soon as unreferenced jobs. It's considered important
- AC2: Jobs just marked with a issue link are not considered "important"
- AC3: Jobs marked automatically by carry-over are not considered "important" (as there was no human action on this job and therefore less valuable)
- AC4: Deleted job pages show something more reasonable than a page which is just telling that the page does not exist
tasks¶
- Define a label to be used or a database entry, e.g.
label:linked
- On cleanup consider the jobs with
label:linked as important
- optional: Automatically add label based on if the page has been called or the buglink has been added there based on human intervention
- optional: redirect from job not found page to something more helpful, e.g. when the job page probably existed at a time but does not anymore, e.g. because the job number is smaller than the current existing jobs
original suggestion¶
- Do not delete jobs which have a bugref
2a. Do not completely delete old jobs
or
2b. show a custom details page for old jobs with at least some explanation of what is happening, e.g. link to latest
- 1. is unfeasible - too many jobs have a bug reference thanks to carry over. And unfortunately having a bug reference or not does not relate to the test linked somewhere
- 2a. you will need to specify what not to remove
- 2b. link to what latest? The only information left from a removed job is that it's not there
- Description updated (diff)
- Category changed from Regressions/Crashes to Feature requests
We discussed in a call some ideas:
- use labels, e.g.
label:linked for every job that was linked in a bug report and keep these jobs around longer as "important" ones, same as all jobs that belong to "important" builds.
- use automatic script based on apache logs or maybe from within openQA when a job page is called and it has a bug reference which has not been added by automatic means, consider this job as "important" by putting the automatic label in the comment.
I'll into this. I'm for tracking referral header instead of apache log parsing. How about adding label:linked for all jobs with referral header present? I still need to look into how apache proxy handles them though.
I have one issue with this though and that updating job label if referer is present violates REST rules. Get request should not modify data.
What are your opinions on this?
As long as you don't use the credentionals of the one GETing, I see no difference to the approach with parsing logs
As I am updating comment user queries, I'm wondering do we have something against creating system user?
Currently I'm taking same approach as in case of audit log where event owner db field is nullable and there are explicit checks for valid user. Comments does not have nullable user yet -> schema migration. And there are at least 5 occurrences of $user->... which needs explicit user checking.
Whereas if I check and create system user on webui startup I can use its id for comments instead of undef. Thus no need for migration and unnecessary code changes. This system user wont have proper openid URL thus there is no conflict with existing users. Also I can use fixed string for gravatar -> all openqa instances should have the same picture for admin user from gravatar.
I'm inclined to add this user. WDYT?
- Status changed from New to In Progress
PR with job labeling and expiration merged PR#1060. What is missing is the flagging of label carry-over and considering all labeled jobs important
updating the documentation I'd say
we agreed in the call that @szarate will provide his enhancements to the documentation until start of next week. Next step is then to add some admin document also explaining what is happening here.
In the meantime, so this doesn't gets blocked, it can be a separate document. Until a proper adminguide is written (Or simply add this as part of it, as the first topic, then we could move stuff like Installing.asciidoc#other-database-engines there (If it makes sense also)
@szarate don't try to get out of there. I deliberately mentioned that we are blocked by you to give some additional motivation ;-P
@okurz good one!, Will give some love to that then :)
- Subject changed from job data is cleaned up way too eagerly, e.g. for still open bugs to (missing docu) job data is cleaned up way too eagerly, e.g. for still open bugs
- Priority changed from Urgent to Normal
As the feature itself is done and merged, I move down the priority
- Target version set to Milestone 5
- Related to action #12266: keep tests with any comment around later added
- Status changed from In Progress to Resolved
Also available in: Atom
PDF