action #14926
closed(missing docu) job data is cleaned up way too eagerly, e.g. for still open bugs
0%
Description
motivation¶
Take for example the bug bsc#990254 referencing an openQA job from July. The corresponding job run is https://openqa.suse.de/tests/482779 which is not available anymore. Unfortunately the job information page does not even tell if the link is invalid or has been valid but the job does not exist anymore. Even if the corresponding logfiles would be cleaned already the job would provide at least some information, at least it would show a link to "latest"
user story¶
As a user referencing job results externally (e.g. in bug reports) I want the linked jobs to stay around for way longer to not loose at least the job details
acceptance criteria¶
- AC1: A job used in bug reports is not cleaned as soon as unreferenced jobs. It's considered important
- AC2: Jobs just marked with a issue link are not considered "important"
- AC3: Jobs marked automatically by carry-over are not considered "important" (as there was no human action on this job and therefore less valuable)
- AC4: Deleted job pages show something more reasonable than a page which is just telling that the page does not exist
tasks¶
- Define a label to be used or a database entry, e.g.
label:linked - On cleanup consider the jobs with
label:linkedas important - optional: Automatically add label based on if the page has been called or the buglink has been added there based on human intervention
- optional: redirect from job not found page to something more helpful, e.g. when the job page probably existed at a time but does not anymore, e.g. because the job number is smaller than the current existing jobs
original suggestion¶
- Do not delete jobs which have a bugref 2a. Do not completely delete old jobs or 2b. show a custom details page for old jobs with at least some explanation of what is happening, e.g. link to latest
Updated by coolo over 7 years ago
- 1. is unfeasible - too many jobs have a bug reference thanks to carry over. And unfortunately having a bug reference or not does not relate to the test linked somewhere
- 2a. you will need to specify what not to remove
- 2b. link to what latest? The only information left from a removed job is that it's not there
Updated by okurz over 7 years ago
- Description updated (diff)
- Category changed from Regressions/Crashes to Feature requests
We discussed in a call some ideas:
- use labels, e.g.
label:linkedfor every job that was linked in a bug report and keep these jobs around longer as "important" ones, same as all jobs that belong to "important" builds. - use automatic script based on apache logs or maybe from within openQA when a job page is called and it has a bug reference which has not been added by automatic means, consider this job as "important" by putting the automatic label in the comment.
Updated by oholecek over 7 years ago
- Assignee set to oholecek
I'll into this. I'm for tracking referral header instead of apache log parsing. How about adding label:linked for all jobs with referral header present? I still need to look into how apache proxy handles them though.
Updated by oholecek over 7 years ago
I have one issue with this though and that updating job label if referer is present violates REST rules. Get request should not modify data.
What are your opinions on this?
Updated by coolo over 7 years ago
As long as you don't use the credentionals of the one GETing, I see no difference to the approach with parsing logs
Updated by oholecek over 7 years ago
As I am updating comment user queries, I'm wondering do we have something against creating system user?
Currently I'm taking same approach as in case of audit log where event owner db field is nullable and there are explicit checks for valid user. Comments does not have nullable user yet -> schema migration. And there are at least 5 occurrences of $user->... which needs explicit user checking.
Whereas if I check and create system user on webui startup I can use its id for comments instead of undef. Thus no need for migration and unnecessary code changes. This system user wont have proper openid URL thus there is no conflict with existing users. Also I can use fixed string for gravatar -> all openqa instances should have the same picture for admin user from gravatar.
I'm inclined to add this user. WDYT?
Updated by oholecek over 7 years ago
- Status changed from New to In Progress
PR with job labeling and expiration merged PR#1060. What is missing is the flagging of label carry-over and considering all labeled jobs important
Updated by okurz over 7 years ago
And some documentation should be added, at least on https://progress.opensuse.org/projects/openqav3/wiki if not directly within the documentation of openQA
Updated by okurz over 7 years ago
https://github.com/os-autoinst/openQA/pull/1071 changes the behaviour for labels. They are not carried over anymore. Please think hard if this is what we want.
Updated by okurz over 7 years ago
https://github.com/os-autoinst/openQA/pull/1071 merged, what do we need to do now?
Updated by okurz over 7 years ago
we agreed in the call that @szarate will provide his enhancements to the documentation until start of next week. Next step is then to add some admin document also explaining what is happening here.
Updated by szarate over 7 years ago
In the meantime, so this doesn't gets blocked, it can be a separate document. Until a proper adminguide is written (Or simply add this as part of it, as the first topic, then we could move stuff like Installing.asciidoc#other-database-engines there (If it makes sense also)
Updated by okurz over 7 years ago
@szarate don't try to get out of there. I deliberately mentioned that we are blocked by you to give some additional motivation ;-P
Updated by szarate over 7 years ago
@okurz good one!, Will give some love to that then :)
Updated by coolo over 7 years ago
- Subject changed from job data is cleaned up way too eagerly, e.g. for still open bugs to (missing docu) job data is cleaned up way too eagerly, e.g. for still open bugs
- Priority changed from Urgent to Normal
As the feature itself is done and merged, I move down the priority
Updated by okurz over 7 years ago
- Related to action #12266: keep tests with any comment around later added
Updated by oholecek about 7 years ago
- Status changed from In Progress to Resolved
Documentation added in https://github.com/os-autoinst/openQA/pull/1206