action #37592
closed[functional][y] Record soft-failure for detected errors in logs
0%
Description
Observation¶
openQA test in scenario sle-15-Installer-DVD-x86_64-allmodules+allpatterns@64bit shows a failed thumbnail in
logs_from_installation_system because the logfile shows errors in the yast logs. These are of minor severity for sure and in many cases are acceptable but we should still record soft-failures pointing to according bugs to remind developers about it.
Reproducible¶
Reproducible every time
Expected result¶
A failed thumbnail should prevent the job to turn out passed but record a soft-failure pointing to a specific bug
Suggestions¶
- Understand how currently the log file is parsed from within the SUT
- Ensure we have specific bugs for the individual issues, I guess we already have them reported
- Based on regex matches add record_soft_failure's to the individual bugs observed
Further details¶
Always latest result in this scenario: latest
Updated by riafarov over 5 years ago
- Status changed from Workable to New
Agreed to refine it, do not want to have all jobs soft-failing.
Updated by okurz over 5 years ago
that would probably make every job "soft-fail" and therefore defeat the purpose. I guess we can consider the following levels of severity and how we handle them:
- fail for new unknown errors that we do not know the criticality of and make the jobs fail
- soft-fail referencing existing issues that do not exist everywhere
- record_info referencing existing minor issues or issues that would popup as soft-fail in too many jobs
- ignore issues that would happen everywhere and no one will fix anyway
The option 1. might still happen too often and not provide enough value for us so maybe we would could turn that into "no red border around the thumbnail for unknown issues"?
Updated by riafarov over 5 years ago
- Due date deleted (
2018-11-20)
Let's discuss within the team. Currently YaST team gives us valuable hints what to look for in the logs and what is critical. The part we could do is to ensure that all error in the YaST logs are reported as bugs, and then after some time check what is not fixed and add soft-failures, if none were fixed - reconsider approach and communicate with YaST team.
Updated by okurz over 5 years ago
- Target version changed from Milestone 20 to Milestone 22
Updated by okurz over 5 years ago
- Due date set to 2019-02-12
pre-fill last sprint in M22 with all tickets within milestone not yet assigned to sprints
Updated by okurz over 5 years ago
- Blocked by action #39815: [functional][y] logs_from_installation_system always shows a red border but we do not track it as a bug added
Updated by okurz over 5 years ago
- Due date changed from 2019-02-12 to 2019-03-12
- Status changed from New to Blocked
- Assignee set to okurz
- Target version changed from Milestone 22 to Milestone 23
I suggest to look into #39815 first and then look back here.
Updated by okurz about 5 years ago
- Assignee changed from okurz to riafarov
@riafarov as discussed with you in person I think you can track it best
Updated by riafarov about 5 years ago
Actually is addressed in #39815 We have accepted current errors in order to detect new ones. We will communicate our findings to the YaST team, so we can clean up logs from things which are not errors.
Updated by riafarov about 5 years ago
- Status changed from Blocked to Resolved
As per last comment, this is now addressed and we now accept >80 known errors in the logs and soft-fail those.