action #73273
coordination #39719: [saga][epic] Detect "known failures" and mark jobs as such to make tests more stable, reviewing test results and tracking known issues easier
coordination #62420: [epic] Distinguish all types of incompletes
job incompletes with auto_review:"setup failure: Cache service status error from API.* file is not a database .*":retry
Start date:
2020-10-13
Due date:
% Done:
0%
Estimated time:
Difficulty:
Description
Observation¶
job https://openqa.suse.de/tests/4812517 is incomplete, the log shows:
[2020-10-13T04:14:16.0500 CEST] [debug] [pid:36910] Updating status so job 4812517 is not considered dead. [2020-10-13T04:14:16.0501 CEST] [debug] [pid:36910] REST-API call: POST http://openqa.suse.de/api/v1/jobs/4812517/status [2020-10-13T04:14:21.0568 CEST] [debug] [pid:36910] Updating status so job 4812517 is not considered dead. [2020-10-13T04:14:21.0569 CEST] [debug] [pid:36910] REST-API call: POST http://openqa.suse.de/api/v1/jobs/4812517/status [2020-10-13T04:14:21.0630 CEST] [error] [pid:36910] Unable to setup job 4812517: Cache service status error from API: Minion job #43206 failed: DBD::SQLite::db do failed: file is not a database at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/SQLite.pm line 85.
please see more details in https://openqa.suse.de/tests/4812517/file/worker-log.txt
Acceptance criteria¶
- AC1: The issue is automatically handled or the test maintainer is informed what he or she did wrong
Suggestions¶
- Try to handle this problem with an automatic recovery within openQA if this is not a problem introduced by test maintainers (unlikely)
Workaround¶
Delete cache database on the affected worker and retry
Related issues
History
Updated by #2
Updated by Xiaojing_liu 6 months ago
Updated by - Subject changed from job incompletes with auto_review:"setup failure: Cache service status error from API: Minion job #43206 failed: DBD::SQLite::db do failed: file is not a database at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/SQLite.pm line 85." to job incompletes with auto_review:"setup failure: Cache service status error from API: Minion job.* failed: DBD::SQLite::db do failed: file is not a database .*"
#3
Updated by Xiaojing_liu 6 months ago
- Subject changed from job incompletes with auto_review:"setup failure: Cache service status error from API: Minion job.* failed: DBD::SQLite::db do failed: file is not a database .*" to job incompletes with auto_review:"setup failure: Cache service status error from API.* file is not a database .*"
#4
Updated by okurz 6 months ago
- Related to action #67000: Job incompletes due to malformed worker cache database disk image with auto_review:"Cache service status error.*(database disk image is malformed|Specified job ID is invalid).*":retry added
#5
Updated by okurz 6 months ago
- Subject changed from job incompletes with auto_review:"setup failure: Cache service status error from API.* file is not a database .*" to job incompletes with auto_review:"setup failure: Cache service status error from API.* file is not a database .*":retry
- Description updated (diff)
- Target version changed from Ready to future
#7
Updated by okurz 4 months ago
The underlying problem was introduced by #67000 which we consider solved. We should still have this feature request in mind to improve the situation if the worker cache fails in a similar condition, e.g. automatically retrigger or "un-assign" the job from the worker before it incompletes