Project

General

Profile

Actions

action #64776

closed

[cache][worker] cache service suddenly stopped to download assets, all subsequent jobs needing download incomplete auto_review:"setup failure: Cache service status error: Premature connection close":retry

Added by okurz over 4 years ago. Updated over 4 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Regressions/Crashes
Target version:
Start date:
2020-03-24
Due date:
% Done:

0%

Estimated time:

Description

Observation

Observed on openqaworker6.s.d as reported in #64752 (ticket lost due to db recovery) the cache service out of a sudden failed to download any new requested assets. Jobs that rely on assets still existing in the cache are still running fine, any jobs that trigger a request for download any new asset incomplete. There are no obvious mentions in the log files what went wrong.

Acceptance criteria

  • AC1: There is more detail than just "Failed to download", accessible to users

Suggestions

  • Investigate if more details could be provided and forwarded to the details provided and logs and also the reason that is put on the job details web page.
  • If possible, prevent the situation that was observed in #64752

Workaround

In #64752 it helped to restart the cache service and cache service minion

Further details

Original subject content: just "Failed to download", no further reason.


Related issues 3 (0 open3 closed)

Related to openQA Project (public) - action #65705: Tests can hang when waiting for a cache service requestResolvedokurz2020-04-16

Actions
Related to openQA Project (public) - action #66988: Mojo::Reactor::Poll: I/O watcher failed: Can't use an undefined value as a HASH reference at /usr/share/openqa/script/../lib/OpenQA/CacheService/Response/Status.pm line 20.Resolvedkraih2020-05-18

Actions
Related to openQA Project (public) - action #69178: workaround for #64776 using https://github.com/os-autoinst/scripts/blob/master/openqa-label-known-issuesResolvedokurz2020-07-21

Actions
Actions

Also available in: Atom PDF