action #40973
closed
worker retry seems to be bogus
Added by coolo over 6 years ago.
Updated over 6 years ago.
Description
We had a problem on the openqa webui this morning, so the workers reporting upload progress
went into a 500, but what happened was a mass storm of api calls.
Looking into the worker log, I see
Sep 13 11:20:21 openqaworker13 worker[15148]: [error] 500 response: Internal Server Error (remaining tries: -9)
Sep 13 11:20:21 openqaworker13 worker[15148]: [error] 500 response: Internal Server Error (remaining tries: -11)
Sep 13 11:20:21 openqaworker13 worker[15148]: [error] 500 response: Internal Server Error (remaining tries: -366)
Sep 13 11:20:21 openqaworker13 worker[15148]: [error] 500 response: Internal Server Error (remaining tries: -398)
Sep 13 11:20:21 openqaworker13 worker[15148]: [error] 500 response: Internal Server Error (remaining tries: -420)
Sep 13 11:20:21 openqaworker13 worker[15148]: [error] 500 response: Internal Server Error (remaining tries: -398)
Sep 13 11:20:21 openqaworker13 worker[15148]: [error] 503 response: Service Unavailable (remaining tries: 0)
Sep 13 11:20:21 openqaworker13 worker[15148]: [error] Job aborted because web UI doesn't accept updates anymore (likely considers this job dead)
this is how it all starts:
Sep 13 08:39:52 openqaworker13 worker[15163]: [info] 17581: WORKING 2041202
Sep 13 08:40:02 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: 2)
Sep 13 08:40:07 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: 1)
Sep 13 08:40:12 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: 0)
Sep 13 08:40:17 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -1)
Sep 13 08:40:22 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -2)
Sep 13 08:40:27 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -3)
Sep 13 08:40:32 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -4)
Sep 13 08:40:37 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -5)
Sep 13 08:40:42 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -6)
Sep 13 08:40:47 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -7)
Sep 13 08:40:52 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -8)
Sep 13 08:40:57 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -9)
Sep 13 08:41:02 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -10)
Sep 13 08:41:07 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -11)
Sep 13 08:41:12 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -12)
Sep 13 08:41:17 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -13)
Sep 13 08:41:22 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -14)
Sep 13 08:41:27 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -15)
Sep 13 08:41:32 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -16)
Sep 13 08:41:37 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -17)
Sep 13 08:41:42 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -18)
Sep 13 08:41:47 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -19)
Sep 13 08:41:52 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: 2)
Sep 13 08:41:52 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -20)
Sep 13 08:41:52 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: 2)
Sep 13 08:41:57 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: 1)
Sep 13 08:41:57 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -21)
Sep 13 08:41:57 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: 1)
Sep 13 08:42:02 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: 2)
Sep 13 08:42:02 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: 0)
Sep 13 08:42:02 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -22)
Sep 13 08:42:02 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: 0)
Sep 13 08:42:07 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: 1)
Sep 13 08:42:07 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -1)
Sep 13 08:42:07 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -23)
Sep 13 08:42:07 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -1)
Sep 13 08:42:12 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: 0)
Sep 13 08:42:12 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -2)
Sep 13 08:42:12 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -24)
Sep 13 08:42:12 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -2)
Sep 13 08:42:17 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -1)
Sep 13 08:42:17 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -3)
Sep 13 08:42:17 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -25)
Sep 13 08:42:17 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -3)
Sep 13 08:42:23 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -2)
Sep 13 08:42:23 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -4)
Sep 13 08:42:23 openqaworker13 worker[15163]: [error] 500 response: Internal Server Error (remaining tries: -26)
- Status changed from New to Resolved
- Assignee set to mkittler
- Target version set to Done
Also available in: Atom
PDF