Project

General

Profile

Actions

action #122746

closed

Publishing opensuse.openqa.job.done failed: SSL connect attempt failed

Added by livdywan over 1 year ago. Updated over 1 year ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
-
Target version:
Start date:
2023-01-05
Due date:
% Done:

0%

Estimated time:

Description

Observation

openQA logreport says this:

[2023-01-05T10:16:48.567085Z] [error] [pid:4163] Publishing opensuse.openqa.job.done failed: SSL connect attempt failed error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed
 (9 attempts left)
[...]
[2023-01-05T10:22:03.396156Z] [error] [pid:536] Publishing opensuse.openqa.job.done failed: SSL connect attempt failed error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed
 (9 attempts left)
[2023-01-05T10:22:03.396807Z] [error] [pid:32026] Publishing opensuse.openqa.comment.create failed: SSL connect attempt failed error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed
 (5 attempts left)
[2023-01-05T10:22:04.231239Z] [error] [pid:29209] Publishing opensuse.openqa.job.done failed: SSL connect attempt failed error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed
 (2 attempts left)

Acceptance criteeria

  • AC1: No new TLS connection errors in logs

Suggestions

  • Check if this is a temporary problem caused by server maintenance
Actions #1

Updated by jbaier_cz over 1 year ago

This was likely caused by the OBS major outage today, see https://suse.slack.com/archives/C02BX1X92HM/p1672910988920159

Actions #2

Updated by livdywan over 1 year ago

  • Target version set to Ready
Actions #3

Updated by kraih over 1 year ago

  • Status changed from New to In Progress
  • Assignee set to kraih
Actions #4

Updated by kraih over 1 year ago

  • Tags set to reactive work
Actions #5

Updated by kraih over 1 year ago

  • Status changed from In Progress to Feedback

The error messages came from Mojo::RabbitMQ::Client. Looks like the RabbitMQ machine was updated after the maintenance window, together with other OBS related infrastructure, as Jan suspected. Last error was at 10:22:04:

[2023-01-05T10:22:04.231239Z] [error] [pid:29209] Publishing opensuse.openqa.job.done failed: SSL connect attempt failed error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed
 (2 attempts left)

The error message itself is appropriate for what happened, so i don't think there are any changes necessary. But while investigating this i noticed that we are leaking RabbitMQ credentials in debug log messages, that should not be happening. So i'm opening a ticket for that.

Actions #6

Updated by kraih over 1 year ago

Followup ticket for the RabbitMQ credentials: #122749

Actions #7

Updated by kraih over 1 year ago

  • Description updated (diff)
Actions #8

Updated by kraih over 1 year ago

  • Status changed from Feedback to Resolved
Actions

Also available in: Atom PDF