Project

General

Profile

Actions
Copy link

action #180098

closed

Multi-Machine tests failing on network access

Added by dimstar about 2 months ago. Updated about 2 months ago.

Status:
Resolved
Priority:
High
Assignee:
mkittler
Category:
Regressions/Crashes
Target version:
Ready
Start date:
2025-04-07
Due date:
% Done:

0%

Estimated time:
Tags:
reactive work

Description

Observation

This is being observed for the last 2 days - all MM machines seem to fail to access network resources

openQA test in scenario microos-Tumbleweed-DVD-x86_64-remote_ssh_target@64bit-2G fails in
networking

Test suite description

Maintainer: jrivera Boot with ssh=1 parameter and wait for parallel job (remote_ssh_controller) to install the system.

Reproducible

Fails since (at least) Build 20250404

Expected result

Last good: 20250403 (or more recent)

Further details

Always latest result in this scenario: latest

Files

clipboard-202504070925-nqf7b.png (193 KB) clipboard-202504070925-nqf7b.png dimstar, 2025-04-07 07:25
clipboard-202504070925-nqf7b.png
Actions
Copy link
 #1

Updated by dimstar about 2 months ago

  • Priority changed from Urgent to Immediate

Raising - this blocks all MM tests completely from what we could see

Actions
Copy link
 #2

Updated by livdywan about 2 months ago

Hum. I was just taking a look, the error seems clear:

curl: (6) Could not resolve host: openqa.opensuse.org

However just now there's a passing test: https://openqa.opensuse.org/tests/4976935

Actions
Copy link
 #4

Updated by okurz about 2 months ago

  • Tags set to reactive work
  • Project changed from openQA Tests (public) to openQA Project (public)
  • Category changed from Bugs in existing tests to Regressions/Crashes
  • Priority changed from Immediate to Urgent
  • Target version set to Ready

According to rfan1 and dzedro in https://suse.slack.com/archives/C02CANHLANP/p1744022074232239?thread_ts=1744022074.232239&cid=C02CANHLANP no relevant changes from core or osado side so likely something regressed in the o3 infra. Please also see changes applied by @favogt as mitigation.

Actions
Copy link
 #5

Updated by livdywan about 2 months ago

  • Status changed from New to In Progress
  • Assignee set to livdywan

So I'm not clear on what this is about, but taking the ticket anyway since it's urgent to figure what needs to happen.

Actions
Copy link
 #6

Updated by mkittler about 2 months ago

  • Assignee changed from livdywan to mkittler

I'm busy but with almost the same type of work so I'm having a look regarding this forwarding setting on o3 workers.

Actions
Copy link
 #7

Updated by mkittler about 2 months ago ยท Edited

Note that we enable ip forwarding in os-autoinst/script/os-autoinst-setup-multi-machine. The setting is nevertheless only present on a few hosts. So I re-executed this line on all hosts mentioned on https://progress.opensuse.org/projects/openqav3/wiki/#Manual-command-execution-on-o3-workers - just to be sure. So we should now have forwarding enabled on all hosts in a persistent way. Considering our setup script already does this I don't see anything to improve upstream. I have no idea how this was working before.

Note that I haven't seen the "one host only" setting on o3 workers (except on the power pc worker). I do remember configuring this. Either I misremember or this configuration was lost. The latter would explain how this was working before but it is probably the former.

Actions
Copy link
 #8

Updated by mkittler about 2 months ago

  • Status changed from In Progress to Feedback
  • Priority changed from Urgent to High

I'm lowering the prio due to what I wrote in my last comment. In my opinion we can also resolve this ticket.

Actions
Copy link
 #9

Updated by mkittler about 2 months ago

  • Status changed from Feedback to Resolved

There haven't been further problems with MM tests reported so I'm considering this resolved.

Actions
Copy link

Also available in: Atom PDF