Project

General

Profile

Actions

action #174304

open

Slow network connection between PowerVM/HMC SUT and OpenQA controller host size:S

Added by MDoucha about 2 months ago. Updated 2 days ago.

Status:
Workable
Priority:
Normal
Assignee:
Category:
Support
Start date:
2024-12-12
Due date:
% Done:

0%

Estimated time:
Tags:

Description

Some LTP tests trigger large amounts of kernel log messages and then upload journalctl logfile to OpenQA. In this example, shutdown_ltp-journalctl.txt log file contains 18MB of data and the upload took 111 seconds (upload speed: ~168KB/s, upload timeout: 100s).
https://openqa.suse.de/tests/16073659#step/shutdown_ltp/55

The PowerVM SUT (upload source) is nessberry-2, the OpenQA controller host (upload destination) is worker29.oqa.prg2.suse.org. What is the expected bandwidth between the machines?

Acceptance criteria

  • AC1: A sufficiently high bandwidth is ensured between usual OSD involved testing machines, e.g. ~1GBit/s

Suggestions

  • Follow and respond on https://sd.suse.com/servicedesk/customer/portal/1/SD-175581
  • Check bandwidth between various hosts, e.g. with iperf, and comment back in the SD ticket
    • And use -4 and -6 to check IPv4 vs. IPv6
  • Depending on how the problem can be reproduced, e.g. which hosts are affected, investigate/mitigate/solve the underlying issue, e.g. anything in the range from "just trigger a reboot to the HMC or switch" to work in a task force with IT to identify misbehaving systems severly impacting the available bandwidth
  • Also be aware about #133700 and shove it in the face of IT
  • Don't just bump timeouts here
Actions

Also available in: Atom PDF