Project

General

Profile

Actions

action #129121

open

performance: various tests fail receiving data at usable speed over the internet

Added by dimstar 5 months ago. Updated 5 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Target version:
Start date:
2023-05-11
Due date:
% Done:

0%

Estimated time:
Tags:

Description

Observation

This is seen in multiple tests, where data from sites like github, or flathub, are being transferred at too low rate to finish the test successfully

e.g.

https://openqa.opensuse.org/tests/3282120#step/tvm/11
downloading data fro mgithubusercontent; transfer rates of ~ 110k

https://openqa.opensuse.org/tests/3282069#step/helm_K3S/20
downloading from github.com (according to later output in the tests, download succeeded after the timeout was reached)

https://openqa.opensuse.org/tests/3282034#step/podman_pods/138
downloading from github.com

https://openqa.opensuse.org/tests/3282068#step/flatpak/20
receiving data from flatub; if the grub call would just not match anything, we'd see a return value != 0 on the console, but the command simply does not return

The combination of issues seen does not indicate this being an actual product bug - but rather performance of o3 -> internet being quite poor

Actions #1

Updated by dimstar 5 months ago

quick / simple diagnostics:

on ariel:

:/tmp> wget https://media.githubusercontent.com/media/onnx/models/main/vision/classification/resnet/model/resnet50-v2-7.onnx
--2023-05-11 09:11:04-- https://media.githubusercontent.com/media/onnx/models/main/vision/classification/resnet/model/resnet50-v2-7.onnx

resnet50-v2-7.onnx 100%[=============================================================>] 97.70M 14.1MB/s in 8.0s

2023-05-11 09:11:12 (12.3 MB/s) - ‘resnet50-v2-7.onnx’ saved [102442452/102442452]

on openqaworker4:

/tmp # wget https://media.githubusercontent.com/media/onnx/models/main/vision/classification/resnet/model/resnet50-v2-7.onnx
--2023-05-11 11:11:47-- https://media.githubusercontent.com/media/onnx/models/main/vision/classification/resnet/model/resnet50-v2-7.onnx

resnet50-v2-7.onnx 100%[=============================================================>] 97.70M 5.59MB/s in 83s

2023-05-11 11:13:16 (1.18 MB/s) - ‘resnet50-v2-7.onnx’ saved [102442452/102442452]

the download on ow4 had a dip down to 200k at one point, then recovered again (total time 8s vs 80s)

Actions #2

Updated by maritawerner 5 months ago

  • Project changed from openQA Tests to openQA Infrastructure
  • Category deleted (Bugs in existing tests)

Hello Qe tools team! I had a short discussion with Dominik on Slack:
"as to 'who to look at it' - that's a very good question indeed; I'd expect this to be a joint task for QE-Tools together with infra"

Could you please have a look and if it makes sense file a SD Jira ticket for the Infra team? Since this happened on Thursday morning it is also possible that the usual maintenance window hit us here. Any other proposals?

Actions #3

Updated by okurz 5 months ago

  • Tags set to infra
  • Target version set to future

Well, what one could do is at least crosscheck with SUSE-IT what are expected worst case, best case bandwidths.

Actions

Also available in: Atom PDF