Project

General

Profile

Actions
Copy link

action #138302

closed

Ensure automated openQA tests verify that os-autoinst-setup-multi-machine sets up valid networking size:M

Added by livdywan about 1 year ago. Updated 11 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
dheidler
Category:
Feature requests
Target version:
Ready
Start date:
2023-07-19
Due date:
2024-01-19
% Done:

0%

Estimated time:

Description

Motivation

Thanks to #133025 we cover os-autoinst-setup-multi-machine in openQA tests. However currently those tests never verify the setup and would at best catch more trivial issues with running the script.

Acceptance criteria

  • AC1: Broken networking prepared by os-autoinst-setup-multi-machine is visible as a failed openQA test job

Suggestions

Related issues 6 (2 open — 4 closed)

Related to openQA Infrastructure (public) - action #152098: [research][timeboxed:10h] Learn more about openvswitch with experimenting together size:SWorkable2023-12-05

Actions
Related to openQA Tests (public) - action #153766: [core][sporadic] Handle wild agetty better in tests/network/setup_multimachine.pmNew2024-01-17

Actions
Related to openQA Project (public) - action #155170: [openqa-in-openqa] [sporadic] test fails in test_running: parallel_failed size:MResolvedybonatakis2024-02-082024-02-29

Actions
Related to openQA Project (public) - action #155173: [openqa-in-openqa] [sporadic] test fails in openqa_worker: os-autoinst-setup-multi-machine timed out size:MResolvedmkittler2024-02-082024-03-01

Actions
Blocked by openQA Project (public) - action #153616: Prevent `duplicate key value violates unique constraint` on image uploads size:SResolvedmkittler2024-01-16

Actions
Copied from openQA Project (public) - action #133025: Configure Virtual Interfaces instructions do not work on Leap 15.5 size:MResolveddheidler2023-07-192023-10-31

Actions
Actions
Copy link
 #1

Updated by livdywan about 1 year ago

  • Copied from action #133025: Configure Virtual Interfaces instructions do not work on Leap 15.5 size:M added
Actions
Copy link
 #2

Updated by dheidler about 1 year ago

Followup on #133025
As of now the openQA-in-openQA test calls the os-autoinst-setup-multi-machine script,
but only checks that it runs through without failure.
Still afterwards only a single test is executed.

The testsuite should be extended to run a real multi machine job to verify that the
setup created by the script works.

Actions
Copy link
 #3

Updated by okurz about 1 year ago

  • Due date deleted (2023-10-31)
Actions
Copy link
 #4

Updated by okurz about 1 year ago

  • Subject changed from Ensure automated openQA tests verify that os-autoinst-setup-multi-machine sets up valid networking size:M to Ensure automated openQA tests verify that os-autoinst-setup-multi-machine sets up valid networking
Actions
Copy link
 #5

Updated by okurz about 1 year ago

  • Target version changed from Ready to Tools - Next
Actions
Copy link
 #6

Updated by okurz about 1 year ago

  • Target version changed from Tools - Next to Ready
Actions
Copy link
 #7

Updated by dheidler about 1 year ago

  • Subject changed from Ensure automated openQA tests verify that os-autoinst-setup-multi-machine sets up valid networking to Ensure automated openQA tests verify that os-autoinst-setup-multi-machine sets up valid networking size:M
  • Description updated (diff)
  • Status changed from New to Workable
Actions
Copy link
 #8

Updated by dheidler about 1 year ago

  • Status changed from Workable to In Progress
  • Assignee set to dheidler
Actions
Copy link
 #9

Updated by openqa_review about 1 year ago

  • Due date set to 2023-12-16

Setting due date based on mean cycle time of SUSE QE Tools

Actions
Copy link
 #10

Updated by okurz about 1 year ago

  • Due date deleted (2023-12-16)
  • Status changed from In Progress to Workable

actually wasn't started yet

Actions
Copy link
 #11

Updated by dheidler 12 months ago

  • Status changed from Workable to In Progress
Actions
Copy link
 #12

Updated by openqa_review 12 months ago

  • Due date set to 2024-01-04

Setting due date based on mean cycle time of SUSE QE Tools

Actions
Copy link
 #13

Updated by livdywan 12 months ago

  • Related to action #152098: [research][timeboxed:10h] Learn more about openvswitch with experimenting together size:S added
Actions
Copy link
 #15

Updated by okurz 12 months ago

  • Due date changed from 2024-01-04 to 2024-01-11

As discussed today:
So my full suggestion: Ensure that "full_run" from https://github.com/os-autoinst/os-autoinst-distri-openQA/blob/master/tests/osautoinst/start_test.pm#L33 is actually executed as part of openQA-in-openQA tests, extend that test code to clone the "wireguard_client/server" scenario and just see what happens

Actions
Copy link
 #17

Updated by okurz 11 months ago

As you uncover multiple issues you found I suggest you focus on just getting a test running even with ugly workarounds. Fixing each and every issue can be delayed to other follow-up issues.

Actions
Copy link
 #19

Updated by dheidler 11 months ago

I suspect that the remaining issue is due to insufficient ram to start two nested VMs at the same time.

Actions
Copy link
 #20

Updated by dheidler 11 months ago

The wireguard testsuite can't be used because it needs internet access to install wireguard packages.
Internet access from the inner SUTs is not possible due to conflicting IP addresses.
So creating a simple ping testsuite that doesn't need to install any packages.
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/18440

Actions
Copy link
 #22

Updated by okurz 11 months ago

  • Due date changed from 2024-01-11 to 2024-01-19

As discussed we decided it's ok to go ahead with bumped due date.

Actions
Copy link
 #23

Updated by dheidler 11 months ago

  • Blocked by action #153616: Prevent `duplicate key value violates unique constraint` on image uploads size:S added
Actions
Copy link
 #24

Updated by livdywan 11 months ago

  • Status changed from In Progress to Blocked

For clarity, the related ticket is being worked on now as it makes this test unreliable.

Actions
Copy link
 #25

Updated by dheidler 11 months ago

  • Status changed from Blocked to Feedback
Actions
Copy link
 #26

Updated by okurz 11 months ago

  • Status changed from Feedback to In Progress

As discussed handle the test instability either by fixing or reporting in a corresponding ticket.

Actions
Copy link
 #27

Updated by dheidler 11 months ago

  • Related to action #153766: [core][sporadic] Handle wild agetty better in tests/network/setup_multimachine.pm added
Actions
Copy link
 #29

Updated by dheidler 11 months ago

  • Status changed from In Progress to Resolved
Actions
Copy link
 #30

Updated by okurz 10 months ago

  • Related to action #155170: [openqa-in-openqa] [sporadic] test fails in test_running: parallel_failed size:M added
Actions
Copy link
 #31

Updated by okurz 10 months ago

  • Related to action #155173: [openqa-in-openqa] [sporadic] test fails in openqa_worker: os-autoinst-setup-multi-machine timed out size:M added
Actions
Copy link

Also available in: Atom PDF