action #109190
closedInvalid reusage of VLAN-Tag in multi-machine scenario, when using job posts
0%
Description
Observation¶
- The job http://openqa-3.wicked.suse.de/tests/80283#step/before_test/23 show a
eth0: IPv4 duplicate address 10.0.2.11 detected (in use by 52:54:00:12:00:56)!
message - The mac 52:54:00:12:00:56 belongs to http://openqa-3.wicked.suse.de/admin/workers/86 which runs the job http://openqa-3.wicked.suse.de/tests/80207 during that time.
The job 80207 is a multi machine job and the parent is http://openqa-3.wicked.suse.de/tests/80206, which start at 2022-03-29T09:27:13.195681+02:00
and end at 2022-03-29T10:04:05.167885+02:00
, while job 80207 ends at [2022-03-29T10:06:21.469947+02:00]
.
The failing job show the qemu command at: [2022-03-29T10:04:26.913030+02:00] [debug] starting: /usr/bin/qemu-system-x86_64 -vga cirrus -only-migratable...
, thus
it start a qemu instance with the same VLAN which is still used by job 80207.
Simple reproducer, create two parallel boot jobs
id=$(openqa-cli api --host http://openqa-3.wicked.suse.de -X POST jobs 'ARCH=x86_64' 'DISTRI=opensuse' 'FLAVOR=CI' 'MACHINE=x86_64' 'VERSION=Tumbleweed' '_GROUP_ID=0' \
'BOOT_HDD_IMAGE=1' 'DESKTOP=textmode' 'HDD_1=tumbleweed.qcow2' 'KEEP_GRUB_TIMEOUT=1' \
'BACKEND=qemu' 'NICTYPE=tap' 'WORKER_CLASS=tap,qemu_x86_64' \
'SCHEDULE=tests/boot/boot_to_desktop' 'TEST=check_vlan_on_mm_job_parent' | jq -r '.id')
echo "PARENT_ID:$id"
openqa-cli api --host http://openqa-3.wicked.suse.de -X POST jobs 'ARCH=x86_64' 'DISTRI=opensuse' 'FLAVOR=CI' 'MACHINE=x86_64' 'VERSION=Tumbleweed' '_GROUP_ID=0' \
'BOOT_HDD_IMAGE=1' 'DESKTOP=textmode' 'HDD_1=tumbleweed.qcow2' 'KEEP_GRUB_TIMEOUT=1' \
'BACKEND=qemu' 'NICTYPE=tap' 'WORKER_CLASS=tap,qemu_x86_64' \
'SCHEDULE=tests/boot/boot_to_desktop' 'TEST=check_vlan_on_mm_job_child' \
"_PARALLEL_JOBS=$id"
Updated by cfconrad almost 3 years ago
- Project changed from openQA Tests (public) to openQA Project (public)
- Category deleted (
Bugs in existing tests)
Updated by okurz almost 3 years ago
- Category set to Regressions/Crashes
- Target version set to future
Updated by cfconrad about 1 month ago
- Status changed from New to Closed
Not seen for quite some time.
Also changed the trigger to use the VAR_NAME:XX=value
notation. With this, it is actually possible to post two or more jobs in parallel without to hussle with the parent id.
Example:
openqa-cli api --host http://openqa.wicked.suse.de -X POST jobs 'ARCH:1=x86_64' 'ARCH:2=x86_64' 'BACKEND:1=qemu' 'BACKEND:2=qemu' 'BOOT_HDD_IMAGE:1=1' 'BOOT_HDD_IMAGE:2=1' 'BUILD:1=wicked-migrate' 'BUILD:2=wicked-migrate' 'CASEDIR:1=https://github.com/cfconrad/os-autoinst-distri-opensuse#poc_wicked_migration-1' 'CASEDIR:2=https://github.com/cfconrad/os-autoinst-distri-opensuse#poc_wicked_migration-1' 'DESKTOP:1=textmode' 'DESKTOP:2=textmode' 'DISTRI:1=opensuse' 'DISTRI:2=opensuse' 'EXTRATEST:1=wicked' 'EXTRATEST:2=wicked' 'FLAVOR:1=CI' 'FLAVOR:2=CI' 'HDDSIZEGB:1=30' 'HDDSIZEGB:2=30' 'HDD_1:1=tumbleweed.qcow2' 'HDD_1:2=tumbleweed.qcow2' 'IS_WICKED_REF:1=1' 'KEEP_GRUB_TIMEOUT:1=1' 'KEEP_GRUB_TIMEOUT:2=1' 'MACHINE:1=x86_64' 'MACHINE:2=x86_64' 'NICTYPE:1=tap' 'NICTYPE:2=tap' 'NOIMAGES:1=1' 'NOIMAGES:2=1' 'NOVIDEO:1=1' 'NOVIDEO:2=1' 'OVS_DEBUG:1=1' 'OVS_DEBUG:2=1' 'PARALLEL_WITH:2=nm_migrate_basic_ref' 'QEMUCPU:1=qemu64' 'QEMUCPU:2=qemu64' 'QEMURAM:2=2048' 'QEMUVGA:1=cirrus' 'QEMUVGA:2=cirrus' 'TAPDEV:1=auto,auto' 'TAPDEV:2=auto,auto' 'TEST:1=nm_migrate_basic_ref' 'TEST:2=nm_migrate_basic_sut' 'UEFI_PFLASH_VARS:1=tumbleweed-uefi-vars.qcow2' 'UEFI_PFLASH_VARS:2=tumbleweed-uefi-vars.qcow2' 'VERSION:1=Tumbleweed' 'VERSION:2=Tumbleweed' 'VIDEOMODE:1=text' 'VIDEOMODE:2=text' 'VIRTIO_CONSOLE:1=1' 'VIRTIO_CONSOLE:2=1' 'VIRTIO_CONSOLE_NUM:1=2' 'VIRTIO_CONSOLE_NUM:2=2' 'WICKED:1=nm_migrate_basic' 'WICKED:2=nm_migrate_basic' 'WICKED_CHECK_LOG_FAIL:1=1' 'WICKED_CHECK_LOG_FAIL:2=1' 'WICKED_NEED_NETWORK_TWEAKS:1=0' 'WICKED_NEED_NETWORK_TWEAKS:2=0' 'WICKED_NM_MIGRATE_CONTINUE_ON_FAILURE:1=1' 'WICKED_NM_MIGRATE_CONTINUE_ON_FAILURE:2=1' 'WORKER_CLASS:1=tap,qemu_x86_64,agabus' 'WORKER_CLASS:2=tap,qemu_x86_64,agabus' '_GROUP_ID:1=10' '_GROUP_ID:2=10' '_PARALLEL:2=1' '_QUIET_SCRIPT_CALLS:1=1' '_QUIET_SCRIPT_CALLS:2=1'