Actions
action #67870
closed[u] test fails in bootloader_svirt - virsh define failed for domain: openqa-SUT-2
Added by szarate over 4 years ago. Updated over 4 years ago.
Start date:
2020-06-09
Due date:
% Done:
0%
Estimated time:
Difficulty:
Description
Observation¶
openQA test in scenario sle-15-SP2-Online-x86_64-allpatterns@svirt-xen-hvm fails in
bootloader_svirt
Test suite description¶
Maintainers: okurz Installation with all patterns selected for installation to check for potential package conflicts, how the system handles big space usage, etc.
allpatterns installations can take longer, especially on non-x86_64 architectures.
Reproducible¶
Fails since (at least) Build 207.1 (current job)
Expected result¶
Last good: 205.1 (or more recent)
Further details¶
Always latest result in this scenario: latest
Files
| libxl-driver.log (207 KB) libxl-driver.log | szarate, 2020-06-09 08:40 |
Updated by szarate over 4 years ago
https://gitlab.suse.de/snippets/658
2020-06-09 08:34:02.833+0000: libxl: libxl_device.c:398:libxl__device_disk_set_backend: Disk vdev=xvda spec.backend=qdisk
2020-06-09 08:34:02.833+0000: libxl: libxl_device.c:347:disk_try_backend: Disk vdev=xvda, uses script=(null) on qdisk backend
2020-06-09 08:34:02.833+0000: libxl: libxl_device.c:398:libxl__device_disk_set_backend: Disk vdev=xvdb spec.backend=qdisk
2020-06-09 08:34:02.833+0000: libxl: libxl_device.c:347:disk_try_backend: Disk vdev=xvdb, uses script=(null) on qdisk backend
2020-06-09 08:34:02.833+0000: libxl: libxl_create.c:1017:initiate_domain_create: Domain 1129:running bootloader
2020-06-09 08:34:02.833+0000: libxl: libxl_bootloader.c:335:libxl__bootloader_run: Domain 1129:no bootloader configured, using user supplied kernel
2020-06-09 08:34:02.833+0000: libxl: libxl_event.c:686:libxl__ev_xswatch_deregister: watch w=0x7f1a600029a0: deregister unregistered
2020-06-09 08:34:02.835+0000: libxl: libxl_numa.c:502:libxl__get_numa_candidate: New best NUMA placement candidate found: nr_nodes=1, nr_cpus=16, nr_vcpus=36, free_memkb=2529
2020-06-09 08:34:02.835+0000: libxl: libxl_numa.c:502:libxl__get_numa_candidate: New best NUMA placement candidate found: nr_nodes=1, nr_cpus=16, nr_vcpus=35, free_memkb=2605
2020-06-09 08:34:02.836+0000: libxl: libxl_dom.c:204:numa_place_domain: NUMA placement candidate with 1 nodes, 16 cpus and 2605 KB free selected
2020-06-09 08:34:02.836+0000: domainbuilder: xc_dom_allocate: cmdline="", features=""
2020-06-09 08:34:02.836+0000: libxl: libxl_dom.c:784:libxl__build_pv: pv kernel mapped 0 path /usr/lib/grub2/x86_64-xen/grub.xen
2020-06-09 08:34:02.836+0000: domainbuilder: xc_dom_kernel_file: filename="/usr/lib/grub2/x86_64-xen/grub.xen"
2020-06-09 08:34:02.836+0000: domainbuilder: xc_dom_malloc_filemap : 2165 kB
2020-06-09 08:34:02.836+0000: domainbuilder: xc_dom_boot_xen_init: ver 4.11, caps xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
2020-06-09 08:34:02.836+0000: domainbuilder: xc_dom_parse_image: called
2020-06-09 08:34:02.836+0000: domainbuilder: xc_dom_find_loader: trying multiboot-binary loader ...
2020-06-09 08:34:02.836+0000: domainbuilder: loader probe failed
2020-06-09 08:34:02.837+0000: domainbuilder: xc_dom_find_loader: trying HVM-generic loader ...
2020-06-09 08:34:02.837+0000: domainbuilder: loader probe failed
2020-06-09 08:34:02.837+0000: domainbuilder: xc_dom_find_loader: trying Linux bzImage loader ...
2020-06-09 08:34:02.837+0000: domainbuilder: xc_dom_probe_bzimage_kernel: kernel is not a bzImage
2020-06-09 08:34:02.837+0000: domainbuilder: loader probe failed
2020-06-09 08:34:02.837+0000: domainbuilder: xc_dom_find_loader: trying ELF-generic loader ...
2020-06-09 08:34:02.837+0000: domainbuilder: loader probe OK
2020-06-09 08:34:02.837+0000: xc: ELF: phdr: paddr=0 memsz=0x41d148
2020-06-09 08:34:02.837+0000: xc: ELF: phdr: paddr=0x41d148 memsz=0x20e538
2020-06-09 08:34:02.837+0000: xc: ELF: memory: 0 -> 0x62b680
2020-06-09 08:34:02.837+0000: xc: ELF: note: GUEST_OS = "GRUB2"
2020-06-09 08:34:02.837+0000: xc: ELF: note: LOADER = "generic"
2020-06-09 08:34:02.837+0000: xc: ELF: note: XEN_VERSION = "xen-3.0"
2020-06-09 08:34:02.837+0000: xc: ELF: note: ENTRY = 0
2020-06-09 08:34:02.837+0000: xc: ELF: note: VIRT_BASE = 0
2020-06-09 08:34:02.837+0000: xc: ELF_PADDR_OFFSET unset, using 0
2020-06-09 08:34:02.837+0000: xc: ELF: addresses:
2020-06-09 08:34:02.837+0000: xc: virt_base = 0x0
2020-06-09 08:34:02.837+0000: xc: elf_paddr_offset = 0x0
2020-06-09 08:34:02.837+0000: xc: virt_offset = 0x0
2020-06-09 08:34:02.837+0000: xc: virt_kstart = 0x0
2020-06-09 08:34:02.837+0000: xc: virt_kend = 0x62b680
2020-06-09 08:34:02.837+0000: xc: virt_entry = 0x0
2020-06-09 08:34:02.837+0000: xc: p2m_base = 0xffffffffffffffff
2020-06-09 08:34:02.837+0000: domainbuilder: xc_dom_parse_elf_kernel: xen-3.0-x86_64: 0x0 -> 0x62b680
2020-06-09 08:34:02.837+0000: domainbuilder: xc_dom_mem_init: mem 1024 MB, pages 0x40000 pages, 4k each
2020-06-09 08:34:02.837+0000: domainbuilder: xc_dom_mem_init: 0x40000 pages
2020-06-09 08:34:02.837+0000: domainbuilder: xc_dom_boot_mem_init: called
2020-06-09 08:34:02.837+0000: domainbuilder: x86_compat: guest xen-3.0-x86_64, address size 64
2020-06-09 08:34:02.842+0000: domainbuilder: xc_dom_malloc : 2048 kB
2020-06-09 08:34:02.849+0000: domainbuilder: xc_dom_build_image: called
2020-06-09 08:34:02.850+0000: domainbuilder: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x0+0x62c at 0x7f1a24515000
2020-06-09 08:34:02.853+0000: domainbuilder: xc_dom_alloc_segment: kernel : 0x0 -> 0x62c000 (pfn 0x0 + 0x62c pages)
2020-06-09 08:34:02.853+0000: xc: ELF: phdr 0 at 0x7f1a24515000 -> 0x7f1a24522f2f
2020-06-09 08:34:02.853+0000: xc: ELF: phdr 2 at 0x7f1a24932148 -> 0x7f1a24b40680
2020-06-09 08:34:02.856+0000: domainbuilder: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x62c+0x200 at 0x7f1a24315000
2020-06-09 08:34:02.856+0000: domainbuilder: xc_dom_alloc_segment: phys2mach : 0x62c000 -> 0x82c000 (pfn 0x62c + 0x200 pages)
2020-06-09 08:34:02.856+0000: domainbuilder: xc_dom_alloc_page : start info : 0x82c000 (pfn 0x82c)
2020-06-09 08:34:02.856+0000: domainbuilder: xc_dom_alloc_page : xenstore : 0x82d000 (pfn 0x82d)
2020-06-09 08:34:02.857+0000: domainbuilder: xc_dom_alloc_page : console : 0x82e000 (pfn 0x82e)
2020-06-09 08:34:02.857+0000: domainbuilder: count_pgtables: 0x0000007fffffffff/39: 0x0000000000000000 -> 0x0000007fffffffff, 1 table(s)
2020-06-09 08:34:02.857+0000: domainbuilder: count_pgtables: 0x000000003fffffff/30: 0x0000000000000000 -> 0x000000003fffffff, 1 table(s)
2020-06-09 08:34:02.857+0000: domainbuilder: count_pgtables: 0x00000000001fffff/21: 0x0000000000000000 -> 0x0000000000bfffff, 6 table(s)
2020-06-09 08:34:02.857+0000: domainbuilder: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x82f+0x9 at 0x7f1a6d67f000
2020-06-09 08:34:02.857+0000: domainbuilder: xc_dom_alloc_segment: page tables : 0x82f000 -> 0x838000 (pfn 0x82f + 0x9 pages)
2020-06-09 08:34:02.857+0000: domainbuilder: xc_dom_alloc_page : boot stack : 0x838000 (pfn 0x838)
2020-06-09 08:34:02.857+0000: domainbuilder: xc_dom_build_image : virt_alloc_end : 0x839000
2020-06-09 08:34:02.857+0000: domainbuilder: xc_dom_build_image : virt_pgtab_end : 0xc00000
2020-06-09 08:34:02.857+0000: domainbuilder: xc_dom_boot_image: called
2020-06-09 08:34:02.857+0000: domainbuilder: xc_dom_compat_check: supported guest type: xen-3.0-x86_64 <= matches
2020-06-09 08:34:02.857+0000: domainbuilder: xc_dom_compat_check: supported guest type: xen-3.0-x86_32p
2020-06-09 08:34:02.857+0000: domainbuilder: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32
2020-06-09 08:34:02.857+0000: domainbuilder: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32p
2020-06-09 08:34:02.857+0000: domainbuilder: xc_dom_compat_check: supported guest type: hvm-3.0-x86_64
2020-06-09 08:34:02.857+0000: domainbuilder: xc_dom_update_guest_p2m: dst 64bit, pages 0x40000
2020-06-09 08:34:02.857+0000: domainbuilder: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x82c+0x1 at 0x7f1a6d75a000
2020-06-09 08:34:02.857+0000: domainbuilder: start_info_x86_64: called
2020-06-09 08:34:02.858+0000: domainbuilder: domain builder memory footprint
2020-06-09 08:34:02.858+0000: domainbuilder: allocated
2020-06-09 08:34:02.858+0000: domainbuilder: malloc : 2100 kB
2020-06-09 08:34:02.858+0000: domainbuilder: anon mmap : 0 bytes
2020-06-09 08:34:02.858+0000: domainbuilder: mapped
2020-06-09 08:34:02.858+0000: domainbuilder: file mmap : 2165 kB
2020-06-09 08:34:02.858+0000: domainbuilder: domU mmap : 8408 kB
2020-06-09 08:34:02.859+0000: domainbuilder: bootlate_pv: shared_info: pfn 0x0, mfn 0x414b
2020-06-09 08:34:02.859+0000: domainbuilder: shared_info_x86_64: called
2020-06-09 08:34:02.859+0000: domainbuilder: vcpu_x86_64: called
2020-06-09 08:34:02.859+0000: domainbuilder: vcpu_x86_64: cr3: pfn 0x82f mfn 0x1111e2f
2020-06-09 08:34:02.861+0000: domainbuilder: xc_dom_release: called
2020-06-09 08:34:02.879+0000: libxl: libxl_device.c:398:libxl__device_disk_set_backend: Disk vdev=xvda spec.backend=qdisk
2020-06-09 08:34:02.880+0000: libxl: libxl_device.c:347:disk_try_backend: Disk vdev=xvda, uses script=(null) on qdisk backend
2020-06-09 08:34:02.913+0000: libxl: libxl_linux.c:226:libxl__get_hotplug_script_info: Domain 1129:backend_kind 3, no need to execute scripts
2020-06-09 08:34:02.914+0000: libxl: libxl_device.c:1204:device_hotplug: Domain 1129:No hotplug script to execute
2020-06-09 08:34:02.914+0000: libxl: libxl_event.c:686:libxl__ev_xswatch_deregister: watch w=0x7f1a60003f60: deregister unregistered
2020-06-09 08:34:02.914+0000: libxl: libxl_device.c:398:libxl__device_disk_set_backend: Disk vdev=xvdb spec.backend=qdisk
2020-06-09 08:34:02.914+0000: libxl: libxl_device.c:347:disk_try_backend: Disk vdev=xvdb, uses script=(null) on qdisk backend
2020-06-09 08:34:02.950+0000: libxl: libxl_linux.c:226:libxl__get_hotplug_script_info: Domain 1129:backend_kind 3, no need to execute scripts
2020-06-09 08:34:02.950+0000: libxl: libxl_device.c:1204:device_hotplug: Domain 1129:No hotplug script to execute
2020-06-09 08:34:02.950+0000: libxl: libxl_event.c:686:libxl__ev_xswatch_deregister: watch w=0x7f1a60006aa0: deregister unregistered
2020-06-09 08:34:03.002+0000: libxl: libxl_dm.c:2372:libxl__spawn_local_dm: Domain 1129:Spawning device-model /usr/bin/qemu-system-i386 with arguments:
2020-06-09 08:34:03.002+0000: libxl: libxl_dm.c:2374:libxl__spawn_local_dm: Domain 1129: /usr/bin/qemu-system-i386
2020-06-09 08:34:03.002+0000: libxl: libxl_dm.c:2374:libxl__spawn_local_dm: Domain 1129: -xen-domid
2020-06-09 08:34:03.002+0000: libxl: libxl_dm.c:2374:libxl__spawn_local_dm: Domain 1129: 1129
2020-06-09 08:34:03.002+0000: libxl: libxl_dm.c:2374:libxl__spawn_local_dm: Domain 1129: -chardev
2020-06-09 08:34:03.002+0000: libxl: libxl_dm.c:2374:libxl__spawn_local_dm: Domain 1129: socket,id=libxl-cmd,path=/var/run/xen/qmp-libxl-1129,server,nowait
2020-06-09 08:34:03.002+0000: libxl: libxl_dm.c:2374:libxl__spawn_local_dm: Domain 1129: -no-shutdown
2020-06-09 08:34:03.002+0000: libxl: libxl_dm.c:2374:libxl__spawn_local_dm: Domain 1129: -mon
2020-06-09 08:34:03.002+0000: libxl: libxl_dm.c:2374:libxl__spawn_local_dm: Domain 1129: chardev=libxl-cmd,mode=control
2020-06-09 08:34:03.003+0000: libxl: libxl_dm.c:2374:libxl__spawn_local_dm: Domain 1129: -chardev
2020-06-09 08:34:03.003+0000: libxl: libxl_dm.c:2374:libxl__spawn_local_dm: Domain 1129: socket,id=libxenstat-cmd,path=/var/run/xen/qmp-libxenstat-1129,server,nowait
2020-06-09 08:34:03.003+0000: libxl: libxl_dm.c:2374:libxl__spawn_local_dm: Domain 1129: -mon
2020-06-09 08:34:03.003+0000: libxl: libxl_dm.c:2374:libxl__spawn_local_dm: Domain 1129: chardev=libxenstat-cmd,mode=control
2020-06-09 08:34:03.003+0000: libxl: libxl_dm.c:2374:libxl__spawn_local_dm: Domain 1129: -nodefaults
2020-06-09 08:34:03.003+0000: libxl: libxl_dm.c:2374:libxl__spawn_local_dm: Domain 1129: -no-user-config
2020-06-09 08:34:03.003+0000: libxl: libxl_dm.c:2374:libxl__spawn_local_dm: Domain 1129: -xen-attach
2020-06-09 08:34:03.003+0000: libxl: libxl_dm.c:2374:libxl__spawn_local_dm: Domain 1129: -name
2020-06-09 08:34:03.003+0000: libxl: libxl_dm.c:2374:libxl__spawn_local_dm: Domain 1129: openQA-SUT-22
2020-06-09 08:34:03.003+0000: libxl: libxl_dm.c:2374:libxl__spawn_local_dm: Domain 1129: -vnc
2020-06-09 08:34:03.003+0000: libxl: libxl_dm.c:2374:libxl__spawn_local_dm: Domain 1129: 0.0.0.0:2,password
2020-06-09 08:34:03.003+0000: libxl: libxl_dm.c:2374:libxl__spawn_local_dm: Domain 1129: -display
2020-06-09 08:34:03.003+0000: libxl: libxl_dm.c:2374:libxl__spawn_local_dm: Domain 1129: none
2020-06-09 08:34:03.003+0000: libxl: libxl_dm.c:2374:libxl__spawn_local_dm: Domain 1129: -machine
2020-06-09 08:34:03.003+0000: libxl: libxl_dm.c:2374:libxl__spawn_local_dm: Domain 1129: xenpv
2020-06-09 08:34:03.003+0000: libxl: libxl_dm.c:2374:libxl__spawn_local_dm: Domain 1129: -m
2020-06-09 08:34:03.003+0000: libxl: libxl_dm.c:2374:libxl__spawn_local_dm: Domain 1129: 1024
2020-06-09 08:34:03.003+0000: libxl: libxl_dm.c:2376:libxl__spawn_local_dm: Domain 1129:Spawning device-model /usr/bin/qemu-system-i386 with additional environment:
2020-06-09 08:34:03.003+0000: libxl: libxl_dm.c:2378:libxl__spawn_local_dm: Domain 1129: XEN_QEMU_CONSOLE_LIMIT=1048576
2020-06-09 08:34:03.003+0000: libxl: libxl_event.c:636:libxl__ev_xswatch_register: watch w=0x7f1a60002ca8 wpath=/local/domain/0/device-model/1129/state token=2/8: register slotnum=2
2020-06-09 08:34:03.025+0000: libxl: libxl_event.c:573:watchfd_callback: watch w=0x7f1a60002ca8 wpath=/local/domain/0/device-model/1129/state token=2/8: event epath=/local/domain/0/device-model/1129/state
2020-06-09 08:34:03.025+0000: libxl: libxl_exec.c:398:spawn_watch_event: domain 1129 device model: spawn watch p=(null)
2020-06-09 08:34:03.186+0000: libxl: libxl_event.c:673:libxl__ev_xswatch_deregister: watch w=0x7f1a60002ca8 wpath=/local/domain/0/device-model/1129/state token=2/8: deregister slotnum=2
2020-06-09 08:34:03.186+0000: libxl: libxl_dm.c:2468:device_model_spawn_outcome: Domain 1129:domain 1129 device model: spawn failed (rc=-3)
2020-06-09 08:34:03.186+0000: libxl: libxl_create.c:1577:domcreate_devmodel_started: Domain 1129:device model did not start: -3
2020-06-09 08:34:03.188+0000: libxl: libxl_dm.c:2582:kill_device_model: Device Model already exited
2020-06-09 08:34:05.193+0000: libxl: libxl_linux.c:226:libxl__get_hotplug_script_info: Domain 1129:backend_kind 3, no need to execute scripts
2020-06-09 08:34:05.193+0000: libxl: libxl_device.c:1204:device_hotplug: Domain 1129:No hotplug script to execute
2020-06-09 08:34:05.193+0000: libxl: libxl_event.c:686:libxl__ev_xswatch_deregister: watch w=0x7f1a6000ce70: deregister unregistered
2020-06-09 08:34:05.216+0000: libxl: libxl_linux.c:226:libxl__get_hotplug_script_info: Domain 1129:backend_kind 3, no need to execute scripts
2020-06-09 08:34:05.216+0000: libxl: libxl_device.c:1204:device_hotplug: Domain 1129:No hotplug script to execute
2020-06-09 08:34:05.216+0000: libxl: libxl_event.c:686:libxl__ev_xswatch_deregister: watch w=0x7f1a6000d190: deregister unregistered
2020-06-09 08:34:05.230+0000: libxl: libxl_linux.c:226:libxl__get_hotplug_script_info: Domain 1129:backend_kind 5, no need to execute scripts
2020-06-09 08:34:05.230+0000: libxl: libxl_device.c:1204:device_hotplug: Domain 1129:No hotplug script to execute
2020-06-09 08:34:05.230+0000: libxl: libxl_event.c:686:libxl__ev_xswatch_deregister: watch w=0x7f1a6000d4b0: deregister unregistered
2020-06-09 08:34:05.242+0000: libxl: libxl_linux.c:226:libxl__get_hotplug_script_info: Domain 1129:backend_kind 6, no need to execute scripts
2020-06-09 08:34:05.242+0000: libxl: libxl_device.c:1204:device_hotplug: Domain 1129:No hotplug script to execute
2020-06-09 08:34:05.242+0000: libxl: libxl_event.c:686:libxl__ev_xswatch_deregister: watch w=0x7f1a60007950: deregister unregistered
2020-06-09 08:34:05.281+0000: libxl: libxl_domain.c:1193:devices_destroy_cb: Domain 1129:Forked pid 3282 for destroy of domain
2020-06-09 08:34:05.344+0000: libxl: libxl_event.c:573:watchfd_callback: watch w=0x7f1a0c0064f0 wpath=@releaseDomain token=3/0: event epath=@releaseDomain
2020-06-09 08:34:05.345+0000: libxl: libxl_domain.c:792:domain_death_xswatch_callback: [evg=0] all reported
2020-06-09 08:34:05.345+0000: libxl: libxl_domain.c:854:domain_death_xswatch_callback: domain death search done
2020-06-09 08:34:05.350+0000: libxl: libxl_event.c:1873:libxl__ao_complete: ao 0x7f1a600048b0: complete, rc=-3
2020-06-09 08:34:05.350+0000: libxl: libxl_event.c:1842:libxl__ao__destroy: ao 0x7f1a600048b0: destroy
Updated by szarate over 4 years ago
- File libxl-driver.log libxl-driver.log added
Updated by szarate over 4 years ago
- Subject changed from test fails in bootloader_svirt - virsh define failed on openqa-SUT-2 to [u] test fails in bootloader_svirt - virsh define failed for domain: openqa-SUT-2
So, running the qemu command, shows:
qemu-system-i386: -vnc 0.0.0.0:2,password: Failed to start VNC server: Failed to find an available port: Address already in use
Updated by szarate over 4 years ago
so after
ss -nutlp | grep qemu
And killing the leftover process:
tcp LISTEN 0 1 0.0.0.0:5902 0.0.0.0:* users:(("qemu-system-i38",pid=5121,fd=41))
vm's can be spawned again
as a bonus changed libvirtd to log libxl activity as suggested by Olaf Hering:
#file:/etc/libvirt/libvirtd.conf
log_level = 1
log_outputs="1:file:/dev/null"
Updated by szarate over 4 years ago
- Status changed from In Progress to Resolved
virsh desc openQA-SUT-2
openQA Instance 2: 04330472-sle-15-SP2-Online-x86_64-Build207.1-allpatterns@svirt-xen-pv
Actions