Project

General

Profile

Actions

action #10206

closed

openQA Project - action #9680: libvirt backend

[tools]libvirt tests (Xen, Hyper-V, VMware)

Added by RBrownSUSE over 8 years ago. Updated almost 5 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
michalnowak
Category:
New test
Target version:
-
Start date:
2016-01-13
Due date:
% Done:

100%

Estimated time:
Difficulty:
hard

Description

We have the 'svirt' (libvirt over ssh) backend completed openqa#9680

Therefore we should use it to have openQA run tests on different libvirt supported hypervisors

We specifically need Xen (PV and FullVirt), HyperV, and VMware

For details about the requested guests see Fate #320424, #320425, #320426:
https://fate.suse.com/320424
https://fate.suse.com/320425
https://fate.suse.com/320426


Checklist

  • SLE

Related issues 12 (0 open12 closed)

Related to openQA Tests - action #33064: [functional][u][hyperv][hard] svirt-hyperv tests loose key presses. Related to FreeRDP update?Resolvedoorlov2018-03-122018-04-24

Actions
Related to openQA Tests - action #33091: [functional][hyperv][u] test fails in accept_license on slow systemResolvedmichalnowak2018-03-122018-05-22

Actions
Related to openQA Tests - action #32926: [sle][functional][y][hyperv][medium] avoid typing username before switched tty (was: test fails in yast2_i - (mising needles?, rather too low timeout for hyperv) for Installation Report succesful)Resolvedokurz2018-03-082018-05-22

Actions
Related to openQA Tests - action #32929: [sle][functional][u][hyperv] test fails in postgresql_server - SubState=running not foundRejectedokurz2018-03-082018-07-03

Actions
Related to openQA Tests - action #33970: [sle][functional][u][hyperv] test fails in apache_nss - script died or timeoutRejectedokurz2018-03-282018-07-03

Actions
Blocked by openQA Project - action #13532: DIE short read for zlre data on 'svirt' on KVM/Xen hostsResolvedcoolo2016-09-01

Actions
Blocked by openQA Project - action #17370: [svirt] serial0.txt is distorted on Xen HVMResolved2017-02-28

Actions
Blocked by openQA Project - action #15718: System not getting 'ctrl-alt-delete' on svirt backendResolved2017-01-02

Actions
Blocked by openQA Project - action #15732: Exit code missing from "magic string" on svirtResolved2017-01-02

Actions
Blocked by openQA Project - action #16418: [tools][sprint 201712.2] svirt: openQA won't perform another query_isotovideo check when VM is down -> incompletesResolvedmichalnowak2017-02-02

Actions
Blocked by openQA Project - action #13540: send_key() sometimes gets lost on svirt backendResolved2016-09-01

Actions
Blocked by openQA Project - action #17658: VNC reconnect feature needed for xen/svirt and stalled VNC connectionsResolved2017-03-10

Actions
Actions #1

Updated by okurz over 8 years ago

  • Description updated (diff)
Actions #2

Updated by RBrownSUSE over 8 years ago

  • Priority changed from Normal to High
Actions #3

Updated by RBrownSUSE over 8 years ago

  • Checklist item changed from to [ ] SLE
  • Target version deleted (156)
Actions #4

Updated by maritawerner over 8 years ago

For details about the requested guests see Fate #319284

Actions #5

Updated by maritawerner over 8 years ago

  • Description updated (diff)

RBrownSUSE wrote:

We have the 'svirt' (libvirt over ssh) backend completed openqa#9680

Therefore we should use it to have openQA run tests on different libvirt supported hypervisors

We specifically need Xen (PV and FullVirt), HyperV, and VMware

Actions #6

Updated by maritawerner about 8 years ago

  • Description updated (diff)
Actions #7

Updated by RBrownSUSE about 8 years ago

  • Assignee set to mkravec
  • Target version set to Milestone 2
Actions #8

Updated by RBrownSUSE about 8 years ago

  • Assignee changed from mkravec to michalnowak
Actions #9

Updated by RBrownSUSE about 8 years ago

  • Target version changed from Milestone 2 to 168
Actions #10

Updated by michalnowak about 8 years ago

Created FATE#320778: "Support HyperV driver in libvirt". Otherwise there's no way to connect to hyperv.

Still with ESX and HyperV we don't have a VNC on the hypervisor by default. Microsoft and VMware have their native solutions (RDP, and 'VMware Remote Console', respectively) in this regard. We might use RDP with FreeRDP, unlike VMware Remote Console which is so far supported on Windows and OS X only.

Actions #11

Updated by michalnowak about 8 years ago

It seems to me that there's no way to test SLES on VMware in openQA.

  1. For JeOS deployment or SLES install from DVD I need a VNC connection to the SUT ("the VM itself"). But I don't have it as VMware does not provide a VNC service on the hypervisor.
  2. Network installation on VMware is impossible as it does not support direct kernel boot (kernel + initrd get stripped from the XML on virsh create time).
  3. VNC/SSH installation from DVD is impossible because kernel arguments like "sshd=1" or "vnc=1" needs to be provided but that's not possible as there's no VNC.

Perhaps it's possible to do DVD installation with 'VMware Remote Console' on Windows but (1) I don't have access to it, and (2) I am not sure we want to have Windows in the mix.

Let me know if there's a SLES-install vector I omitted here.

Actions #12

Updated by RBrownSUSE about 8 years ago

Can't 1, 2, and 3 all be resolved by using Direct Kernel Boot?

https://libvirt.org/formatdomain.html#elementsOSKernel

I've used this approach on VMware before, as well as other scenarios where all seemed equally impossible (Debian to openSUSE conversions on bare metal with no IPMI or other management channel)

https://en.opensuse.org/SDB:Remote_installation
https://en.opensuse.org/SDB:Linuxrc

Actions #13

Updated by michalnowak about 8 years ago

I was wrong claiming "there's no VNC on VMware", there is (RemoteDisplay.vnc.* in VMX file) - though the only reference for "VNC on VMware" on the Internet is.. libvirt tests. So at least JeOS and installs from DVD seems to be possible.

Ad "Direct Kernel Boot": libvirt strips

<os>
    ...
    <kernel>/var/lib/libvirt/images/openQA-SUT-1.kernel</kernel>
    <initrd>/var/lib/libvirt/images/openQA-SUT-1.initrd</initrd>
</os>

on define time (verified by dumpxml). Do you know of any other way?

Actions #14

Updated by michalnowak about 8 years ago

Current status

JeOS works on KVM, Xen (both), VMware; net install works on KVM and Xen (VMware does not support direct kernel boot); installation from DVD is pending. Polishing is needed.

HyperV

FATE#320424 requests support of following host environments wrt HyperV: Microsoft Windows 2008 SP2+, 2008 R2 SP1+, 2012+, 2012 R2+, and 2016 (once released). However 'hyperv' driver from libvirt supports only Windows Server 2008 R2 (and perhaps - presumably - the 2008 R1, see https://libvirt.org/drvhyperv.html), WS2008R2 is now in extended support only. There's no way to communicate with WS 2012/2016 via libvirt.

There are two other hurdles along the way: (1) no VNC on HyperV, we may need to use RDP instead, and (2) can't find a way how to output serial port from VM (`virsh console' is not implemented in 'hyperv' driver). Plus 'hyperv' needs to be enabled in our libvirt.

Marita, Richard: In light of this, should I pursue the task on HyperV?

Actions #15

Updated by RBrownSUSE about 8 years ago

1) No VNC on HyperV - there is no VNC on zKVM or z/VM either but we manage to test both there using the VNC of the OS, instead of the VNC from the hypervisor. Have a look at the s390x backend or talk to Matthias for help with that - should not be a blocker in theory

2) While I admit what I am about to propose is a little hackish, it should be a simple case of using something like netcat from the SUT to forward the data from the Serial to the worker. This wouldn't be that dissimilar from how we have gotten the s390x backend working with it's x3270 emulation of a serial console. Matthias might again be the best person to help with that.

3) the platform support - SUSE OpenStack Cloud supports Windows 2012 https://www.suse.com/products/suse-openstack-cloud/technical-information/
OpenStack uses libvirt...so I guess you need to ask our Cloud guys how the hell they manage to get that working, because it clearly seems to be possible if we support it there..

Actions #16

Updated by okurz almost 8 years ago

  • Target version changed from 168 to Milestone 3
Actions #17

Updated by michalnowak over 7 years ago

  • Status changed from New to In Progress

Status update

KVM, Xen (PV and HVM), Hyper-V, and VMware bootloader tests and necessary backend changes were implemented. VMware waits for Christian Mueller to provide openQA-only ESXi server, for KVM we have PowerKVM - rbrown reports -, the rest of hypervizors waits for additional HW.

Actions #18

Updated by michalnowak over 7 years ago

Status update

I integrated JeOS under svirt backend to 'x-released SLE 12 SP1 6.JeOS group' OSD group. Currently is 12SP2 JeOS images tested under KVM, KVM+UEFI, Xen HVM, and Xen PV. E.g.: https://openqa.suse.de/tests/overview?distri=sle&version=12-SP1&build=25.58&groupid=23.

With snapshots support in svirt (https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/1830) much more tests should be executed.

Currently this Qemu (probably) 12-SP2 bug https://bugzilla.suse.com/show_bug.cgi?id=998613 prevents casual testing because high portion of runs triggers that bug.

Plan:

  1. some Server-DVD scenarios via svirt,
  2. use kvm and xen virtsh host as a openQA worker
Actions #19

Updated by maritawerner over 7 years ago

Uodate about VMWare Licenses: we do not have any left, need to buy new ones, on Harald's desk

Actions #20

Updated by maritawerner over 7 years ago

  • Target version deleted (Milestone 3)

Under broader Discussion.

Actions #21

Updated by michalnowak about 7 years ago

  • Blocked by action #13532: DIE short read for zlre data on 'svirt' on KVM/Xen hosts added
Actions #22

Updated by michalnowak about 7 years ago

  • Blocked by action #17370: [svirt] serial0.txt is distorted on Xen HVM added
Actions #23

Updated by michalnowak about 7 years ago

  • Blocked by action #15718: System not getting 'ctrl-alt-delete' on svirt backend added
Actions #24

Updated by michalnowak about 7 years ago

  • Blocked by action #15732: Exit code missing from "magic string" on svirt added
Actions #25

Updated by michalnowak about 7 years ago

  • Blocked by action #16418: [tools][sprint 201712.2] svirt: openQA won't perform another query_isotovideo check when VM is down -> incompletes added
Actions #26

Updated by michalnowak about 7 years ago

  • Blocked by action #13540: send_key() sometimes gets lost on svirt backend added
Actions #27

Updated by michalnowak about 7 years ago

  • Blocked by action #17658: VNC reconnect feature needed for xen/svirt and stalled VNC connections added
Actions #28

Updated by RBrownSUSE about 7 years ago

  • Subject changed from libvirt tests (Xen, HyperV, VMware) to [tools]libvirt tests (Xen, HyperV, VMware)
  • Priority changed from High to Normal
Actions #29

Updated by okurz about 6 years ago

  • Related to action #33064: [functional][u][hyperv][hard] svirt-hyperv tests loose key presses. Related to FreeRDP update? added
Actions #30

Updated by okurz about 6 years ago

  • Related to action #33091: [functional][hyperv][u] test fails in accept_license on slow system added
Actions #31

Updated by okurz about 6 years ago

  • Related to action #32926: [sle][functional][y][hyperv][medium] avoid typing username before switched tty (was: test fails in yast2_i - (mising needles?, rather too low timeout for hyperv) for Installation Report succesful) added
Actions #32

Updated by okurz about 6 years ago

  • Related to action #32929: [sle][functional][u][hyperv] test fails in postgresql_server - SubState=running not found added
Actions #33

Updated by okurz about 6 years ago

  • Related to action #33970: [sle][functional][u][hyperv] test fails in apache_nss - script died or timeout added
Actions #34

Updated by michalnowak almost 5 years ago

  • Checklist item changed from [ ] SLE to [x] SLE
  • Subject changed from [tools]libvirt tests (Xen, HyperV, VMware) to [tools]libvirt tests (Xen, Hyper-V, VMware)
  • Status changed from In Progress to Resolved
  • % Done changed from 0 to 100
  • Difficulty set to hard

Status update

KVM (zKVM), Xen, Hyper-V, and VMware guest support is implemented in openQA backend and tests. We use them regularly in the JeOS, Functional, YaST, and Virtualization groups.

Feature-wise we are on par with the qemu backend and validations of disk (JeOS, MicroOS) as well as ISO images are performed regularly. Testsuite-wise we do image deployments, installations, and migrations.

Considering that there are very few limitations, or bugs, I consider this big task resolved. The rest is being solved via respective QA-VT processes.

Actions

Also available in: Atom PDF