action #106083
closedcoordination #109668: [saga][epic] Stable and updated non-qemu backends for SLE validation
coordination #100688: [epic][virtualization][3rd party hypervisor] Add svirt backend compatibility for vmware 7.0
[virtualization][3rd party hypervisor][timeboxed:10h][research] Learn about VMWare VirtualMachine.AcquireTicket("webmks") API size:S
Added by okurz about 2 years ago. Updated about 2 years ago.
Description
Tasks¶
- For #100688 research task: Learn about VirtualMachine.AcquireTicket("webmks") API first and refine ticket to understand if we can use "VNC as-is" or need further tunneling, etc.
- Optional: Try it out on the live production VMWare 7 server , find credentials in #100688#note-25
- Refine and estimate #100688 with the team
Acceptance criteria¶
- AC1: #100688 is updated with what is required to establish a VNC connection with VMWare 7
Updated by okurz about 2 years ago
- Copied from coordination #100688: [epic][virtualization][3rd party hypervisor] Add svirt backend compatibility for vmware 7.0 added
Updated by okurz about 2 years ago
- Description updated (diff)
- Priority changed from Normal to High
Updated by livdywan about 2 years ago
- Subject changed from [virtualization][3rd party hypervisor][timeboxed:10h][research] Learn about VMWare VirtualMachine.AcquireTicket("webmks") API to [virtualization][3rd party hypervisor][timeboxed:10h][research] Learn about VMWare VirtualMachine.AcquireTicket("webmks") API size:S
- Description updated (diff)
Updated by mkittler about 2 years ago
- Assignee set to mkittler
- See section "VNC Server in ESXi" on https://www.stevenbright.com/2020/04/vmware-vsphere-7-0-now-available for the overall problem.
- The
AcquireTicketmethod is referenced here: https://developer.vmware.com/apis/1192/vsphere (search for "AcquireTicket"), frame: https://vdc-download.vmware.com/vmwb-repository/dcr-public/bf660c0a-f060-46e8-a94d-4b5e6ffc77ad/208bc706-e281-49b6-a0ce-b402ec19ef82/SDK/vsphere-ws/docs/ReferenceGuide/vim.VirtualMachine.html#acquireTicket webmksis mentioned here but with no further details: https://vdc-download.vmware.com/vmwb-repository/dcr-public/bf660c0a-f060-46e8-a94d-4b5e6ffc77ad/208bc706-e281-49b6-a0ce-b402ec19ef82/SDK/vsphere-ws/docs/ReferenceGuide/vim.VirtualMachine.TicketType.html- The documentation doesn't go into detail but I assume
AcquireTicketwill return the host and port for the VNC server (see https://vdc-download.vmware.com/vmwb-repository/dcr-public/bf660c0a-f060-46e8-a94d-4b5e6ffc77ad/208bc706-e281-49b6-a0ce-b402ec19ef82/SDK/vsphere-ws/docs/ReferenceGuide/vim.VirtualMachine.Ticket.html). - I assume they leave out the details expecting users to use their HTML-SDK (https://vdc-download.vmware.com/vmwb-repository/dcr-public/b7ae1097-9dc8-4323-9877-6e5380c54d17/48aca922-d7ac-455d-865d-50c9194de37a/HTML-Console-SDK-21.pdf).
- I found one implementation in Go: https://github.com/hashicorp/packer/pull/9938/files#diff-3e7e9fecf2da616495990c6c91245d6affda62cb5e489e6248d1e5e7305a8e0fR51
- Call
AcquireTicketfrom vSphere Web Services API. I haven't found a concrete example but in doubt it shouldn't be too hard to check what their HTML-SDK or this Go project does. - The response of the
AcquireTicketcall should contain the host and port. One can establish a ws connection via an URL like this:websocketUrl := fmt.Sprintf("wss://%s:%d/ticket/%s", host, port, ticket.Ticket). The VNC traffic is then likely simply sent via binary ws messages.
- Call
- I'm not sure whether documentation on https://developer.vmware.com/apis/vsphere-automation/latest/vcenter is relevant.
Updated by mkittler about 2 years ago
Login via ssh root@10.67.131.2 and on https://10.67.131.2/ui/#/login is possible using credentials from #100688#note-25. Apparently the test license for ESXi is expired. The UI denies powering on the VM due to that. I suppose that's going to be a problem. It also says the machine isn't managed via vSphere which could also be a problem.
Updated by mkittler about 2 years ago
- Status changed from Workable to Feedback
I tried to create a new VM but unfortunately this doesn't workaround the licensing issue.
Updated by xlai about 2 years ago
mkittler wrote:
Login via
ssh root@10.67.131.2and on https://10.67.131.2/ui/#/login is possible using credentials from #100688#note-25. Apparently the test license for ESXi is expired. The UI denies powering on the VM due to that. I suppose that's going to be a problem. It also says the machine isn't managed via vSphere which could also be a problem.
@nanzhang Would you please help on this?
Updated by nanzhang about 2 years ago
I've assigned an evaluation license. Please take a try again.
Updated by mkittler about 2 years ago
It works. Now that I could have a look at a live machine I can give more concrete info:
- The URL for requesting VNC via websocket looks like this:
https://10.67.131.2/sdk/¹- One needs the
operationIDand number of the VM. - One gets a web socket URL in return.
- One needs the
- The websocket URL looks like this:
wss://10.67.131.2/ticket/c5d1da02b468acb1²
¹ Request headers:
POST /sdk/ HTTP/1.1
Host: 10.67.131.2
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:97.0) Gecko/20100101 Firefox/97.0
Accept: */*
Accept-Language: de,en-US;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate, br
Content-Type: text/xml
SOAPAction: urn:vim25/7.0.2.0
VMware-CSRF-Token: 2svu170jkxfez9og06zbfybyd1zoghm9
Content-Length: 310
Origin: https://10.67.131.2
Connection: keep-alive
Referer: https://10.67.131.2/ui/
Cookie: vmware_client=VMware; vmware_soap_session="6944eb445e8bcc9a1880b9ec68c6ece484146886"
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
Request data:
<Envelope xmlns="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><Header><operationID>esxui-c51d</operationID></Header><Body><AcquireTicket xmlns="urn:vim25"><_this type="VirtualMachine">2</_this><ticketType>webmks</ticketType></AcquireTicket></Body></Envelope>
Response headers:
HTTP/1.1 200 OK
Date: Wed, 16 Feb 2022 07:15:10 GMT
Cache-Control: no-cache
Connection: Keep-Alive
Content-Type: text/xml; charset=utf-8
X-Frame-Options: DENY
Content-Length: 726
Response data:
<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<soapenv:Body>
<AcquireTicketResponse xmlns="urn:vim25"><returnval><ticket>c5d1da02b468acb1</ticket><cfgFile>/vmfs/volumes/5ffc09f2-d9e8dde4-1604-0cc47ac51e38/tools-team-testvm/tools-team-testvm.vmx</cfgFile><port>443</port><sslThumbprint>60:57:01:9A:EB:0F:59:9E:F9:65:D5:7C:C0:F9:C7:3A:3F:FD:C6:12</sslThumbprint><url>wss://vh002.qa2.suse.asia:443/ticket/c5d1da02b468acb1</url></returnval></AcquireTicketResponse>
</soapenv:Body>
</soapenv:Envelope>
² Request headers:
GET /ticket/c5d1da02b468acb1 HTTP/1.1
Host: 10.67.131.2
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:97.0) Gecko/20100101 Firefox/97.0
Accept: */*
Accept-Language: de,en-US;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate, br
Sec-WebSocket-Version: 13
Origin: https://10.67.131.2
Sec-WebSocket-Protocol: binary, vmware-vvc
Sec-WebSocket-Extensions: permessage-deflate
Sec-WebSocket-Key: u0aBMK+ibS5+ZzoR3Z3qBg==
Connection: keep-alive, Upgrade
Cookie: vmware_client=VMware; vmware_soap_session="6944eb445e8bcc9a1880b9ec68c6ece484146886"
Sec-Fetch-Dest: websocket
Sec-Fetch-Mode: websocket
Sec-Fetch-Site: same-origin
Pragma: no-cache
Cache-Control: no-cache
Upgrade: websocket
Response headers:
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Protocol: binary
Sec-WebSocket-Accept: ylCtg6T1Wqyd+CfO3um5lxl0NrM=
Traffic looks like VNC. (Unfortunately Firefox's dev tools aren't well suited for looking at binary code.)
Updated by mkittler about 2 years ago
- Status changed from Workable to Resolved
This is the minimum I could come up with for the acquiring the ticket:
Get session cookie (I replaced the credentials with "…"):
curl --insecure -i -X POST -H "Content-Type: text/xml" -H "SOAPAction: urn:vim25/7.0.2.0" -H "Cookie: vmware_client=VMware" -d '<Envelope xmlns="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><Header><operationID>esxui-8fb8</operationID></Header><Body><Login xmlns="urn:vim25"><_this type="SessionManager">ha-sessionmgr</_this><userName>…</userName><password>…</password></Login></Body></Envelope>' https://10.67.131.2/sdk
Request the web socket URL:
curl --insecure -X POST -H "Content-Type: text/xml" -H "SOAPAction: urn:vim25/7.0.2.0" -H "Cookie: vmware_client=VMware; vmware_soap_session="84d5ef6d4cd8b80f32b8ef1a514d623c12013083"" -d '<Envelope xmlns="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><Header><operationID>esxui-c51d</operationID></Header><Body><AcquireTicket xmlns="urn:vim25"><_this type="VirtualMachine">2</_this><ticketType>webmks</ticketType></AcquireTicket></Body></Envelope>' https://10.67.131.2/sdk
So the header VMware-CSRF-Token: 2svu170jkxfez9og06zbfybyd1zoghm9 is not required.
I checked the web socket headers in Chromium which can display binary data much better:
00000000: 0014 0004 0000 0000 0010 0010 574d 567f ............WMV.
00000001: 0100 0500 0316 018a 0000 0000 0010 0010 ................
00000002: ffff fefc a0df 8000 8950 4e47 0d0a 1a0a .........PNG....
00000003: 0000 000d 4948 4452 0000 0010 0000 0010 ....IHDR........
00000004: 0802 0000 0090 9168 3600 0000 2649 4441 .......h6...&IDA
00000005: 5428 5363 6420 1130 1252 800e 4635 1003 T(Scd .0.R..F5..
00000006: 4635 40c0 aa55 ab30 0521 202c 2c8c 1a1a F5@..U.0.! ,,...
00000007: f003 004d ca06 11d9 feb7 6300 0000 0049 ...M......c....I
00000008: 454e 44ae 4260 8200 3001 8000 1000 1057 END.B`..0......W
00000009: 4d56 7f02 0005 0001 c000 0000 0000 0000 MV..............
0000000a: 0057 4d56 7c52 1059 6c95 0000 00 .WMV|R.Yl....
When I interpret it correctly, the encoding is specified as 574d 567f (2136362327). That's not specified in https://datatracker.ietf.org/doc/html/rfc6143#section-7.7. The fact that the PNG magic number (8950 4E47 0D0A 1A0A) is part of the payload let's me think that PNG is used here. Our VNC implementation doesn't support it. I hope the server will not use it in case the client doesn't support it.
I suppose with all of that information it should be possible to start working on #100688.
Updated by okurz about 2 years ago
- Status changed from Resolved to Feedback
please see AC1. The important parts should be in the parent epic or a sibling specific ticket with suggestions what to implement
Updated by okurz about 2 years ago
Well, we need to ensure that the team can estimate #100688 accordingly. So whatever is necessary to get that done we need :)