Media acceleration for virtual computing services

• US 9,407,613 B2
• Filed: 05/01/2012
• Issued: 08/02/2016
• Est. Priority Date: 04/15/2008
• Status: Active Grant

• Current Assignee: VMware, Inc. (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: Hobgood, Andrew W., Baron, Steve, Battersby, Clinton B.
• Primary Examiner(s): Dang, Hung
• Assistant Examiner(s): Park, Sunghyoun
• Application Number:
  US13/461,380
• Publication Number:
  US 20120213294A1
• Time in Patent Office: 1,554 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 9/5061 Partitioning or combining of resources
• CPC Class Codes: H04L 12/4654 wherein a VLAN tag represents a customer VLAN, e.g. C-Tag
• CPC Class Codes: H04L 45/02 Topology update or discovery
• CPC Class Codes: H04L 45/50 using label swapping, e.g. multi-protocol label switch [MPLS]
• CPC Class Codes: H04L 47/70 Admission control; Resource allocation
• CPC Class Codes: H04L 49/70 Virtual switches
• CPC Class Codes: H04L 63/02 for separating internal from external traffic, e.g. firewalls
• CPC Class Codes: H04L 63/0272 Virtual private networks
• CPC Class Codes: H04L 63/0281 Proxies
• CPC Class Codes: H04L 63/08 for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 63/0815 providing single-sign-on or federations
• CPC Class Codes: H04L 63/10 for controlling access to devices or network resources
• CPC Class Codes: H04L 65/61 for supporting one-way streaming services, e.g. Internet radio
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 65/764 at the destination reformatting of additional data in video clients H04N21/4355
• CPC Class Codes: H04L 67/08 specially adapted for terminal emulation, e.g. Telnet
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]

Abstract

Streaming media is problematic for thin clients using remoting protocols like RDP that were never designed to handle the volume of data associated with multimedia. The result is large demands on the host computer and thin client CPU and excessive bandwidth on the network, which results in a poor display quality. A process running on a host computer detects an existing multimedia acceleration channel to a thin client and also identifies unaccelerated media streams like Adobe Flash. The unaccelerated content is automatically re-encoded using a codec format supported by the thin client acceleration channel. This results in a significant improvement in the quality of the streaming media displayed on the thin client and overall reductions in host CPU load, network bandwidth and thin client CPU load. No additional software is required on the thin clients to support new media types including Adobe Flash.

Claims

1. A method for transmitting a media stream between a local host and a remote rendering device comprising:

transmitting desktop information between the local host and the remote rendering device using a first channel using a remoting protocol;

identifying an uncompressed media stream that is being rendered on the local host, wherein the media stream is being rendered with the desktop information transmitted using the remoting protocol;

capturing bitmap content for the identified media stream after the media stream has been rendered on the local host, wherein the capturing prevents a remoting protocol from processing the rendered bitmap content for transmission to the remote rendering device, and wherein the capturing includes intercepting calls to render the identified media stream as bitmap content;

encoding the identified media stream using an encoder configured to encode streaming media and supported on both the local host and the remote rendering device, wherein encoding includes encapsulating the bitmap content in a software wrapper; and

transmitting the encoded media stream to the remote rendering device using a second channel comprising a media acceleration channel associated with the remoting protocol to cause the encoded content of the media stream to be decoded and displayed with the desktop information at the remote rendering device.

2. The method of claim 1 further comprising identifying the encoding scheme, by:

receiving information identifying codecs available on the remote rendering device; and

selecting, from the available codecs, a codec common to both the local host and the remote rendering device.

3. The method of claim 1 wherein encoding the identified media stream further comprises identifying one or more filters and invoking the identified one or more filters for encoding the identified media stream.

4. The method of claim 3 where the remote rendering device does not have native support for the identified media stream.

5. The method of claim 1 wherein the local host comprises a virtual desktop running in a virtual machine.

6. The method of claim 1 wherein the identified media stream is uncompressed and adapted for bitmap rendering, the method further comprising:

applying a timestamp to the encapsulated bitmap content; and

outputting the encapsulated bitmap content as an uncompressed media stream to the remote rendering device.

7. The method of claim 1 wherein outputting the encapsulated bitmap content further comprises invoking a filter on the encapsulated bitmap content.

8. A computer system, comprising:

a local host hosting a desktop environment, including a remoting protocol for transmitting a rendered desktop between the local host and a remote rendering device on a first channel over a computer network, the remoting protocol having an associated second channel including a media acceleration channel supporting an encoder configured to encode a media stream on the local host, the local host being configured to;

identify the media stream for the rendered desktop that is being rendered on the local host wherein the media stream is being rendered with the rendered desktop transmitted using the remoting protocol;

capture bitmap content for the identified media stream after the media stream has been rendered on the local host, wherein the capture prevents a remoting protocol from processing the rendered bitmap content for transmission to the remote rendering device, and wherein the capturing includes intercepting calls to render the identified media stream as bitmap content;

encode the identified media stream using the encoder configured to encode streaming media and supported by the remote rendering device, wherein encoding includes encapsulating the bitmap content in a software wrapper;

transmit desktop information between the local host and the remote rendering device using the first channel using the remoting protocol; and

transmit the encoded media stream to the remote rendering device using the media acceleration channel associated with the remoting protocol to cause the remote rendering device to decode and display the encoded content of the media stream with the desktop information.

9. The computer system of claim 8, wherein the local host is further configured to identify the encoding scheme.

10. The computer system of claim 9, wherein, to identify the encoding scheme, the local host is further configured to:

receive information identifying codecs available on the remote rendering device; and

select, from the available codecs, a codec common to both the local host and the remote rendering device.

11. The computer system of claim 8 wherein the local host comprises a virtual desktop running in a virtual machine.

12. The computer system of claim 8 wherein the identified media stream is uncompressed and adapted for bitmap rendering, and the local host is further configured to:

apply a timestamp to the encapsulated bitmap content; and

output the encapsulated bitmap content as an uncompressed media stream to the remote rendering device.

13. The computer system of claim 8, wherein to output the encapsulated bitmap content, the local host is further configured to invoke a filter on the encapsulated bitmap content.

14. A computer program product, comprising:

nontransitory computer readable storage including computer program instructions stored in locations on the computer readable storage for processing by a local host, wherein the local host hosts a desktop environment including a remoting protocol for transmitting a rendered desktop between the local host and a remote rendering device on a first channel over a computer network, wherein the remoting protocol has an associated second channel including a media acceleration channel supporting an encoder configured to encode streaming media, wherein the local host includes the encoder, wherein the local host, when processing the computer program instructions, is configured to;

identify a media stream for the rendered desktop that is being rendered on the local host, wherein the media stream is being rendered with the rendered desktop transmitted using the remoting protocol;

capture bitmap content for the identified media stream after the media stream has been rendered on the local host, wherein the capture prevents the remoting protocol from processing the rendered bitmap content for transmission to the remote rendering device, and wherein the capturing includes intercepting calls to render the identified media stream as bitmap content;

encode the identified media stream using the encoder configured to encode streaming media and supported by the remote rendering device, wherein encoding includes encapsulating the bitmap content in a software wrapper;

transmit desktop information between the local host and the remote rendering device using the first channel using the remoting protocol; and

transmit the encoded media stream to the remote rendering device using the media acceleration channel associated with the remoting protocol to cause the remote rendering device to decode and display the encoded content of the media stream with the desktop information.

15. The computer program product of claim 14, wherein the local host is further configured to identify the encoding scheme.

16. The computer program product of claim 15, wherein, to identify the encoding scheme, the local host is further configured to:

receive information identifying codecs available on the remote rendering device; and

select, from the available codecs, a codec common to both the local host and the remote rendering device.

17. The computer program product of claim 14 wherein the local host comprises a virtual desktop running in a virtual machine.

18. The computer program product of claim 14 wherein the identified media stream is uncompressed and adapted for bitmap rendering, and the local host is further configured to:

apply a timestamp to the encapsulated bitmap content; and

output the encapsulated bitmap content as an uncompressed media stream to the remote rendering device.