Media acceleration for virtual computing services

• US 9,973,557 B2
• Filed: 07/29/2016
• Issued: 05/15/2018
• Est. Priority Date: 04/15/2008
• Status: Active Grant

• Current Assignee: VMware, Inc. (Broadcom, Inc.), Vmware LLC (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: Hobgood, Andrew W., Baron, Steve, Battersby, Clinton B.
• Primary Examiner(s): Dang, Hung
• Assistant Examiner(s): Park, Sunghyoun
• Application Number:
  US15/224,082
• Publication Number:
  US 20160337420A1
• Time in Patent Office: 655 Days
• CPC Class Codes: G06F 9/5061 Partitioning or combining of resources
• CPC Class Codes: H04L 12/4654 wherein a VLAN tag represents a customer VLAN, e.g. C-Tag
• CPC Class Codes: H04L 45/02 Topology update or discovery
• CPC Class Codes: H04L 45/50 using label swapping, e.g. multi-protocol label switch [MPLS]
• CPC Class Codes: H04L 47/70 Admission control; Resource allocation
• CPC Class Codes: H04L 49/70 Virtual switches
• CPC Class Codes: H04L 63/02 for separating internal from external traffic, e.g. firewalls
• CPC Class Codes: H04L 63/0272 Virtual private networks
• CPC Class Codes: H04L 63/0281 Proxies
• CPC Class Codes: H04L 63/08 for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 63/0815 providing single-sign-on or federations
• CPC Class Codes: H04L 63/10 for controlling access to devices or network resources
• CPC Class Codes: H04L 65/61 for supporting one-way streaming services, e.g. Internet radio
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 65/764 at the destination reformatting of additional data in video clients H04N21/4355
• CPC Class Codes: H04L 67/08 specially adapted for terminal emulation, e.g. Telnet
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]

Abstract

Streaming media is problematic for thin clients using remoting protocols like RDP that were never designed to handle the volume of data associated with multimedia. The result is large demands on the host computer and thin client CPU and excessive bandwidth on the network, which results in a poor display quality. A process running on a host computer detects an existing multimedia acceleration channel to a thin client and also identifies unaccelerated media streams like Adobe Flash. The unaccelerated content is automatically re-encoded using a codec format supported by the thin client acceleration channel. This results in a significant improvement in the quality of the streaming media displayed on the thin client and overall reductions in host CPU load, network bandwidth and thin client CPU load. No additional software is required on the thin clients to support new media types including Adobe Flash.

Claims

1. A method comprising:

identifying a media stream that is being rendered locally on a local host for transmission to a remote rendering device;

detecting existence of a media acceleration channel associated with a remoting protocol, wherein the remoting protocol transmits desktop information between the local host and the remote rendering device using a different channel;

intercepting calls to transmit bitmap content for the identified media stream after the media stream has been rendered on the local host and redirecting the bitmap content of the media stream for transmission using the media acceleration channel;

identifying an encoding scheme supported on both the local host and the remote rendering device;

encoding the bitmap content for the identified media stream using the identified encoding scheme; and

transmitting the encoded bitmap content for the identified media stream to the remote rendering device for decoding using the media acceleration channel associated with the remoting protocol.

2. The method of claim 1, wherein identifying the encoding scheme further comprises:

determining a type of the remote rendering device on which the identified media stream is to be rendered;

identifying codecs available on the remote rendering device for decoding received media streams; and

selecting, from the available codecs, a codec common to both the local host and the remote rendering device.

3. The method of claim 1, wherein encoding the bitmap content for the identified media stream further comprises identifying filters and invoking the identified filters for encoding the bitmap content for the identified media stream.

4. The method of claim 1, wherein the remoting protocol is prevented from processing the encoded bitmap content on the local host.

5. The method of claim 1, wherein the local host comprises a virtual desktop running in a virtual machine.

6. The method of claim 1, wherein the remote rendering device does not have native support for the identified media stream.

7. The method of claim 1, wherein the identified media stream is in an uncompressed form adapted for bitmap rendering, and wherein the method further comprises:

encapsulating the media bitmap content in the identified calls in a software wrapper.

8. The method of claim 7, further comprising:

applying a timestamp to the encapsulated bitmap content; and

outputting the encapsulated bitmap content as an uncompressed media stream to the remote rendering device.

9. The method of claim 8, wherein outputting the encapsulated bitmap content further comprises invoking a filter on the encapsulated bitmap content.

10. A system, comprising:

a local host;

a remote rendering device connected to the local host over a computer network;

the local host hosting a desktop environment, including a remoting protocol for transmitting a rendered desktop between the local host and the remote rendering device over the computer network, the remoting protocol having an associated media acceleration channel supporting an encoding scheme;

the local host being configured to;

identify a media stream that is being rendered locally on the local host;

intercepting calls to transmit, using a different channel, bitmap content for the identified media stream after the media stream has been rendered on the local host and redirecting the bitmap content of the media stream for transmission using the media acceleration channel;

encode the bitmap content for the identified media stream using the encoder, wherein the encoding is based on an identified encoding scheme supported by the local host and the remote rendering device; and

transmit the encoded bitmap content for the identified media stream to the remote rendering device using the media acceleration channel associated with the remoting protocol.

11. The system of claim 10, wherein identifying the encoding scheme further comprises:

determining a type of the remote rendering device on which the identified media stream is to be rendered;

identifying codecs available on the remote rendering device for decoding received media streams; and

selecting, from the available codecs, a codec common to both the local host and the remote rendering device.

12. The system of claim 10, wherein encoding the bitmap content for the identified media stream further comprises identifying filters and invoking the identified filters for encoding the bitmap content for the identified media stream.

13. The system of claim 10, wherein the remoting protocol is prevented from processing the encoded bitmap content on the local host.

14. The system of claim 10, wherein the local host comprises a virtual desktop running in a virtual machine.

15. The system of claim 10, wherein the remote rendering device does not have native support for the identified media stream.

16. The system of claim 10, wherein the identified media stream is in an uncompressed form adapted for bitmap rendering, and wherein the method further comprises:

encapsulating the media bitmap content in the identified calls in a software wrapper.

17. The system of claim 16, further comprising:

applying a timestamp to the encapsulated bitmap content; and

outputting the encapsulated bitmap content as an uncompressed media stream to the remote rendering device.

18. The system of claim 17, wherein outputting the encapsulated bitmap content further comprises invoking a filter on the encapsulated bitmap content.

19. The system of claim 10, wherein the remote rendering device is configured to decode the encoded bitmap content using a decoder to provide decoded media content and the display the decoded media content.

20. A computer program product comprising nontangible computer readable storage including instructions that when executed are configured to perform operations comprising:

identifying a media stream that is being rendered locally on a local host for transmission to a remote rendering device;

detecting existence of a media acceleration channel associated with a remoting protocol, wherein the remoting protocol transmits desktop information between the local host and the remote rendering device using a different channel;

intercepting calls to transmit bitmap content for the identified media stream after the media stream has been rendered on the local host and redirecting the bitmap content of the media stream for transmission using the media acceleration channel;

identifying an encoding scheme supported on both the local host and the remote rendering device;

encoding the bitmap content for the identified media stream using the identified encoding scheme; and

transmitting the encoded bitmap content for the identified media stream to the remote rendering device for decoding using the media acceleration channel associated with the remoting protocol.