DECOMPOSED MULTI-STREAM (DMS) TECHNIQUES FOR VIDEO DISPLAY SYSTEMS

• US 20110145879A1
• Filed: 12/13/2010
• Published: 06/16/2011
• Est. Priority Date: 12/14/2009
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Wang, Xiaodong, Shaukat, Fawad, Rajamani, Krishnan
• Granted Patent: US 9,582,238 B2
• Time in Patent Office: Days
• US Class Current: 725/116
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G09G 2310/04 Partial updating of the display screen
• CPC Class Codes: G09G 2320/0613 The adjustment depending on the type of the information to be displayed
• CPC Class Codes: G09G 2320/10 Special adaptations of display systems for operation with variable images
• CPC Class Codes: G09G 2340/02 Handling of images in compressed format, e.g. JPEG, MPEG
• CPC Class Codes: G09G 2350/00 Solving problems of bandwidth in display systems
• CPC Class Codes: G09G 2352/00 Parallel handling of streams of display data
• CPC Class Codes: G09G 2370/16 Use of wireless transmission of display information
• CPC Class Codes: G09G 5/14 Display of multiple viewports
• CPC Class Codes: H04L 65/70 Media network packetisation

Abstract

This disclosure describes techniques that can improve the generation of a decomposed multi-stream (DMS) by a host device of a video display system and the display of a DMS by a client device of the video display system. The techniques may apply different frame rates to different streams within a DMS, and the frame rates may depend on the content. For example, one stream within a DMS may comprise a sequence of full-motion video information, which may be rendered at a relatively high frame rate. However, another stream within the DMS may be associated with a background of the display, various graphic user interface control windows or elements, or a display window that includes non-video content (such as e-mail or a document). The second stream in the DMS may be rendered at a much slower frame rate than that used for the sequence of full-motion video information.

Claims

1. A method comprising:

generating a decomposed multi-stream (DMS) via a host device of a video display system, wherein the DMS defines first content in a first area of a display window and second content in a second area of the display window, wherein the first content defines a first frame rate in the DMS and the second display content defines a second frame rate in the DMS, the first frame rate being different than the second frame rate; and

communicating the DMS from the host device to a client device of the video display system.

2. The method of claim 1, further comprising dynamically adjusting the first frame rate and the second frame rate in the DMS based on the first content and the second content.

3. The method of claim 1, wherein generating the DMS includes intercepting the first content from an application and capturing the second content from a display buffer in the host device.

4. The method of claim 3, further comprising dynamically reducing a capture rate associated with capturing the second content from the display buffer based on the first content and the second content.

5. The method of claim 1, wherein generating the DMS includes encoding the second content.

6. The method of claim 1, wherein the first area and the second area each include an overlapping area, the method further comprising:

generating information in the DMS that identifies which of the first content or the second content is on top in the overlapping area.

7. The method of claim 6, wherein generating the DMS includes reducing or eliminating data for either the first content or the second content in the overlapping area, wherein the reduced or eliminated data is below other data in the overlapping area.

8. The method of claim 1, wherein generating the DMS includes intercepting the first content or the second content from an application.

9. The method of claim 8, further comprising:

determining that intercepting the first content or the second content from the application is not possible; and

generating a signal composed stream (SCS) that includes the first content and the second content directly from a display buffer at the host device in response to determining that intercepting the first content or the second content from the application is not possible.

10. The method of claim 1, wherein the first content comprises a video sequence and the second content comprises output of a non-video application.

11. The method of claim 10, wherein the first frame rate is greater than the second frame rate.

12. The method of claim 1, wherein the first content comprises a video sequence and the second content comprises a graphical user interface element.

13. The method of claim 1, further comprising adjusting the DMS to only include one of the first content or the second content in response to user input.

14. The method of claim 1, further comprising:

determining a bandwidth available between the host device and the client device; and

adjusting one or both of the first frame rate and the second frame rate based on the bandwidth available.

15. The method of claim 1, further comprising:

determining a bandwidth available between the host device and the client device; and

dynamically encoding one or both of the first content and the second content based on the bandwidth available.

16. The method of claim 1, further comprising dynamically encoding one or both of the first content and the second content.

17. A method comprising:

receiving a decomposed multi-stream (DMS) at a client device of a video display system from a host device of the video display system, wherein the DMS defines first content in a first area of a display window and second content in a second area of the display window, wherein the first content defines a first frame rate in the DMS and the second display content defines a second frame rate in the DMS, the first frame rate being different than the second frame rate; and

rendering the first display content and the second display content on the client device.

18. The method of claim 17, wherein the first content comprises a video sequence and the second content comprises output of a non-video application.

19. The method of claim 18, wherein the first frame rate is greater than the second frame rate.

20. The method of claim 17, wherein the first content comprises a video sequence and the second content comprises a graphical user interface element.

21. The method of claim 17, further comprising:

buffering the first content differently than the second content in the client device.

22. The method of claim 21, further comprising applying different latency rules for display of the first content by the client device and display of the second content by the client device.

23. A host device of a video display system, the host device comprising:

a decomposed multi-stream (DMS) generator that generates a DMS, wherein the DMS defines first content in a first area of a display window and second content in a second area of the display window, wherein the first content defines a first frame rate in the DMS and the second display content defines a second frame rate in the DMS, the first frame rate being different than the second frame rate; and

a transport unit that communicates the DMS from the host device to a client device of the video display system.

24. The host device of claim 23, wherein the DMS generator dynamically adjusts the first frame rate and the second frame rate in the DMS based on the first content and the second content.

25. The host device of claim 23, wherein the DMS generator intercepts the first content from an application and captures the second content from a display buffer in the host device.

26. The host device of claim 25, wherein the DMS generator dynamically reduces a capture rate associated with capturing the second content from the display buffer based on the first content and the second content.

27. The host device of claim 25, wherein the DMS generator uses an encoder to encode the second content in the DMS.

28. The host device of claim 23, wherein the first area and the second area each include an overlapping area, wherein the DMS generator:

generates information in the DMS that identifies which of the first content or the second content is on top in the overlapping area.

29. The host device of claim 28, wherein the DMS generator reduces or eliminates data for either the first content or the second content in the overlapping area, wherein the reduced or eliminated data is below other data in the overlapping area.

30. The host device of claim 23, wherein the DMS generator intercepts the first content or the second content from an application.

31. The host device of claim 30, wherein the DMS generator:

determines that intercepting the first content or the second content from the application is not possible; and

generates a signal composed stream (SCS) that includes the first content and the second content directly from a display buffer at the host device in response to determining that intercepting the first content or the second content from the application is not possible.

32. The host device of claim 23, wherein the first content comprises a video sequence and the second content comprises output of a non-video application.

33. The host device of claim 32, wherein the first frame rate is greater than the second frame rate.

34. The host device of claim 23, wherein the first content comprises a video sequence and the second content comprises a graphical user interface element.

35. The host device of claim 23, wherein the DMS generator adjusts the DMS to only include one of the first content or the second content in response to user input.

36. The host device of claim 23, wherein the DMS generator:

determines a bandwidth available between the host device and the client device; and

adjusts one or both of the first frame rate and the second frame rate based on the bandwidth available.

37. The host device of claim 23, wherein the DMS generator:

determines a bandwidth available between the host device and the client device; and

dynamically encodes one or both of the first content and the second content based on the bandwidth available.

38. The host device of claim 23, wherein the DMS generator dynamically encodes one or both of the first content and the second content.

39. A client device of a video display system, the client device comprising:

a transport unit that receives a decomposed multi-stream (DMS) from a host device, wherein the DMS defines first content in a first area of a display window and second content in a second area of the display window, wherein the first content defines a first frame rate in the DMS and the second display content defines a second frame rate in the DMS, the first frame rate being different than the second frame rate;

a display unit; and

a DMS render unit that renders the first display content and the second display content on the display unit.

40. The client device of claim 39, wherein the first content comprises a video sequence and the second content comprises output of a non-video application.

41. The client device of claim 40, wherein the first frame rate is greater than the second frame rate.

42. The client device of claim 39, wherein the first content comprises a video sequence and the second content comprises a graphical user interface element.

43. The client device of claim 39, wherein the display unit includes buffers that are configured to buffer the first content differently than the second content.

44. The client device of claim 39, wherein the render unit is configured to apply different latency rules for display of the first content by the display unit, and display of the second content by the display unit.

45. A computer-readable storage medium comprising instructions that upon execution in a processor of a host device of a video display system, cause the host device to:

generate a decomposed multi-stream (DMS), wherein the DMS defines first content in a first area of a display window and second content in a second area of the display window, wherein the first content defines a first frame rate in the DMS and the second display content defines a second frame rate in the DMS, the first frame rate being different than the second frame rate; and

communicate the DMS from a host device to a client device of the video display system.

46. A computer-readable storage medium comprising instructions that upon execution in a processor of a client device of a video display system, cause the client device to:

upon receiving a decomposed multi-stream (DMS) from a host device of the video display system, wherein the DMS defines first content in a first area of a display window and second content in a second area of the display window, wherein the first content defines a first frame rate in the DMS and the second display content defines a second frame rate in the DMS, the first frame rate being different than the second frame rate,

render the first display content and the second display content on the client device

47. A host device of a video display system, the host device comprising:

means for generating a decomposed multi-stream (DMS), wherein the DMS defines first content in a first area of a display window and second content in a second area of the display window, wherein the first content defines a first frame rate in the DMS and the second display content defines a second frame rate in the DMS, the first frame rate being different than the second frame rate; and

means for communicating the DMS from the host device to a client device of the video display system.

48. A client device of a video display system, the client device comprising:

means for receiving a decomposed multi-stream (DMS) from a host device of the video display system, wherein the DMS defines first content in a first area of a display window and second content in a second area of the display window, wherein the first content defines a first frame rate in the DMS and the second display content defines a second frame rate in the DMS, the first frame rate being different than the second frame rate; and

means for rendering the first display content and the second display content on the client device.