Content provisioning for wireless back channel

• US 10,108,386 B2
• Filed: 02/02/2012
• Issued: 10/23/2018
• Est. Priority Date: 02/04/2011
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Raveendran, Vijayalakshmi R., Wang, Xiaodong, Shaukat, Fawad, Huang, Xiaolong
• Primary Examiner(s): Nguyen, Vu
• Application Number:
  US13/364,576
• Publication Number:
  US 20130033496A1
• Time in Patent Office: 2,455 Days
• Field of Search: 345428
• CPC Class Codes: G06F 3/1407 General aspects irrespective of display type, e.g. determination of decimal point position, display with fixed or driving decimal point, suppression of non-significant zeros
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: G09G 2370/04 Exchange of auxiliary data, i.e. other than image data, between monitor and graphics controller
• CPC Class Codes: G09G 2370/16 Use of wireless transmission of display information

Abstract

In one example, this disclosure describes a method that includes, in an example, determining, by a source device, that display data rendered by the source device comprises one or more displayable graphical objects. The method also includes in response to determining an event, selecting, by the source device, a set of the one or more graphical objects for rendering at a sink device. The method also includes generating, by the source device, configuration data based at least in part on the set of the one or more graphical objects, wherein the configuration data is usable by the sink device to render representations of graphical objects in the set of one or more graphical objects based at least in part on one or more display parameters associated with the sink device. The method also includes sending, by the source device, the configuration data to the sink device.

Claims

1. A method comprising:

determining, by a source device, that display data rendered and displayed by the source device comprises one or more displayable graphical objects;

in response to input, provided at the source device, that corresponds to a set of the one or more graphical objects displayed by the source device, selecting, by the source device, the set of the one or more graphical objects for rendering and displaying at both the source device and a sink device, wherein a size of a display of the sink device is larger than a size of a display of the source device;

generating, by the source device, configuration data based at least in part on the set of the one or more graphical objects being rendered and displayed by the source device and based at least in part on one or more display parameters associated with the sink device, wherein the configuration data comprises one or more commands of a graphics programming language that are directly usable by a graphics processing unit (GPU) of the sink device to render pixel-level representations of graphical objects in the set of one or more graphical objects on the sink device; and

sending, by the source device, the configuration data to the sink device for rendering the set of one or more graphical objects at the sink device.

2. The method of claim 1, wherein the configuration data further comprises at least one of graphical commands to modify the set of one or more graphical objects for rendering at the sink device.

3. The method of claim 2, wherein the graphical commands do not comprise pixel values of a colorspace.

4. The method of claim 1, further comprising:

receiving, by the source device and from the sink device, a display resolution of the display of the sink device;

generating, by the source device, a mapping between the display resolution of the display of the sink device and a display resolution of the display of the source device; and

generating, by the source device, the configuration data based at least in part on the mapping between the display resolution of the display of the sink device and the display resolution of the display of the source device.

5. The method of claim 1, wherein generating configuration data based at least in part on the set of the one or more graphical objects further comprises:

generating, by the source device, configuration data to render the set of the one or more graphical objects only at the sink device.

6. The method of claim 1, wherein the one or more displayable graphical objects comprise a first group of displayable graphical objects and a second group of displayable graphical objects, wherein the method further comprises:

rendering, by the source device, the first group of displayable graphical objects, and wherein the second group of displayable graphical objects comprise the set of the one or more graphical objects for rendering at the sink device.

7. The method of claim 1, wherein the display of the source device comprises a touch sensitive display, the method further comprising:

receiving, at the touch sensitive display of the source device, a first indication of user input;

in response to receiving the first indication, generating, by the source device, a first event based at least in part on the first indication that is associated with a first timestamp;

receiving, by the source device and from the sink device, a second event that is associated with a second timestamp;

determining, by the source device, whether the second timestamp represents an earlier time than the first timestamp; and

when the second timestamp represents the earlier time than the first timestamp, generating, by the source device, configuration data based at least in part on the first timestamp and the second timestamp.

8. The method of claim 1, further comprising:

executing, by the source device, a first application;

determining, by the source device, an indication to execute the first application at the sink device;

in response to determining the indication, determining, by the source device, that a second instance of the application is executable by the sink device at the sink device; and

generating, by the source device, the configuration data to cause the sink device to execute the second instance of the application.

9. The method of claim 1, further comprising:

receiving, by the source device, information specifying processing capabilities of a graphics processing unit (GPU) of the sink device; and

generating, by the source device, the configuration data based at least in part on the capabilities of the GPU of the sink device.

10. A wireless source device comprising:

a display;

one or more processors; and

a non-transitory memory storing instructions that, when executed, cause the one or more processors to;

determine that display data rendered and displayed by the display of the source device comprises one or more displayable graphical objects;

in response to input, provided at the source device, that corresponds to a set of the one or more graphical objects displayed by the display of source device, select the set of the one or more graphical objects for rendering and displaying at both the source device and a sink device, wherein a size of a display of the sink device is larger than a size of the display of the source device;

generate configuration data based at least in part on the set of the one or more graphical objects being rendered and displayed by the source device and based at least in part on one or more display parameters associated with the sink device, wherein the configuration data comprises one or more commands of a graphics programming language that are directly usable by a graphics processing unit (GPU) of the sink device to render pixel-level representations of graphical objects in the set of one or more graphical objects on the sink device; and

send the configuration data to the sink device for rendering the set of one or more graphical objects at the sink device.

11. The wireless source device of claim 10, wherein the configuration data further comprises at least one of graphical commands to modify the set of one or more graphical objects for rendering at the sink device.

12. The wireless source device of claim 11, wherein the graphical commands do not comprise pixel values of a colorspace.

13. The wireless source device of claim 10, wherein, when executed, the instructions further cause the one or more processors to:

receive, from the sink device, a display resolution of the display of the sink device;

generate a mapping between the display resolution of the display of the sink device and a display resolution of the display of the source device; and

generate the configuration data based at least in part on the mapping between the display resolution of the display of the sink device and the display resolution of the display of the source device.

14. The wireless source device of claim 10, wherein, when executed, the instructions further cause the one or more processors to:

generate configuration data to render the set of the one or more graphical objects only at the sink device.

15. The wireless source device of claim 10, wherein, when executed, the instructions further cause the one or more processors to:

render the first group of displayable graphical objects, and wherein the second group of displayable graphical objects comprise the set of the one or more graphical objects for rendering at the sink device.

16. The wireless source device of claim 10, wherein the display of the source device comprises a touch sensitive display, and wherein, when executed, the instructions further cause the one or more processors to:

receive, at the touch sensitive display of the source device, a first indication of user input;

generate, in response to receiving the first indication, a first event based at least in part on the first indication that is associated with a first timestamp;

receive, from the sink device, a second event that is associated with a second timestamp;

determine whether the second timestamp represents an earlier time than the first timestamp; and

generate, when the second timestamp represents the earlier time than the first timestamp, configuration data based at least in part on the first timestamp and the second timestamp.

17. The wireless source device of claim 10, wherein, when executed, the instructions further cause the one or more processors to:

execute a first application;

determine an indication to execute the first application at the sink device;

determine, in response to determining the indication, that a second instance of the application is executable by the sink device at the sink device; and

generate the configuration data to cause the sink device to execute the second instance of the application.

18. The wireless source device of claim 10, wherein upon execution of the instructions the one or more processors further cause:

receiving information specifying processing capabilities of a graphics processing unit (GPU) of the sink device; and

generating the configuration data based at least in part on the capabilities of the GPU of the sink device.

19. A non-transitory computer-readable storage medium storing instructions that upon execution by one or more processors of a wireless source device cause the one or more processors to:

determine that display data rendered and displayed by the source device comprises one or more displayable graphical objects;

in response to input, provided at the source device, that corresponds to a set of the one or more graphical objects displayed by the source device, select the set of the one or more graphical objects for rendering and displaying at the sink device based at both the source device and a sink device, wherein a size of a display of the sink device is larger than a size of a display of the source device;

generate configuration data based at least in part on the set of the one or more graphical objects being rendered and displayed by the source device and based at least in part on one or more display parameters associated with the sink device, wherein the configuration data comprises one or more commands of a graphics programming language that are directly usable by a graphics processing unit (GPU) of the sink device to render pixel-level representations of graphical objects in the set of one or more graphical objects on the sink device; and

send the configuration data to the sink device for rendering the set of one or more graphical objects at the sink device.

20. The non-transitory computer readable storage medium of claim 19, wherein the configuration data further comprises at least one of graphical commands to modify the set of one or more graphical objects for rendering at the sink device.

21. The non-transitory computer readable storage medium of claim 20, wherein the graphical commands do not comprise pixel values of a colorspace.

22. The non-transitory computer readable storage medium of claim 19, storing instructions that upon execution by one or more processors of the wireless source device further cause the one or more processors to:

upon receiving, from the sink device, a display resolution of the display of the sink device, generate a mapping between the display resolution of the display of the sink device and a display resolution of the display of the source device; and

generate the configuration data based at least in part on the mapping between the display resolution of the display of the sink device and the display resolution of the display of the source device.

23. The non-transitory computer readable storage medium of claim 19, storing instructions that upon execution by one or more processors of the wireless source device further cause the one or more processors to generate configuration data to render the set of the one or more graphical objects only at the sink device.

24. The non-transitory computer readable storage medium of claim 19, storing instructions that upon execution by one or more processors of the wireless source device further cause the one or more processors to render the first group of displayable graphical objects, and wherein the second group of displayable graphical objects comprise the set of the one or more graphical objects for rendering at the sink device.

25. The non-transitory computer readable storage medium of claim 19, wherein the display of the source device comprises a touch sensitive display, and further storing instructions that upon execution by one or more processors of the wireless source device further cause the one or more processors to:

upon receiving, at the touch sensitive display of the source device, a first indication of user input, and in response to receiving the first indication, generate a first event based at least in part on the first indication that is associated with a first timestamp;

upon receiving, from the sink device, a second event that is associated with a second timestamp, determine whether the second timestamp represents an earlier time than the first timestamp; and

when the second timestamp represents the earlier time than the first timestamp, generate configuration data based at least in part on the first timestamp and the second timestamp.

26. The non-transitory computer readable storage medium of claim 19, storing instructions that upon execution by one or more processors of the wireless source device further cause the one or more processors to:

execute a first application;

determine an indication to execute the first application at the sink device;

in response to determining the indication, determine that a second instance of the application is executable by the sink device at the sink device; and

generate the configuration data to cause the sink device to execute the second instance of the application.

27. The non-transitory computer readable storage medium of claim 19, storing instructions that upon execution by one or more processors of the wireless source device further cause the one or more processors to:

upon receiving information specifying processing capabilities of a graphics processing unit (GPU) of the sink device, generate the configuration data based at least in part on the capabilities of the GPU of the sink device.

28. A wireless source device comprising:

means for determining that display data rendered and displayed by the source device comprises one or more displayable graphical objects;

means for selecting, in response to input, provided at the source device, that corresponds to a set of the one or more graphical objects displayed by the source device, the set of the one or more graphical objects for rendering and displaying at both the source device and the sink device based at a sink device, wherein a size of a display of the sink device is larger than a size of a display of the source device;

means for generating configuration data based at least in part on the set of the one or more graphical objects being rendered and displayed by the source device and based at least in part on one or more display parameters associated with the sink device, wherein the configuration data comprises one or more commands of a graphics programming language that are directly usable by a graphics processing unit (GPU) of the sink device to render pixel-level representations of graphical objects in the set of one or more graphical objects on the sink device; and

means for sending the configuration data to the sink device for rendering the set of one or more graphical objects at the sink device.

29. The wireless source device of claim 28, wherein the configuration data further comprises at least one of graphical commands to modify the set of one or more graphical objects for rendering at the sink device.

30. The wireless source device of claim 29, wherein the graphical commands do not comprise pixel values of a colorspace.

31. The wireless source device of claim 28, comprising:

means for receiving, from the sink device, a display resolution of the display of the sink device;

means for generating a mapping between the display resolution of the display of the sink device and a display resolution of the display of the source device; and

means for generating the configuration data based at least in part on the mapping between the display resolution of the display of the sink device and a display resolution of the display of the source device.

32. The wireless source device of claim 28, comprising:

means for generating configuration data to render the set of the one or more graphical objects only at the sink device.

33. The wireless source device of claim 28, comprising:

means for rendering the first group of displayable graphical objects, and wherein the second group of displayable graphical objects comprise the set of the one or more graphical objects for rendering at the sink device.

34. The wireless source device of claim 28, wherein the display of the source device comprises a touch sensitive display, the wireless source device further comprising:

means for receiving, at the touch sensitive display of the source device, a first indication of user input;

means for generating, in response to receiving the first indication, a first event based at least in part on the first indication that is associated with a first timestamp;

means for receiving a second event that is associated with a second timestamp;

means for determining whether the second timestamp represents an earlier time than the first timestamp; and

means for generating configuration data based at least in part on the first timestamp and the second timestamp, when the second timestamp represents the earlier time than the first timestamp.

35. The wireless source device of claim 28, comprising:

means for executing a first application;

means for determining an indication to execute the first application at the sink device;

means for determining, in response to determining the indication, that a second instance of the application is executable by the sink device at the sink device; and

means for generating the configuration data to cause the sink device to execute the second instance of the application.

36. The wireless source device of claim 28, comprising:

means for receiving information specifying processing capabilities of a graphics processing unit (GPU) of the sink device; and

means for generating the configuration data based at least in part on the capabilities of the GPU of the sink device.