User input back channel for wireless displays

• US 9,582,239 B2
• Filed: 01/05/2012
• Issued: 02/28/2017
• Est. Priority Date: 01/21/2011
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Raveendran, Vijayalakshmi R., Huang, Xiaolong, Wang, Xiaodong, Shaukat, Fawad
• Primary Examiner(s): Gray, Ryan M
• Application Number:
  US13/344,424
• Publication Number:
  US 20130009996A1
• Time in Patent Office: 1,881 Days
• Field of Search: 345/173, 345/660, 455/68
• US Class Current: 1/1
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G09G 2370/06 Consumer Electronics Control, i.e. control of another device by a display or vice versa
• CPC Class Codes: G09G 2370/10 Use of a protocol of communication by packets in interfaces along the display data pipeline
• CPC Class Codes: G09G 2370/16 Use of wireless transmission of display information

Abstract

As part of a communication session, a wireless source device can transmit audio and video data to a wireless sink device, and the wireless sink device can transmit user input data received at the wireless sink device back to the wireless source device. In this manner, a user of the wireless sink device can control the wireless source device and control the content that is being transmitted from the wireless source device to the wireless sink device. The input data received at the wireless sink device can have associated coordinate information that is scaled or normalized by either the wireless sink device or the wireless source device.

Claims

1. A method of transmitting user data from a wireless sink device to a wireless source device, the method comprising:

receiving from the wireless source device, an indication of a resolution of the wireless source device;

obtaining user input data at the wireless sink device, wherein the user input data has associated coordinate data;

at the wireless sink device, normalizing the associated coordinate data based on the resolution of the wireless source device and a resolution of the wireless sink device to generate normalized coordinate data;

generating a data packet comprising a data packet header and payload data, wherein the data packet header includes a value for an input category field indicating that the payload data comprises generic information elements, wherein the generic information elements describe the user input data according to an application-level protocol being executed by the wireless source device and the wireless sink device, and wherein the payload data includes one or more generic information elements comprising the normalized coordinate data, wherein the value is selected from a list of supported input categories, the list of supported input categories comprising the value for the generic input and a value indicating the payload data is formatted according to a type of human interface device command used to receive the input data; and

transmitting the data packet to a wireless source device from the wireless sink device.

2. The method of claim 1, further comprising:

determining if the associated coordinate data is within a display window for content being received from the wireless source device.

3. The method of claim 1, further comprising:

determining a resolution of a display window for content being received from the wireless source device.

4. The method of claim 3, wherein normalizing the coordinate data comprises scaling the associated coordinate data based on a ratio of the resolution of the display window and the resolution of the display of the source.

5. The method of claim 1, wherein the associated coordinate data corresponds to a location of a mouse click event.

6. The method of claim 1, wherein the associated coordinate data corresponds to a location of a touch event.

7. A wireless sink device for transmitting user data to a wireless source device, the wireless sink device comprising:

a memory storing instructions;

one or more processors configured to execute the instructions, wherein upon execution of the instructions the one or more processors cause;

receiving from the wireless source device, an indication of a resolution of the wireless source device;

obtaining user input data at the wireless sink device, wherein the user input data has associated coordinate data;

normalizing the associated coordinate data based on the resolution of the wireless source device and a resolution of the wireless sink device to generate normalized coordinate data; and

generating a data packet comprising a data packet header and payload data, wherein the data packet header includes a value for an input category field indicating that the payload data comprises generic information elements, wherein the generic information elements describe the user input data according to an application-level protocol being executed by the wireless source device and the wireless sink device, and wherein the payload data includes one or more generic information elements comprising the normalized coordinate data, wherein the value is selected from a list of supported input categories, the list of supported input categories comprising the value for the generic input and a value indicating the payload data is formatted according to a type of human interface device command used to receive the input data;

a transport unit to transmit the data packet to the wireless source device.

8. The wireless sink device of claim 7, wherein upon execution of the instructions the one or more processors further cause:

determining if the associated coordinate data is within a display window for content being received from the wireless source device.

9. The wireless sink device of claim 7, wherein upon execution of the instructions the one or more processors further cause:

determining a resolution of a display window for content being received from the wireless source device.

10. The wireless sink device of claim 9, wherein normalizing the coordinate data comprises scaling the associated coordinate data based on a ratio of the resolution of the display window and the resolution of the display of the source.

11. The wireless sink device of claim 7, wherein the associated coordinate data corresponds to a location of a mouse click event.

12. The wireless sink device of claim 7, wherein the associated coordinate data corresponds to a location of a touch event.

13. A non-transitory, computer-readable storage medium storing instructions that upon execution by one or more processors cause the one or more processors to:

receive from a wireless source device, an indication of a resolution of the wireless source device;

obtain user input data at a wireless sink device, wherein the user input data has associated coordinate data;

at the wireless sink device, normalize the associated coordinate data based on the resolution of the wireless source device and a resolution of the wireless sink device to generate normalized coordinate data;

generate a data packet comprising a data packet header and payload data, wherein the data packet header includes a value for an input category field indicating that the payload data comprises generic information elements, wherein the generic information elements describe the user input data according to an application-level protocol being executed by the wireless source device and the wireless sink device, and wherein the payload data includes one or more generic information elements comprising the normalized coordinate data, wherein the value is selected from a list of supported input categories, the list of supported input categories comprising the value for the generic input and a value indicating the payload data is formatted according to a type of human interface device command used to receive the input data; and

transmit the data packet to a wireless source device.

14. The non-transitory computer-readable storage medium of claim 13, storing further instructions that when executed by the one or more processors cause the one or more processors to:

determine if the associated coordinate data is within a display window for content being received from the wireless source device.

15. The non-transitory computer-readable storage medium of claim 13, storing further instructions that when executed by the one or more processors cause the one or more processors to:

determine a resolution of a display window for content being received from the wireless source device.

16. The non-transitory computer-readable storage medium of claim 13, wherein to normalize the coordinate data, the one or more processors scale the associated coordinate data based on a ratio of the resolution of the display window and the resolution of the display of the source.

17. The non-transitory computer-readable storage medium of claim 13, wherein the associated coordinate data corresponds to a location of a mouse click event.

18. The non-transitory computer-readable storage medium of claim 13, wherein the associated coordinate data corresponds to a location of a touch event.

19. A wireless sink device for transmitting user data to a wireless source device, the wireless sink device comprising:

means for receiving from the wireless source device, an indication of a resolution of the wireless source device;

means for obtaining user input data at the wireless sink device, wherein the user input data has associated coordinate data;

means for normalizing based on the resolution of the wireless source device and a resolution of the wireless sink device the associated coordinate data to generate normalized coordinate data;

means for generating a data packet comprising a data packet header and payload data, wherein the data packet header includes a value for an input category field indicating that the payload data comprises generic information elements, wherein the generic information elements describe the user input data according to an application-level protocol being executed by the wireless source device and the wireless sink device, and wherein the payload data includes one or more generic information elements comprising the normalized coordinate data, wherein the value is selected from a list of supported input categories, the list of supported input categories comprising the value for the generic input and a value indicating the payload data is formatted according to a type of human interface device command used to receive the input data; and

means for transmitting the data packet to a wireless source device.

20. The wireless sink device of claim 19, further comprising:

means for determining if the associated coordinate data is within a display window for content being received from the wireless source device.

21. The wireless sink device of claim 19, further comprising:

means for determining a resolution of a display window for content being received from the wireless source device.

22. The wireless sink device of claim 21, wherein the means for normalizing the coordinate data comprises means for scaling the associated coordinate data based on a ratio of the resolution of the display window and the resolution of the display of the source.

23. The wireless sink device of claim 19, wherein the associated coordinate data corresponds to a location of a mouse click event.

24. The wireless sink device of claim 19, wherein the associated coordinate data corresponds to a location of a touch event.