User input back channel for wireless displays

• US 10,382,494 B2
• Filed: 10/06/2017
• Issued: 08/13/2019
• Est. Priority Date: 01/21/2011
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Huang, Xiaolong, Raveendran, Vijayalakshmi R, Wang, Xiaodong, Shaukat, Fawad
• Primary Examiner(s): Yao, Kwang B
• Assistant Examiner(s): Ngo, Nguyen H
• Application Number:
  US15/726,452
• Publication Number:
  US 20180048679A1
• Time in Patent Office: 676 Days
• Field of Search: 370230, 370236, 370252, 370310, 370349, 370389, 370392, 370471, 370474
• CPC Class Codes: H04L 65/00 Network arrangements, protocols or services for supporting real-time applications in data packet communication real-time or near real-time messaging, e.g. instant messaging [IM] H04L51/04; selective video distribution H04N21/00
• CPC Class Codes: H04L 69/24 Negotiation of communication capabilities
• CPC Class Codes: H04W 99/00 Subject matter not provided for in other groups of this subclass

Abstract

As part of a communication session, a wireless source device can transmit audio and video data to a wireless sink device, and the wireless sink device can transmit user input data received at the wireless sink device back to the wireless source device. In this manner, a user of the wireless sink device can control the wireless source device and control the content that is being transmitted from the wireless source device to the wireless sink device. The user input data transmitted by the wireless sink device can be input data obtained at a third party device and forwarded to the wireless source device.

Claims

1. A method of transmitting user input data from a wireless sink device to a wireless source device, the method comprising:

obtaining, by the wireless sink device and from an external device, user input data captured while a particular frame of video data received from the wireless source device is displayed;

generating, by the wireless sink device, a data packet header for a data packet, wherein the data packet header includes;

a four-bit input category field; and

a timestamp flag that indicates whether or not the data packet header includes a sixteen-bit timestamp field;

including, by the wireless sink device, a representation of a timestamp of the particular frame of video data in the timestamp field of the data packet header when the timestamp flag indicates that the data packet header includes the timestamp field;

generating, by the wireless sink device, payload data comprising the user input data;

generating, by the wireless sink device, the data packet comprising the data packet header and the payload data; and

transmitting, by the wireless sink device, the data packet to the wireless source device.

2. The method of claim 1, wherein the user input data is received from a user input interface of the external device selected from a group consisting of a touch screen, a microphone, and a camera.

3. The method of claim 1, wherein the user input data is data associated with video captured by a camera.

4. The method of claim 1, wherein the user input data is data associated with a voice command.

5. The method of claim 1, wherein the user input data is data associated with a multi-touch event.

6. The method of claim 1, wherein the user input data is data associated with a user gesture.

7. The method of claim 1, wherein a value of the input category field is set to indicate the payload data comprises forwarded user input data.

8. The method of claim 1, wherein the external device is an input device communicatively coupled to the wireless sink device.

9. The method of claim 1, wherein the data packet header is an application layer packet header.

10. The method of claim 1, wherein the data packet is configured to control audio data or video data of the wireless source device.

11. A wireless sink device, comprising:

a processor configured with processor-executable instructions to;

obtain, from an external device, user input data captured while a particular frame of video data received from a wireless source device is displayed;

generate a data packet header for a data packet, wherein the data packet header includes;

a four-bit input category field; and

a timestamp flag that indicates whether or not the data packet header includes a sixteen-bit timestamp field;

include a representation of a timestamp of the particular frame of video data in the timestamp field of the data packet header when the timestamp flag indicates that the data packet header includes the timestamp field;

generate payload data comprising the user input data;

generate the data packet comprising the data packet header and the payload data; and

transmit the data packet to the wireless source device.

12. The wireless sink device of claim 11, wherein the processor is further configured with processor-executable instructions such that the user input data is received from a user input interface of the external device selected from a group consisting of a touch screen, a microphone, and a camera.

13. The wireless sink device of claim 11, further comprising a camera coupled to the processor, wherein the processor is further configured with processor-executable instructions such that the user input data is data associated with video captured by the camera.

14. The wireless sink device of claim 11, wherein the processor is further configured with processor-executable instructions such that the user input data is data associated with a voice command.

15. The wireless sink device of claim 11, wherein the processor is further configured with processor-executable instructions such that the user input data is data associated with a multi-touch event.

16. The wireless sink device of claim 11, wherein the processor is further configured with processor-executable instructions such that the user input data is data associated with a user gesture.

17. The wireless sink device of claim 11, wherein the processor is further configured with processor-executable instructions such that a value of the input category field is set to indicate the payload data comprises forwarded user input data.

18. The wireless sink device of claim 11, wherein the external device is an input device communicatively coupled to the wireless sink device.

19. The wireless sink device of claim 11, wherein the processor is further configured with processor-executable instructions such that the data packet header is an application layer packet header.

20. The wireless sink device of claim 11, wherein the processor is further configured with processor-executable instructions such that the data packet is configured to control audio data or video data of the wireless source device.

21. A wireless sink device, comprising:

means for obtaining, from an external device, user input data captured while a particular frame of video data received from a wireless source device is displayed;

means for generating a data packet header for a data packet, wherein the data packet header includes;

a four-bit input category field; and

a timestamp flag that indicates whether or not the data packet header includes a sixteen-bit timestamp field;

means for including a representation of a timestamp of the particular frame of video data in the timestamp field of the data packet header when the timestamp flag indicates that the data packet header includes the timestamp field;

means for generating payload data comprising the user input data;

means for generating the data packet comprising the data packet header and the payload data; and

means for transmitting the data packet to the wireless source device.

22. The wireless sink device of claim 21, wherein means for obtaining the user input data comprises means for receiving the user input data from a user input interface of the external device selected from a group consisting of a touch screen, a microphone, and a camera.

23. The wireless sink device of claim 21, further comprising means for capturing video, wherein the user input data is data associated with video.

24. The wireless sink device of claim 21, wherein the user input data is data associated with a voice command.

25. The wireless sink device of claim 21, wherein the user input data is data associated with a multi-touch event.

26. The wireless sink device of claim 21, wherein the user input data is data associated with a user gesture.

27. The wireless sink device of claim 21, wherein a value of the input category field is set to indicate the payload data comprises forwarded user input data.

28. The wireless sink device of claim 21, wherein the external device is an input device communicatively coupled to the wireless sink device.

29. The wireless sink device of claim 21, wherein means for generating a data packet header for a data packet comprises means for generating an application layer packet header.

30. The wireless sink device of claim 21, wherein means for generating a data packet header for a data packet comprises means for generating a data packet configured to control audio data or video data of the wireless source device.

31. A non-transitory processor readable storage medium having stored thereon processor-executable instructions configured to cause a processor of a wireless sink device to perform operations, comprising:

obtaining, from an external device, user input data captured while a particular frame of video data received from a wireless source device is displayed;

generating a data packet header for a data packet, wherein the data packet header includes;

a four-bit input category field; and

a timestamp flag that indicates whether or not the data packet header includes a sixteen-bit timestamp field;

including a representation of a timestamp of the particular frame of video data in the timestamp field of the data packet header when the timestamp flag indicates that the data packet header includes the timestamp field;

generating payload data comprising the user input data;

generating the data packet comprising the data packet header and the payload data; and

transmitting the data packet to the wireless source device.

32. The non-transitory processor readable storage medium of claim 31, wherein the stored processor-executable instructions are configured to cause a processor of a wireless sink device to perform operations such that the user input data is received from a user input interface of the external device selected from a group consisting of a touch screen, a microphone, and a camera.

33. The non-transitory processor readable storage medium of claim 31, wherein the stored processor-executable instructions are configured to cause a processor of a wireless sink device to perform operations such that the user input data is data associated with video captured by a camera.

34. The non-transitory processor readable storage medium of claim 31, wherein the stored processor-executable instructions are configured to cause a processor of a wireless sink device to perform operations such that the user input data is data associated with a voice command.

35. The non-transitory processor readable storage medium of claim 31, wherein the stored processor-executable instructions are configured to cause a processor of a wireless sink device to perform operations such that the user input data is data associated with a multi-touch event.

36. The non-transitory processor readable storage medium of claim 31, wherein the stored processor-executable instructions are configured to cause a processor of a wireless sink device to perform operations such that the user input data is data associated with a user gesture.

37. The non-transitory processor readable storage medium of claim 31, wherein the stored processor-executable instructions are configured to cause a processor of a wireless sink device to perform operations such that a value of the input category field is set to indicate the payload data comprises forwarded user input data.

38. The non-transitory processor readable storage medium of claim 31, wherein the stored processor-executable instructions are configured to cause a processor of a wireless sink device to perform operations such that the external device is an input device communicatively coupled to the wireless sink device.

39. The non-transitory processor readable storage medium of claim 31, wherein the stored processor-executable instructions are configured to cause a processor of a wireless sink device to perform operations such that the data packet header is an application layer packet header.

40. The non-transitory processor readable storage medium of claim 31, wherein the stored processor-executable instructions are configured to cause a processor of a wireless sink device to perform operations such that the data packet is configured to control audio data or video data of the wireless source device.