USER INPUT BACK CHANNEL FOR WIRELESS DISPLAYS

• US 20130003623A1
• Filed: 01/05/2012
• Published: 01/03/2013
• Est. Priority Date: 01/21/2011
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Raveendran, Vijayalakshmi R., Huang, Xiaolong, Wang, Xiaodong, Shaukat, Fawad
• Granted Patent: US 10,135,900 B2
• Time in Patent Office: Days
• US Class Current: 370/310
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04L 65/756 adapting media to device capabilities

Abstract

As part of a communication session, a wireless source device can transmit audio and video data to a wireless sink device, and the wireless sink device can transmit user inputs received at the wireless sink device back to the wireless source device. In this manner, a user of the wireless sink device can control the wireless source device and control the content that is being transmitted from the wireless source device to the wireless sink device. As part of transmitting user input to the source device, the sink device can generate a data packet with a data packet header that includes a timestamp flag indicating the presence of a timestamp field, and the source device can process the data packet based on a timestamp in the timestamp field.

Claims

1. A method of transmitting user input data from a wireless sink device to a wireless source device, the method comprising:

obtaining user input data at the wireless sink device;

generating a data packet header, wherein the data packet header comprises a timestamp flag to indicate if a timestamp field is present in the data packet header;

generating a data packet comprising the data packet header and payload data;

transmitting the data packet to the wireless source device.

2. The method of claim 1, wherein a timestamp entry in the timestamp field is based on multimedia data that was generated by the wireless source device and transmitted to the wireless sink device.

3. The method of claim 1, further comprising:

setting the timestamp flag to indicate that the timestamp field is present in the data packet header;

adding to the data packet header a timestamp in the timestamp field.

4. The method of claim 3, wherein the timestamp in the time stamp field identifies a timestamp associated with a frame of video data being displayed at the wireless sink device at the time the user input data was obtained.

5. The method of claim 4, wherein the timestamp associated with the frame of video data is added to the frame of video data by the wireless source device prior to being transmitted to the wireless sink device.

6. The method of claim 1, further comprising:

generating a second data packet header, wherein a timestamp flag of the second data packet header indicates a timestamp field for the second data packet header is not present in the second data packet header;

generating a second data packet comprising the second data packet header, wherein the second data packet header does not contain a timestamp field.

transmitting the second data packet to the wireless source device.

7. The method of claim 1, wherein the data packet header is an application layer packet header.

8. The method of claim 1, wherein the data packet is to control audio data or video data of the wireless source device.

9. The method of claim 1, wherein the data packet header further comprises a field to identify an input category of the user input data.

10. The method of claim 9, wherein the input category is selected from the group consisting of a generic command and an operating system (OS) specific command.

11. The method of claim 1, wherein the data packet header further comprises a field identifying a version of a protocol being used for communication.

12. The method of claim 1, wherein the data packet header further comprises a length field to indicate a length of the data packet.

13. The method of claim 12, wherein the length is identified in units of 16 bits.

14. The method of claim 1, wherein the data packet is transmitted to the wireless source device over a user input back channel (UIBC).

15. The method of claim 1, wherein the data packet is transmitted over TCP/IP.

16. The method of claim 1, wherein the timestamp flag is a 1 bit flag and the timestamp field is a 16 bit field.

17. A wireless sink device configured to transmit user input data to a wireless source device, the wireless sink device comprising:

a memory storing instructions;

one or more processors configured to execute the instructions, wherein upon execution of the instructions the one or more processors cause;

obtaining user input data at the wireless sink device;

generating a data packet header, wherein the data packet header comprises a timestamp flag to indicate if a timestamp field is present in the data packet header;

generating a data packet comprising the data packet header and payload data;

a transport unit to transmit the data packet to the wireless source device.

18. The wireless sink device of claim 17, wherein a timestamp entry in the timestamp field is based on multimedia data that was generated by the wireless source device and transmitted to the wireless sink device.

19. The wireless sink device of claim 17, further comprising:

setting the timestamp flag to indicate that the timestamp field is present in the data packet header;

adding to the data packet header a timestamp in the timestamp field.

20. The wireless sink device of claim 19, wherein the timestamp in the time stamp field identifies a timestamp associated with a frame of video data being displayed at the wireless sink device at the time the user input data was obtained.

21. The wireless sink device of claim 20, wherein the timestamp associated with the frame of video data is added to the frame of video data by the wireless source device prior to being transmitted to the wireless sink device.

22. The wireless sink device of claim 17, wherein upon execution of the instructions the one or more processors further cause:

generating a second data packet header, wherein a timestamp flag of the second data packet header indicates a timestamp field for the second data packet header is not present in the second data packet header;

generating a second data packet comprising the second data packet header, wherein the second data packet header does not contain a timestamp field.

transmitting the second data packet to the wireless source device.

23. The wireless sink device of claim 17, wherein the data packet header is an application layer packet header.

24. The wireless sink device of claim 17, wherein the data packet is to control audio data or video data of the wireless source device.

25. The wireless sink device of claim 17, wherein the data packet header further comprises a field to identify an input category of the user input data.

26. The wireless sink device of claim 25, wherein the input category is selected from the group consisting of a generic command and an operating system (OS) specific command.

27. The wireless sink device of claim 17, wherein the data packet header further comprises a field identifying a version of a protocol being used for communication.

28. The wireless sink device of claim 17, wherein the data packet header further comprises a length field to indicate a length of the data packet.

29. The wireless sink device of claim 28, wherein the length is identified in units of 16 bits.

30. The wireless sink device of claim 17, wherein the data packet is transmitted to the wireless source device over a user input back channel (UIBC).

31. The wireless sink device of claim 17, wherein the data packet is transmitted over TCP/IP.

32. The wireless sink device of claim 17, wherein the timestamp flag is a 1 bit flag and the timestamp field is a 16 bit field.

33. A computer-readable storage medium storing instructions that upon execution by one or more processors cause the one or more processors to perform a method of transmitting user input data from a wireless sink device to a wireless source device, the method comprising:

obtaining user input data at the wireless sink device;

generating a data packet header, wherein the data packet header comprises a timestamp flag to indicate if a timestamp field is present in the data packet header;

generating a data packet comprising the data packet header and payload data;

transmitting the data packet to the wireless source device.

34. A wireless sink device configured to transmit user input data to a wireless source device, the wireless sink device comprising:

means for obtaining user input data at the wireless sink device;

means for generating a data packet header, wherein the data packet header comprises a timestamp flag to indicate if a timestamp field is present in the data packet header;

means for generating a data packet comprising the data packet header and payload data; and

means for transmitting the data packet to the wireless source device.

35. A method of receiving user input data from a wireless sink device at a wireless source device, the method comprising:

receiving, from the wireless sink device, a data packet comprising a data packet header and payload data;

parsing the data packet header to determine if a timestamp field is present in the data packet header;

in response to the timestamp field being present, processing the payload data based on a timestamp in the timestamp field.

36. The method of claim 35, wherein the timestamp is based on multimedia data that was generated by the wireless source device and transmitted to the wireless sink device.

37. The method of claim 35, further comprising:

in response to the timestamp field not being present, processing the payload data without a timestamp.

38. The method of claim 35, wherein parsing the data packet header comprises identifying a timestamp flag in the data packet header.

39. The method of claim 38, wherein the timestamp flag is a 1 bit flag and the timestamp field is a 16 bit field.

40. The method of claim 35 wherein the timestamp field identifies a timestamp associated with a frame of video data being displayed at the wireless sink device at the time a user input data was obtained.

41. The method of claim 40, wherein the timestamp was added to the frame of video data by the wireless source device prior to being transmitted to the wireless sink device.

42. The method of claim 35, wherein processing the payload data based on the timestamp in timestamp field comprises:

based on the timestamp, identifying a frame of video data being displayed at the wireless sink device at the time a user input data was obtained, wherein the payload data describes the user input data;

interpreting the payload data based on content of the frame.

43. The method of claim 35, wherein processing the payload data based on the timestamp comprises comparing the timestamp of the timestamp field to a current timestamp for a current frame of video being transmitted by the wireless source device.

44. The method of claim 43, wherein processing the payload data based on the timestamp further comprises performing a command described in the payload data in response to a time difference between the timestamp and the current timestamp being less than a threshold value.

45. The method of claim 43, wherein processing the payload data based on the timestamp further comprises not performing a command described in the payload data in response to a time difference between the timestamp and the current timestamp being greater than a threshold value.

46. The method of claim 35, wherein the data packet header is an application layer packet header.

47. The method of claim 35, wherein the data packet is to control audio data or video data of the wireless source device.

48. The method of claim 35, wherein the data packet header further comprises a field to identify an input category of the user input data.

49. The method of claim 48, wherein the input category is selected from the group consisting of a generic command and an operating system (OS) specific command.

50. The method of claim 35, wherein the data packet header further comprises a field identifying a version of a protocol being used for communication.

51. The method of claim 35, wherein the data packet header further comprises a length field to indicate a length of the data packet.

52. The method of claim 51, wherein the length is identified in units of 16 bits.

53. The method of claim 35, wherein the data packet is transmitted to the wireless source device over a user input back channel (UIBC).

54. The method of claim 35, wherein the data packet is transmitted over TCP/IP.

55. A wireless source device configured to receive user input data from a wireless sink device, the wireless source device comprising:

a transport unit to receive, from the wireless sink device, a data packet comprising a data packet header and payload data;

a memory storing instructions;

one or more processors configured to execute the instructions, wherein upon execution of the instructions the one or more processors cause;

parsing the data packet header to determine if a timestamp field is present in the data packet header;

in response to the timestamp field being present, processing the payload data based on a timestamp in the timestamp field.

57. The wireless source device of claim 55, wherein upon execution of the instructions the one or more processors further cause:

in response to the timestamp field not being present, processing the payload data without a timestamp.

58. The wireless source device of claim 55, wherein parsing the data packet header comprises identifying a timestamp flag in the data packet header.

59. The wireless source device of claim 58, wherein the timestamp flag is a 1 bit flag and the timestamp field is a 16 bit field.

60. The wireless source device of claim 55, wherein the timestamp field identifies a timestamp associated with a frame of video data being displayed at the wireless sink device at the time a user input data was obtained.

61. The wireless source device of claim 60, wherein the timestamp was added to the frame of video data by the wireless source device prior to being transmitted to the wireless sink device.

62. The wireless source device of claim 55, wherein processing the payload data based on the timestamp in timestamp field comprises:

based on the timestamp, identifying a frame of video data being displayed at the wireless sink device at the time a user input data was obtained, wherein the payload data describes the user input data; and

interpreting the payload data based on content of the frame.

63. The wireless source device of claim 55, wherein processing the payload data based on the timestamp comprises comparing the timestamp of the timestamp field to a current timestamp for a current frame of video being transmitted by the wireless source device.

64. The wireless source device of claim 63, wherein processing the payload data based on the timestamp further comprises performing a command described in the payload data in response to a time difference between the timestamp and the current timestamp being less than a threshold value.

65. The wireless source device of claim 63 wherein processing the payload data based on the timestamp further comprises not performing a command described in the payload data in response to a time difference between the timestamp and the current timestamp being greater than a threshold value.

66. The wireless source device of claim 55, wherein the data packet header is an application layer packet header.

67. The wireless source device of claim 55, wherein the data packet is to control audio data or video data of the wireless source device.

68. The wireless source device of claim 55, wherein the data packet header further comprises a field to identify an input category of the user input data.

69. The wireless source device of claim 68, wherein the input category is selected from the group consisting of a generic command and an operating system (OS) specific command.

70. The wireless source device of claim 55, wherein the data packet header further comprises a field identifying a version of a protocol being used for communication.

71. The wireless source device of claim 55, wherein the data packet header further comprises a length field to indicate a length of the data packet.

72. The wireless source device of claim 71, wherein the length is identified in units of 16 bits.

73. The wireless source device of claim 65, wherein the data packet is transmitted to the wireless source device over a user input back channel (UIBC).

74. The wireless source device of claim 65, wherein the data packet is transmitted over TCP/IP.

56. The wireless source device 55, wherein the timestamp is based on multimedia data that was generated by the wireless source device and transmitted to the wireless sink device.

75. A computer-readable storage medium storing instructions that upon execution by one or more processors cause the one or more processors to perform a method of receiving user input data from a wireless sink device at a wireless source device, the method comprising:

receiving, from the wireless sink device, a data packet comprising a data packet header and payload data;

parsing the data packet header to determine if a timestamp field is present in the data packet header;

in response to the timestamp field being present, processing the payload data based on a timestamp in the timestamp field.

76. A wireless source device configured to receive user input data from a wireless sink device, the wireless source device comprising:

means for receiving, from the wireless sink device, a data packet comprising a data packet header and payload data;

means for parsing the data packet header to determine if a timestamp field is present in the data packet header;

means for processing, based on a timestamp in the timestamp field, the payload data in response to the timestamp field being present.