Direct streaming for wireless display

• US 9,665,336 B2
• Filed: 12/22/2014
• Issued: 05/30/2017
• Est. Priority Date: 07/29/2014
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Gorrepati, Sateesh Naidu, Gupta, Devraj Gajraj, Karunakaran, Sanal Kumar, Agrawal, Sanjay Kumar
• Primary Examiner(s): Kumar, Pankaj
• Assistant Examiner(s): Andramuno, Franklin
• Application Number:
  US14/578,964
• Publication Number:
  US 20160034245A1
• Time in Patent Office: 890 Days
• Field of Search: 730350, 725 90
• CPC Class Codes: G06F 3/1407 General aspects irrespective of display type, e.g. determination of decimal point position, display with fixed or driving decimal point, suppression of non-significant zeros
• CPC Class Codes: G06F 3/1423 controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04N 21/43637 involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11] arrangements for wireless networking or broadcasting of information in indoor or near-field type systems H04B10/114

Abstract

A source device comprises a memory to store video data and one or more processors, wherein the one or more processors are configured to obtain encoded video data for a frame of video data; output, via a display interface for the source device and for display at a display device, a predefined frame for the frame of video data; and send, to a sink device via a wireless communication channel, the encoded video data for display at a display device associated with the sink device.

Claims

1. A method of transmitting video data from a wireless display source device to a wireless display sink device, the method comprising:

obtaining encoded video data for a frame of video data;

receiving, by a video codec, the encoded video data from a media player of the wireless display source device;

storing, by the video codec, a predefined frame to an output buffer for the media player;

outputting the predefined frame for the frame of video data;

receiving, by the video codec, an indication the output buffer has been consumed by the media player;

storing, by the video codec in response to receiving the indication, a timestamp that represents a presentation time for the predefined frame for the frame of video data; and

sending the encoded video data for display at a display device associated with the wireless display sink device in association with the timestamp.

2. The method of claim 1, further comprising:

determining a presentation time for the predefined frame for the frame of video data,

wherein sending the encoded video data comprises sending the encoded video data in association with the presentation time.

3. The method of claim 1, wherein the predefined frame is selected from the group consisting of a black frame and a user message.

4. The method of claim 1,

wherein the wireless display source device comprises a Wi-Fi Display source device and the wireless display sink device comprises a Wi-Fi Display sink device, and

wherein sending the encoded video data comprises sending the encoded video data via a communication session using one or more communications protocols based on the Wi-Fi Display standard.

5. The method of claim 1, wherein sending the encoded video data comprises sending the encoded video data via a communication session that operates according to one of the Wi-Fi Direct and tunneled direct link setup (TDLS) protocols.

6. The method of claim 1, further comprising:

receiving an indication of a user input;

in response to receiving the indication of the user input, establishing a mirroring session with the wireless display sink device,

wherein outputting the predefined frame for the frame of video data comprises outputting the predefined frame for the frame of video data only if the wireless display source device has an active mirroring session with the wireless display sink device.

7. The method of claim 1, wherein the video codec is registered for use by the media player and has an interface usable by the media player for providing encoded video frames to the video codec, the method further comprising:

loading the video codec for use by the media player in response to receiving an indication of a user input enabling a mirroring session with the wireless display sink device,

wherein receiving the encoded video data from the media player comprises receiving, by the video codec, the encoded video data via an input buffer, and

wherein storing the predefined frame to an output buffer for the media player comprises storing the predefined frame to the output buffer for the media player in association with any of metadata and timing information needed to cause the media player to display the predefined frame as if the predefined frame is a decoded video frame of the encoded video data.

8. The method of claim 1, further comprising:

determining a presentation time for audio data corresponding to the frame of video data;

sending encoded audio data for the audio data corresponding to the frame of video data in association with the presentation time for the audio data corresponding to the frame of video data; and

determining a presentation time for the predefined frame for the frame of video data,

wherein sending the encoded video data comprises sending the encoded video data in association with the presentation time for the predefined frame for the frame of video data to synchronize presentation of the audio data corresponding to the frame of video data and the frame of video data at the wireless display sink device.

9. The method of claim 1, wherein the indication the output buffer has been consumed by the media player comprises an indication that audio-video synchronization and rendering were performed by the media player.

10. A source device comprising:

a memory to store video data; and

one or more processors, wherein the one or more processors are configured to;

obtain encoded video data for a frame of video data;

output, via a display interface of the source device and for display at a display device, a predefined frame for the frame of video data;

load a video codec for use by a media player in response to receiving an indication of a user input enabling a mirroring session with the wireless display sink device, wherein the video codec is registered for use by the media player and has an interface usable by the media player for providing encoded video frames to the video codec;

receive, by a video codec of the source device via an input buffer, the encoded video data from a media player of the source device;

store, by the video codec, the predefined frame to an output buffer for the media player in association with any of metadata and timing information needed to cause the media player to display the predefined frame as if the predefined frame is a decoded video frame of the encoded video data; and

send, to a sink device via a wireless communication channel, the encoded video data for display at a display device associated with the sink device.

11. The source device of claim 10, wherein the one or more processors are further configured to:

determine a presentation time for the predefined frame for the frame of video data,

wherein to send the encoded video data the one or more processors are further configured to send, to the sink device, the encoded video data in association with the presentation time.

12. The source device of claim 10, wherein the predefined frame is selected from the group consisting of a black frame and a user message.

13. The source device of claim 10,

wherein the source device comprises a Wi-Fi Display source device and the sink device comprises a Wi-Fi Display sink device, and

wherein to send the encoded video data the one or more processors are further configured to send, to the sink device, the encoded video data via a communication session using one or more communications protocols based on the Wi-Fi Display standard.

14. The source device of claim 10, wherein to send the encoded video data the one or more processors are further configured to send, to the sink device, the encoded video data via a communication session that operates according to one of the Wi-Fi Direct and tunneled direct link setup (TDLS) protocols.

15. The source device of claim 10, wherein the one or more processors are further configured to:

receive, by the source device, an indication of user input; and

in response to receiving the indication of the user input, establish a mirroring session with the sink device,

wherein to output the predefined frame for the frame of video data the one or more processors are further configured to output the predefined frame for the frame of video data only if the source device has an active mirroring session with the sink device.

16. The source device of claim 10, wherein the one or more processors are further configured to:

receive, by the video codec, an indication the output buffer has been consumed by the media player; and

store, by the video codec in response to receiving the indication, a timestamp that represents a presentation time for the predefined frame for the frame of video data,

wherein to send the encoded video data the one or more processors are further configured to send, to the sink device, the encoded video data in association with the timestamp.

17. The source device of claim 10, wherein the one or more processors are further configured to:

determine a presentation time for audio data corresponding to the frame of video data;

send, to the sink device, encoded audio data for the audio data corresponding to the frame of video data in association with the presentation time for the audio data corresponding to the frame of video data; and

determine a presentation time for the predefined frame for the frame of video data,

wherein to send the encoded video data the one or more processors are further configured to send, to the sink device, the encoded video data in association with the presentation time for the predefined frame for the frame of video data to synchronize presentation of the audio data corresponding to the frame of video data and the frame of video data at the sink device.

18. A source device comprising:

means for obtaining encoded video data for a frame of video data;

means for outputting a predefined frame for the frame of video data;

means for loading the video codec for use by the media player in response to receiving an indication of a user input enabling a mirroring session with the wireless display sink device;

means for receiving by a video codec registered for use by the media player comprising an interface usable by the media player for providing encoded video frames to the video codec via an input buffer, the encoded video data from a media player of the wireless display source device;

means for storing, by the video codec, the predefined frame to an output buffer for the media player in association with any of metadata and timing information needed to cause the media player to display the predefined frame as if the predefined frame is a decoded video frame of the encoded video data; and

means for sending, to a sink device via a wireless communication channel, the encoded video data for display at a display device associated with the sink device.

19. The source device of claim 18, further comprising:

means for determining a presentation time for the predefined frame for the frame of video data,

wherein the means for sending the encoded video data comprises means for sending, to the sink device, the encoded video data in association with the presentation time.

20. The source device of claim 18, wherein the predefined frame is selected from the group consisting of a black frame and a user message.

21. The source device of claim 18,

wherein the source device comprises a Wi-Fi Display source device and the sink device comprises a Wi-Fi Display sink device, and

wherein the means for sending the encoded video data comprises means for sending, to the sink device, the encoded video data via a communication session using one or more communications protocols based on the Wi-Fi Display standard.

22. The source device of claim 18, wherein the means for sending the encoded video data comprises means for sending, to the sink device, the encoded video data via a communication session that operates according to one of the Wi-Fi Direct and tunneled direct link setup (TDLS) protocols.

23. The source device of claim 18, further comprising:

means for receiving an indication of a user input; and

means for, in response to receiving the indication of the user input, establishing a mirroring session with the sink device,

wherein the means for outputting the predefined frame for the frame of video data comprises means for outputting the predefined frame for the frame of video data only if the source device has an active mirroring session with the sink device.

24. The source device of claim 18, further comprising:

means for receiving, by the video codec of the source device, an indication the output buffer has been consumed by the media player; and

means for storing, by the video codec in response to receiving the indication, a timestamp that represents a presentation time for the frame of video data,

wherein the means for sending the encoded video data comprises means for sending, to the sink device, the encoded video data in association with the timestamp.

25. A non-transitory computer-readable storage medium comprising instructions stored thereon that, when executed, configure one or more processors of a wireless display source device to:

obtain encoded video data for a frame of video data;

receive, by a video codec, the encoded video data from a media player of the wireless display source device;

store, by the video codec, a predefined frame to an output buffer for the media player;

output a predefined frame for the frame of video data;

receiving, by the video codec, an indication the output buffer has been consumed by the media player;

storing, by the video codec in response to receiving the indication, a timestamp that represents a presentation time for the predefined frame for the frame of video data; and

send the encoded video data for display at a display device associated with a wireless display sink device in association with the timestamp.

26. The non-transitory computer-readable storage medium of claim 25, wherein the indication the output buffer has been consumed by the media player comprises an indication that audio-video synchronization and rendering were performed by the media player.