Preparing video data in accordance with a wireless display protocol

• US 9,131,279 B2
• Filed: 05/13/2010
• Issued: 09/08/2015
• Est. Priority Date: 08/06/2009
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Raveendran, Vijayalakshmi R., Martinez Bauza, Judit, Soliman, Samir S.
• Primary Examiner(s): Senfi, Behrooz
• Assistant Examiner(s): VAZQUEZ COLON, MARIA E
• Application Number:
  US12/779,661
• Publication Number:
  US 20110032334A1
• Time in Patent Office: 1,944 Days
• Field of Search: 348/42, 348/43, 348/46, 348/51, 348/739, 348/E13.001, 348/E13.074, 348/E13.075, 348/E13.003, 348/E13.02, 348/E13.027, 345/419
• US Class Current: 1/1
• CPC Class Codes: H04N 13/10 Processing, recording or transmission of stereoscopic or multi-view image signals
• CPC Class Codes: H04N 13/139 Format conversion, e.g. of frame-rate or size
• CPC Class Codes: H04N 21/2381 Adapting the multiplex stream to a specific network, e.g. an Internet Protocol [IP] network transmission of MPEG streams over ATM H04L12/5601
• CPC Class Codes: H04N 21/4122 additional display device, e.g. video projector digital output for controlling a plurality of local displays G06F3/1423
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/4363 Adapting the video stream to a specific local network, e.g. a Bluetooth® network
• CPC Class Codes: H04N 21/43637 involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11] arrangements for wireless networking or broadcasting of information in indoor or near-field type systems H04B10/114
• CPC Class Codes: H04N 21/4381 Recovering the multiplex stream from a specific network, e.g. recovering MPEG packets from ATM cells transmission of MPEG streams over ATM H04L12/5601
• CPC Class Codes: H04N 21/440218 by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 conversion of standards in analogue television systems H04N7/01
• CPC Class Codes: H04N 21/44231 Monitoring of peripheral device or external card, e.g. to detect processing problems in a handheld device or the failure of an external recording device configuring of peripheral devices in general G06F9/4411; reporting information sensed by appliance or service execution status of appliance services in a home automation network H04L12/2823; monitoring the status of connected device in data switched networks H04L43/0817
• CPC Class Codes: H04N 21/6437 Real-time Transport Protocol [RTP]
• CPC Class Codes: H04N 21/816 involving special video data, e.g 3D video

Abstract

In general, techniques are described for preparing video data in accordance with a wireless display protocol. For example, a portable device comprising a module to store video data, a wireless display host module and a wireless interface may implement the techniques of this disclosure. The wireless display host module determines one or more display parameters of a three-dimensional (3D) display device external from the portable device and prepares the video data to generate 3D video data based on the determined display parameters. The wireless interface then wirelessly transmits the 3D video data to the external 3D display device. In this way, a portable device implements the techniques to prepare video data in accordance with a wireless display protocol.

Claims

1. A portable device comprising:

a module to store video data;

a wireless interface;

a wireless display host module that;

wirelessly communicates with a three-dimensional (3D) display device external from the portable device via the wireless interface using a wireless display protocol;

determines one or more display parameters of the 3D display device based on the wireless communication with the 3D display device; and

prepares the stored video data in accordance with the wireless display protocol to generate 3D video data based on the determined display parameters,

a transformation module that transforms the video data to generate the 3D video data, wherein the wireless interface wirelessly transmits the 3D video data to the external 3D display device in accordance with the wireless display protocol;

a user interface module that presents a user interface that receives input data defining one or more user preferences; and

a Human Visual System (HVS) feedback module that, while the transformation module transforms the video data, determines one or more metrics using a HVS model that reflect a quality of 3D visualization of the generated 3D video data with respect to a human visual system, wherein the metrics comprise metrics measuring visual discomfort based on the one or more user preferences, and

wherein the HVS feedback module generates configuration data based on the one or more metrics and the defined user preferences, and reconfigures the transformation module based on the configuration data to refine the generation of the 3D video data so that the transformation module generates the 3D video data in accordance with the defined user preferences.

2. The portable device of claim 1, wherein the wireless display host module prepares the video data by at least in part by processing one or more of the video data and metadata associated with the video data in accordance with one or more of the display parameters.

3. The portable device of claim 1, wherein the stored video data comprises two-dimensional (2D) video data,

wherein the wireless display host module automatically determines the display parameters, and wherein the display parameters include an input format supported by the 3D display device, and wherein the wireless display host module configures the transformation module so that the video processing module generates the 3D video data in accordance with the automatically determined input format.

4. The portable device of claim 1, wherein the wireless display protocol comprises a transport protocol that encapsulates each of a plurality of video data and a plurality of depth data segments of the 3D video data in a different one of a plurality of packets.

5. The portable device of claim 4,

wherein the transport protocol comprises an application layer transport protocol, and

wherein the application layer transport protocol comprises a real-time transport protocol (RTP).

6. The portable device of claim 4, wherein the transport protocol further encapsulates metadata associated with the depth data segments.

7. The portable device of claim 1, wherein the wireless interface wirelessly transmits the 3D video data to a wireless display client module coupled to the external 3D display device, and wherein the wireless display host module and the wireless display client module both support the wireless display protocol.

8. The portable device of claim 1,

wherein the wireless display host module automatically determines the display parameters, and wherein the display parameters include an input format supported by the 3D display device, and wherein the wireless display host module configures the transformation module so that the transformation module generates the 3D video data in accordance with the automatically determined input format.

9. The portable device of claim 8, wherein the input format comprises one of a 2D-plus-depth input format a 2D-plus-depth-occlusion-and-global-effects input format, and a multiview stream input format, and

wherein the 2D-plus-depth-occlusion-and-global-effects input format defines one or more of a transparency, a global illumination model and an atmospheric effect.

10. The portable device of claim 8, wherein the user interface module presents a user interface that receives input data selecting the 3D display device from among a plurality of 3D display devices, wherein the wireless interface wirelessly communicates with the selected 3D display device in accordance with the wireless display protocol to automatically determine the display parameters.

11. The portable device of claim 1, wherein the wireless interface transmits the 3D video data to the 3D display device as the 3D video data is generated.

12. The portable device of claim 1, wherein the portable device comprises one of a mobile cellular phone, a mobile computing device, a portable device and a handset device.

13. A method comprising:

storing, with a portable device, video data;

wirelessly communicating with a three-dimensional (3D) display device external from the portable device using a wireless display protocol;

determining, with a wireless display host module of the portable device, one or more display parameters of the 3D display device external from the portable device based on the wireless communication with the 3D display device;

preparing, with the wireless display host module, the video data in accordance with the wireless display protocol to generate 3D video data based on the determined display parameters;

transforming, with a transformation module, the video data to generate the 3D video data;

transmitting, with the portable device, the 3D video data to the external 3D display device in accordance with the wireless display protocol;

presenting a user interface that receives input data defining one or more user preferences;

while the transformation module transforms the video data, determining, with a Human Visual System (HVS) feedback module, one or more metrics using a HVS model that reflect a quality of 3D visualization of the generated 3D video data with respect to a human visual system, wherein the metrics comprise metrics measuring visual discomfort based on the one or more user preferences;

generating, with the HVS feedback module, configuration data based on the one or more metrics and the defined user preferences; and

reconfiguring, with the HVS feedback module, the transformation module based on the configuration data to refine the generation of the 3D video data so that the transformation module generates the 3D video data in accordance with the defined user preferences.

14. The method of claim 13, wherein preparing the video data comprises processing one or more of the video data and metadata associated with the video data in accordance with one or more of the display parameters.

15. The method of claim 13, wherein the stored video data comprises two-dimensional (2D) video data,

wherein determining the one or more display parameters comprises automatically determining the one or more display parameters, and

wherein the display parameters include an input format supported by the 3D display device, and

the method further comprises configuring the transformation module so that the video processing module generates the 3D video data in accordance with the automatically determined input format.

16. The method of claim 13, wherein the wireless display protocol comprises a transport protocol that encapsulates each of a plurality of video data and a plurality of depth data segments of the 3D video data in a different one of a plurality of packets.

17. The method of claim 16,

wherein the transport protocol comprises an application layer transport protocol, and

wherein the application layer transport protocol comprises a real-time transport protocol (RTP).

18. The method of claim 16, wherein the transport protocol further encapsulates metadata associated with the depth data segments.

19. The method of claim 13, wherein transmitting the 3D video data comprises wirelessly transmitting the 3D video data to a wireless display client module coupled to the external 3D display device, and wherein the wireless display host module and the wireless display client module both support the wireless display protocol.

20. The method of claim 13,

wherein determining the one or more display parameters comprises automatically determining the one or more display parameters, and

wherein the display parameters include an input format supported by the 3D display device, and

the method further comprising configuring the transformation module so that the transformation module generates the 3D video data in accordance with the automatically determined input format.

21. The method of claim 20, wherein the input format comprises one of a 2D-plus-depth input format, a 2D-plus-depth-occlusion-and-global-effects input format, and a multiview stream input format, and

wherein the 2D-plus-depth-occlusion-and-global-effects input format defines one or more of a transparency, a global illumination model and an atmospheric effect.

22. The method of claim 20, further comprising:

presenting a user interface that receives input data selecting the 3D display device from among a plurality of 3D display devices; and

wirelessly communicating with the selected 3D display device in accordance with the wireless display protocol to automatically determine the display parameters.

23. The method of claim 13, wherein transmitting the 3D video data comprises transmitting the 3D video data to the 3D display device as the 3D video data is generated.

24. The method of claim 13, wherein the portable device comprises one of a mobile cellular phone, a mobile computing device, a portable device and a handset device.

25. A portable device comprising:

means for storing video data;

means for wirelessly communicating with a three-dimensional (3D) display device external from the portable device using a wireless display protocol;

means for determining one or more display parameters of the 3D display device based on the wireless communication with the 3D display device;

means for preparing the stored video data in accordance with the wireless display protocol to generate 3D video data based on the determined display parameters;

means for transforming the video data to generate the 3D video data;

means for transmitting the 3D video data to the external 3D display device in accordance with the wireless display protocol;

means for presenting a user interface that receives input data defining one or more user preferences;

means for determining, while transforming the video data, one or more metrics using a Human Visual System (HVS) model that reflect a quality of 3D visualization of the generated 3D video data with respect to a human visual system, wherein the metrics comprise metrics measuring visual discomfort based on the one or more user preferences;

means for generating configuration data based on the one or more metrics and the defined user preferences; and

means for reconfiguring the means for transforming based on the configuration data to refine the generation of the 3D video data so as to generate the 3D video data in accordance with the defined user preferences.

26. The portable device of claim 25, wherein the means for preparing the video data comprises means for processing one or more of the video data and metadata associated with the video data in accordance with one or more of the display parameters.

27. The portable device of claim 25, wherein the stored video data comprises two-dimensional (2D) video data, wherein the means for transforming comprises means for transforming the 2D video data into the 3D video data,

wherein the means for determining the one or more display parameters comprises means for automatically determining the one or more display parameters, and

wherein the display parameters include an input format supported by the 3D display device, and

wherein the portable device further comprises means for configuring the means for preparing so that the means for preparing generates the 3D video data in accordance with the automatically determined input format.

28. The portable device of claim 25, wherein the wireless display protocol comprises a transport protocol that encapsulates each of a plurality of video data and a plurality of depth data segments of the 3D video data in a different one of a plurality of packets.

29. The portable device of claim 28,

wherein the transport protocol comprises an application layer transport protocol, and

wherein the application layer transport protocol comprises a real-time transport protocol (RTP).

30. The portable device of claim 28, wherein the transport protocol further encapsulates metadata associated with the depth data segments.

31. The portable device of claim 25, wherein the means for transmitting the 3D video data comprises means for wirelessly transmitting the 3D video data to a wireless display client module coupled to the external 3D display device, and wherein the wireless display host module and the wireless display client module both support the wireless display protocol.

32. The portable device of claim 25, further comprising:

wherein the means for determining the one or more display parameters comprises means for automatically determining the one or more display parameters, and

wherein the display parameters include an input format supported by the 3D display device; and

wherein the portable device further comprises means for configuring the means for preparing so that the means for preparing generates the 3D video data in accordance with the automatically determined input format.

33. The portable device of claim 32, wherein the input format comprises one of a 2D-plus-depth input format, a 2D-plus-depth-occlusion-and-global-effects input format, and a multiview stream input format, and

wherein the 2D-plus-depth-occlusion-and-global-effects input format defines one or more of a transparency, a global illumination model and an atmospheric effect.

34. The portable device of claim 32, further comprising:

means for presenting a user interface that receives input data selecting the 3D display device from among a plurality of 3D display devices; and

means for wirelessly communicating with the selected 3D display device in accordance with the wireless display protocol to automatically determine the display parameters.

35. The portable device of claim 25, wherein the means for transmitting the 3D video data comprises means for transmitting the 3D video data to the 3D display device as the 3D video data is generated.

36. The portable device of claim 25, wherein the portable device comprises one of a mobile cellular phone, a mobile computing device, a portable device and a handset device.

37. A non-transitory computer-readable storage medium comprising instructions that cause a processor to:

store video data;

wirelessly communicate with a three-dimensional (3D) display device external from a portable device via the wireless interface using a wireless display protocol;

determine one or more display parameters of the 3D display device based on the wireless communication with the 3D display device;

prepare the stored video data to generate 3D video data in accordance with the wireless display protocol based on the determined display parameters;

transform the video data to generate the 3D video data with a transformation module;

wirelessly transmit the 3D video data to the external 3D display device in accordance with the wireless display protocol;

present a user interface that receives input data defining one or more user preferences;

while the transforming the video data, determine one or more metrics using a Human Visual System (HVS) model that reflect a quality of 3D visualization of the generated 3D video data with respect to a human visual system, wherein the metrics comprise metrics measuring visual discomfort based on the one or more user preferences;

generate configuration data based on the one or more metrics and the defined user preferences; and

reconfigure the transformation module based on the configuration data to refine the generation of the 3D video data so that the transformation module generates the 3D video data in accordance with the defined user preferences.