Apparatus and method for providing three dimensional media content

• US 8,428,342 B2
• Filed: 08/12/2010
• Issued: 04/23/2013
• Est. Priority Date: 08/12/2010
• Status: Active Grant

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Vaishampayan, Vinay Anant, Zhang, Yifu, Tian, Chao
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US12/855,669
• Publication Number:
  US 20120039525A1
• Time in Patent Office: 985 Days
• Field of Search: 382/154, 382/285, 345419-427, 356 12- 14, 348 42- 60
• US Class Current: 382/154
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 17/00 Diagnosis, testing or measuring for television systems or their details
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

A system that incorporates teachings of the exemplary embodiments may include, for example, means for generating a disparity map based on a depth map, means for determining accuracy of pixels in the depth map where the determining means identifies the pixels as either accurate or inaccurate based on a confidence map and the disparity map, and means for providing an adjusted depth map where the providing means adjusts inaccurate pixels of the depth map using a cost function associated with the inaccurate pixels. Other embodiments are disclosed.

Claims

1. A system comprising:

means for generating a disparity map based on a depth map;

means for determining accuracy of pixels in the depth map, wherein the determining means identifies the pixels as either accurate or inaccurate based on a confidence map and the disparity map; and

means for providing an adjusted depth map, wherein the providing means adjusts inaccurate pixels of the depth map using a cost function associated with the inaccurate pixels.

2. The system of claim 1, wherein the providing means applies one of a graph-cut analysis or a belief-propagation analysis to the cost function to provide the adjusted depth map, and wherein the accurate pixels are not adjusted.

3. The system of claim 1, further comprising means for rendering three-dimensional media content, wherein the rendering means uses the adjusted depth map and images from at least one vision camera to render the three-dimensional media content.

4. The system of claim 1, wherein the determining means measures a disparity variance for a pre-determined region surrounding each pixel of the disparity map to identify the pixels as either accurate or inaccurate.

5. The system of claim 1, further comprising means for obtaining range data, wherein the obtaining means generates the depth map based on the range data.

6. The system of claim 5, wherein the obtaining means generates the confidence map.

7. The system of claim 5, wherein the obtaining means measures parameters associated with infrared light to obtain the range data.

8. The system of claim 5, further comprising:

means for capturing stereo image pairs; and

means for calibrating the capturing means with the obtaining means.

9. The system of claim 8, wherein the calibrating means utilizes homographic warping approximation.

10. A method comprising:

obtaining a depth map and a confidence map associated with an object;

obtaining a stereo image pair of the object;

generating a disparity map based on the depth map;

determining accuracy of pixels in the depth map based on the confidence map and the disparity map; and

adjusting inaccurate pixels of the depth map to generate an adjusted depth map using a cost function associated with the inaccurate pixels.

11. The method of claim 10, further comprising transmitting the adjusted depth map and at least one of the stereo image pair to a media processor for generating three-dimensional media content.

12. The method of claim 10, wherein the determining of the accuracy of the pixels is based on measuring a disparity variance for a pre-determined region surrounding each pixel of the disparity map to identify the pixels as either accurate or inaccurate.

13. The method of claim 10, wherein the adjusting of the inaccurate pixels is based on a belief-propagation analysis.

14. The method of claim 10, wherein the adjusting of the inaccurate pixels is based on a graph-cut analysis.

15. The method of claim 10, wherein the obtaining of the depth map is based on measuring parameters associated with infrared light to obtain range data associated with the object.

16. The method of claim 10, further comprising:

generating three-dimensional media content based on the adjusted depth map, the confidence map and at least one of the stereo image pairs; and

presenting the three-dimensional media content on a display.

17. The method of claim 10, further comprising:

obtaining the stereo image pair using a pair of vision cameras;

calibrating the pair of vision cameras;

rectifying the stereo image pair; and

mapping the depth map with the rectified stereo image pair.

18. The method of claim 17, wherein the mapping of the depth map with the rectified stereo image pairs utilizes homographic warping approximation.