Apparatus and method for providing three dimensional media content

• US 8,977,038 B2
• Filed: 03/19/2013
• Issued: 03/10/2015
• Est. Priority Date: 08/12/2010
• Status: Active Grant

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Tian, Chao, Vaishampayan, Vinay Anant, Zhang, Yifu
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US13/847,241
• Publication Number:
  US 20130208094A1
• Time in Patent Office: 721 Days
• Field of Search: 382/154, 382/285, 345419-427, 356 12- 14, 348 42- 60
• US Class Current: 382/154
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 17/00 Diagnosis, testing or measuring for television systems or their details
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

A system that incorporates teachings of the exemplary embodiments may include, for example, means for generating a disparity map based on a depth map, means for determining accuracy of pixels in the depth map where the determining means identifies the pixels as either accurate or inaccurate based on a confidence map and the disparity map, and means for providing an adjusted depth map where the providing means adjusts inaccurate pixels of the depth map using a cost function associated with the inaccurate pixels. Other embodiments are disclosed.

Claims

1. A non-transitory computer-readable storage medium comprising instructions, which, responsive to being executed by a processor, cause the processor to perform operations comprising:

determining an accuracy of pixels in a depth map by identifying pixels as either accurate or inaccurate based on a confidence map and a disparity map;

adjusting the depth map by adjusting inaccurate pixels in the depth map;

generating three-dimensional media content based on images captured by a camera and the adjusted depth map; and

providing the three-dimensional media content for presentation at a display device.

2. The non-transitory computer-readable storage medium of claim 1, wherein the processor is part of a media processor connected with the display device.

3. The non-transitory computer-readable storage medium of claim 1, wherein the inaccurate pixels are identified based on measuring a disparity variance for a pre-determined region surrounding each pixel of a disparity map generated from the depth map.

4. The non-transitory computer-readable storage medium of claim 1, wherein the depth map is captured by a range camera.

5. The non-transitory computer-readable storage medium of claim 1, wherein the adjusted depth map is based on the adjustments to the inaccurate pixels in the depth map using a graph-cut analysis.

6. The non-transitory computer-readable storage medium of claim 1, wherein the adjusted depth map is based on the adjustments to the inaccurate pixels in the depth map using a belief-propagation analysis.

7. The non-transitory computer-readable storage medium of claim 1, wherein the adjustments to the inaccurate pixels in the depth map are based on a cost function being applied to the inaccurate pixels.

8. The non-transitory computer-readable storage medium of claim 1, wherein the depth map is generated based on measuring parameters associated with infrared light to obtain range data associated with an object in the images.

9. The non-transitory computer-readable storage medium of claim 1, wherein homographic warping approximation is applied for calibration associated with the three-dimensional media content.

10. A method comprising:

receiving, by a processor, images captured by a camera;

determining an accuracy of pixels in a depth map by identifying pixels as either accurate or inaccurate based on a confidence map and a disparity map;

adjusting the depth map by adjusting inaccurate pixels in the depth map; and

generating, by the processor, three-dimensional media content based on the images and the adjusted depth map.

11. The method of claim 10, wherein the processor is coupled to a display device, and further comprising providing the three-dimensional media content to the display device for presentation at the display device.

12. The method of claim 10, wherein the inaccurate pixels are identified based on measuring a disparity variance for a pre-determined region surrounding each pixel of the disparity map generated from the depth map.

13. The method of claim 10, wherein the depth map is captured by a range camera.

14. The method of claim 10, wherein the adjusted depth map is based on the adjustments to the inaccurate pixels in the depth map using a graph-cut analysis.

15. The method of claim 10, wherein the adjusted depth map is based on the adjustments to the inaccurate pixels in the depth map using a belief-propagation analysis.

16. The method of claim 10, wherein the adjustments to the inaccurate pixels in the depth map are based on a cost function being applied to the inaccurate pixels.

17. The method of claim 10, wherein the depth map is generated based on measuring parameters associated with infrared light to obtain range data associated with an object in the images.

18. The method of claim 10, wherein homographic warping approximation is applied for calibration associated with the three-dimensional media content.

19. A device comprising:

a memory to store computer instructions; and

a processor coupled with the memory, wherein the processor, responsive to executing the computer instructions, performs operations comprising;

receiving images captured by a camera;

determining an accuracy of pixels in a depth map by identifying pixels as either accurate or inaccurate based on a confidence map and a disparity map;

adjusting the depth map by adjusting inaccurate pixels in the depth map; and

generating three-dimensional media content based on the images and an adjusted depth map, wherein the adjusted depth map is based on adjustments to inaccurate pixels in a depth map.

20. The device of claim 19, wherein the processor is coupled to a display device, and wherein the operations further comprise providing the three-dimensional media content to the display device for presentation at the display device.