Fast Depth Map Generation for 2D to 3D Conversion

• US 20110188773A1
• Filed: 03/07/2011
• Published: 08/04/2011
• Est. Priority Date: 02/04/2010
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Wang, Xiaoling, Wei, Jianing, Berestov, Alexander, Tu, Xue
• Granted Patent: US 8,472,746 B2
• Time in Patent Office: Days
• US Class Current: 382/260
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/20028 Bilateral filtering
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 5/002 Denoising; Smoothing noise processing or correction adapted to be used in an image pickup device containing and electronic image sensor H04N23/81, H04N25/60 - H04N25/67
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06V 10/30 Noise filtering
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity

Abstract

A method for generating a depth map for a 2D image includes receiving the 2D image; analyzing content of the received 2D image; determining a depth map based on a result of the content analysis; refining the determined depth map using an edge-preserving and noise reducing smoothing filter; and providing the refined depth map.

Claims

1. A computer-implemented method for generating a depth map for a 2D image, the method comprising:

receiving the 2D image;

analyzing content of the received 2D image;

determining a depth map based on a result of the content analysis;

refining the determined depth map using an edge-preserving and noise reducing smoothing filter; and

providing the refined depth map.

2. The method of claim 1, wherein analyzing content of the received 2D image comprises:

acquiring a first set of visual features from the received 2D image; and

classifying the received 2D image as one of a plurality of image categories based on the acquired first set of visual features.

3. The method of claim 2, wherein determining a depth map based on a result of the content analysis comprises:

determining the depth map based on a property of one of image categories.

4. The method of claim 2, further comprising:

acquiring a second set of visual features from the received 2D image; and

further classifying the received 2D image as a subcategory of the one image category based on the acquired second set of visual features.

5. The method of claim 1, wherein determining a depth map based on a result of the content analysis comprises:

determining the depth map based on a property of the subcategory.

6. The method of claim 1, further comprising:

computing one or more local features of the received 2D image; and

updating the determined depth map based on the computed one or more local features.

7. The method of claim 6, wherein computing one or more local features from the received 2D image comprises:

dividing the received 2D image into a plurality of blocks; and

computing the one or more local features for each of the plurality of blocks.

8. The method of claim 7, wherein computing the one or more local features for each of the plurality of blocks comprises:

computing at least one of a dark channel value or a horizontal gradient value for the each of the plurality of blocks.

9. The method of claim 7, further comprising:

reducing blockiness of the updated depth map by performing a smoothing method.

10. The method of claim 6, wherein refining the determined depth map using an edge-preserving and noise reducing smoothing filter comprises:

refining the updated depth map using an edge-preserving and noise reducing smoothing filter.

11. The method of claim 1, further comprising:

decomposing the received 2D image into a plurality of regions based on one or more visual features;

assigning depth information to each of the plurality of regions; and

determining the depth map based on the assignment.

12. The method of claim 11, wherein decomposing the received 2D image into a plurality of regions based on one or more visual features comprises:

decomposing the received 2D image into the plurality of regions comprising a sky region, a ground region, or a vertical structure region, based on one or more visual features comprising a color, an edge, a horizontal gradient value, or a vertical gradient value.

13. The method of claim 12, wherein assigning depth information to each of the plurality of regions comprises:

assigning a first depth plane to the sky region, wherein a depth in the first depth plane increases as reaching from a top to a bottom of the first depth plane;

assigning a second depth plane to the ground region, wherein a depth in the second depth plane increases as reaching from a bottom to a top of the second depth plane;

or

assigning a third depth plane to the vertical structure region, wherein the third depth plane stands straight on the ground region.

14. The method of claim 1, wherein determining a depth map based on a result of the content analysis comprises:

determining the depth map based on a global depth model corresponding to the received 2D image.

15. The method of claim 1, wherein refining the determined depth map comprises:

refining the determined depth map using a bilateral filter.

16. The method of claim 1, further comprising:

generating a 3D image based at least on the provided depth map.

17. An apparatus for generating a depth map for a 2D image, the apparatus comprising:

an image content analyzer to analyze content of the received 2D image; and

a depth map generator to determine a depth map based on a result of the content analysis, and refine the determined depth map using an edge-preserving and noise reducing smoothing filter.

18. The apparatus of claim 17, further comprising:

an image rendering engine to generate a 3D image based at least on the refined depth map.

19. The apparatus of claim 17, further comprising:

an image database to store configuration information for the content analysis, or configuration information for depth map generation methods corresponding to the content of the 2D image.

20. A non-transitory computer-readable medium storing instructions that, when executed, cause a computer to perform a method for generating a depth map for a 2D image, the method comprising:

receiving the 2D image;

analyzing content of the received 2D image;

determining a depth map based on a result of the content analysis;

refining the determined depth map using an edge-preserving and noise reducing smoothing filter; and

providing the refined depth map.