2D to 3D Image Conversion Based on Image Content

• US 20110188780A1
• Filed: 02/02/2011
• Published: 08/04/2011
• Est. Priority Date: 02/04/2010
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Wang, Xiaoling, Wei, Jianing, Berestov, Alexander, Tu, Xue
• Granted Patent: US 8,520,935 B2
• Time in Patent Office: Days
• US Class Current: 382/293
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 7/44 using image operators, e.g. filters, edge density metrics or local histograms
• CPC Class Codes: G06T 7/529 from texture
• CPC Class Codes: G06T 7/536 from perspective effects, e.g. by using vanishing points
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion

Abstract

A method for converting a 2D image into a 3D image includes receiving the 2D image; analyzing content of the received 2D image; determining a 2D-to-3D image conversion method based on a result of the content analysis; generating the 3D image by applying the determined method to the received 2D image; and providing the generated 3D image.

Claims

1. A computer-implemented method for converting a 2D image into a 3D image, the method comprising:

receiving the 2D image;

analyzing content of the received 2D image;

determining a 2D-to-3D image conversion method based on a result of the content analysis;

generating the 3D image by applying the determined method to the received 2D image; and

providing the generated 3D image.

2. The method of claim 1, wherein analyzing the content of the received 2D image comprises:

extracting a first set of visual features from the received 2D image; and

classifying the received 2D image as one of image categories based on the extracted first set of visual features.

3. The method of claim 2, wherein classifying the received 2D image as one of image categories based on the extracted first set of visual features comprises:

classifying the received 2D image as one of a landscape image or a geometric structure image based on the extracted first set of visual features.

4. The method of claim 3, wherein classifying the received 2D image as one of a landscape image or a geometric structure image based on the extracted first set of visual features comprises:

converting the received 2D image into a grayscale image;

detecting edges in the grayscale image;

calculating an edge direction histogram representing edge directions based on the detected edges;

extracting edge direction features from the edge direction histogram; and

classifying the received 2D image as one of the landscape image or the geometric structure image based on the extracted edge direction features.

5. The method of claim 3, further comprising:

segmenting objects in the received 2D image; and

identifying each of the segmented objects as one of object classes;

assigning a depth to each of the identified object class;

deriving a depth map based on the assignment; and

generating the 3D image based at least on the derived depth map.

6. The method of claim 2, further comprising:

extracting a second set of visual features from the received 2D image; and

further classifying the received 2D image as one of subcategories of the one of image categories based on the extracted second set of visual features.

7. The method of claim 6, wherein further classifying the received 2D image as one of subcategories of the one of image categories based on the extracted second set of visual features comprises:

further classifying the received 2D image as one of an indoor image or an outdoor image based on the extracted second set of visual features.

8. The method of claim 7, wherein further classifying the received 2D image as one of an indoor image or an outdoor image based on the extracted second set of visual features comprises:

converting a first color space of the received 2D image into a second color space;

dividing the converted image into a plurality of vertical sub-blocks;

generating a color histogram for each color channel in each of the plurality of vertical sub-blocks;

concatenating the generated color histograms;

extracting color distribution features from the concatenated color histograms; and

classifying the received 2D image as one of the indoor image or the outdoor image based on the extracted color distribution features.

9. The method of claim 7, further comprising:

detecting a vanishing point in the received 2D image, wherein the received 2D image is an outdoor image;

dividing the received 2D image into a plurality of depth gradient planes based on the detected vanishing point;

assigning each object in the received 2D image to one of the plurality of depth gradient planes;

deriving a depth map based on the assignment; and

generating the 3D image based at least on the derived depth map.

10. The method of claim 7, further comprising:

detecting a vanishing point in the received 2D image, wherein the received 2D image is an indoor image;

determining a geometric structure for an object in the received 2D image to find a vanishing line;

deriving a depth map based on the detected vanishing point and a geometric direction of the vanishing line; and

generating the 3D image based at least on the derived depth map.

11. The method of claim 1, wherein analyzing the content of the received 2D image comprises:

segmenting objects in the received 2D image; and

identifying each of the segmented objects as one of object classes.

12. The method of claim 11, further comprising:

assigning a depth to each of the identified object class;

deriving a depth map based on the assignment; and

generating the 3D image based at least on the derived depth map.

13. The method of claim 1, wherein determining a 2D-to-3D image conversion method based on a result of the content analysis comprises choosing, based on the result of the content analysis, one of algorithms that include:

shifting a left or right eye image;

shifting image pixels depending on their positions;

shifting edges of objects in the 2D image;

shifting image frequencies;

generating a depth map;

employing a 3D model; and

any other 2D-to-3D image conversion method corresponding to the content of the 2D image.

14. The method of claim 1, further comprising:

deriving a depth map based on the result of the content analysis.

15. The method of claim 14, further comprising:

generating the 3D image based at least on the derived depth map.

16. An apparatus for converting a received 2D image into a 3D image, the apparatus comprising:

an image content analyzer to analyze content of the received 2D image;

a conversion method chooser to determine a 2D-to-3D image conversion method based on a result of the content analysis;

a 3D image generator to generate the 3D image by applying the determined method to the received 2D image; and

an image rendering engine to provide the generated 3D image.

17. The apparatus of claim 16, further comprising:

an image database to store configuration information for the content analysis, or at least one 2D-to-3D image conversion method corresponding to the content of the 2D image.

18. The apparatus of claim 16, further comprising:

a depth map generator to derive a depth map based on the result of the content analysis.

19. The apparatus of claim 18, wherein the 3D image generator is further configured to:

generate the 3D image based at least on the derived depth map.

20. A non-transitory computer-readable medium storing instructions that, when executed, cause a computer to perform a method for converting a 2D image into a 3D image, the method comprising:

receiving the 2D image;

analyzing content of the received 2D image;

determining a 2D-to-3D image conversion method based on a result of the content analysis;

generating the 3D image by applying the determined method to the 2D image; and

providing the generated 3D image.