METHOD AND APPARATUS FOR GENERATING DEPTH INFORMATION FROM IMAGE

• US 20130222543A1
• Filed: 08/30/2012
• Published: 08/29/2013
• Est. Priority Date: 02/27/2012
• Status: Abandoned Application

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: BAE, Kwang-Hyuk, KYUNG, Kyu-Min, KIM, Tae-Chan
• Application Number:
  US13/598,824
• Publication Number:
  US 20130222543A1
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: G01S 17/08 for measuring distance only indirect measurement G01S17/46; active triangulation systems G01S17/48
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps

Abstract

An apparatus for generating depth information includes a sensing unit and a final depth providing unit. The sensing unit is configured to sense light received from multiple subjects, and to provide initial depth data having distance information about the subjects and two-dimensional (2D) image data having 2D image information about an image obtained from the subjects. The final depth providing unit is configured to generate estimated depth data having estimated distance information about the subjects by transforming the 2D image data into three-dimensional (3D) data, and to provide final depth data based on the initial depth data and the estimated depth data.

Claims

1. An apparatus for generating depth information from an image, the apparatus comprising:

a sensing unit configured to sense light received from a plurality of subjects, and to provide initial depth data having distance information about the plurality of subjects and two-dimensional (2D) image data having 2D image information about an image obtained from the plurality of subjects; and

a final depth providing unit configured to generate estimated depth data having estimated distance information about the plurality of subjects by transforming the 2D image data into three-dimensional (3D) data, and to provide final depth data based on the initial depth data and the estimated depth data.

2. The apparatus of claim 1, wherein the final depth providing unit is further configured to divide the image into a first area and a second area, and to provide the final depth data by combining the initial depth data of the first area with the estimated depth data of the second area.

3. The apparatus of claim 2, wherein the first area comprises a foreground of at least one main subject from among the plurality of subjects, and

the second area comprises a background excluding the at least one main subject from among the plurality of subjects.

4. The apparatus of claim 1, wherein the final depth providing unit comprises:

a first segmentation unit configured to divide the image into a plurality of segments, and to classify the plurality of segments into a first area and a second area based on the initial depth data;

a transformation unit configured to generate the estimated depth data by transforming the 2D image data into the 3D data;

an extraction unit configured to extract first data corresponding to the first area from the initial depth data, and to extract second data corresponding to the second area from the estimated depth data; and

a combining unit configured to provide the final depth data by combining the first data with the second data.

5. The apparatus of claim 4, wherein the transformation unit comprises:

a second segmentation unit configured to divide the image into a plurality of segments based on a depth cue in the image;

an indexing unit configured to index depths of the plurality of segments based on the initial depth data; and

a depth map generating unit configured to generate a depth map from the indexed depths of the plurality of segments.

6. The apparatus of claim 5, wherein the transformation unit further comprises an estimated depth providing unit configured to provide the estimated depth data that is the 3D data based on the depth map.

7. The apparatus of claim 1, wherein the 2D image data comprises at least one of intensity data and color data.

8. The apparatus of claim 1, wherein the sensing unit comprises a depth sensor configured to generate the initial depth data and intensity data based on reflected light received from the plurality of subjects.

9. The apparatus of claim 1, wherein the sensing unit comprises:

a depth sensor configured to generate the initial depth data and intensity data based on reflected light received from the plurality of subjects; and

a color sensor configured to generate color data based on visible light received from the plurality of subjects.

10. The apparatus of claim 1, wherein the sensing unit comprises a depth/color sensor configured to simultaneously generate the initial depth data, intensity data, and color data based on reflected light and visible light received from the plurality of subjects.

11. The apparatus of claim 1, wherein the sensing unit comprises a time-of-flight (ToF) sensor for providing the initial depth data.

12. A photographing apparatus, comprising:

an image sensor; and

a processor,

wherein the image sensor comprises;

a sensing unit configured to sense light received from a plurality of subjects, and to provide initial depth data having distance information about the plurality of subjects and two-dimensional (2D) image data having 2D image information about an image obtained from the plurality of subjects; and

a final depth providing unit configured to generate estimated depth data having estimated distance information about the plurality of subjects by transforming the 2D image data into three-dimensional (3D) data, and to provide final depth data based on the initial depth data and the estimated depth data.

13. The photographing apparatus of claim 12, wherein the final depth providing unit is further configured to divide the image into a first area and a second area, and to provide the final depth data by combining the initial depth data of the first area with the estimated depth data of the second area.

14. The photographing apparatus of claim 12, wherein the final depth providing unit comprises:

a first segmentation unit configured to divide the image into a plurality of segments, and to classify the plurality of segments into a first area and a second area based on the initial depth data;

a transformation unit configured to generate the estimated depth data by transforming the 2D image data into 3D data;

an extracting unit configured to extract first data corresponding to the first area from the initial depth data, and to extract second data corresponding to the second area from the estimated depth data; and

a combining unit configured to provide the final depth data by combining the first data with the second data.

15. The photographing apparatus of claim 12, wherein the 2D image data comprises at least one of intensity data and color data.

16-20. -20. (canceled)