THREE DIMENSIONAL IMAGE GENERATING SYSTEM AND METHOD ACCOMODATING MULTI-VIEW IMAGING

• US 20110298898A1
• Filed: 05/04/2011
• Published: 12/08/2011
• Est. Priority Date: 05/11/2010
• Status: Abandoned Application

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Jung, Yong Ju, Wang, Haitao, Kim, Ji Won, Ma, Gengyu, Park, Du Sik, Mei, Xing
• Application Number:
  US13/100,905
• Publication Number:
  US 20110298898A1
• Time in Patent Office: Days
• US Class Current: 348/47
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps

Abstract

Provided is a three-dimensional (3D) image generating system and method accommodating multi-view imaging. The 3D image generating system and method may generate corrected depth maps respectively corresponding to color images by merging disparity information associated with a disparity between color images and depth maps generated respectively from depth images.

Claims

1. A three-dimensional (3D) image generating system for a multi-view image, the system comprising:

stereo color cameras to capture stereo color images for a 3D image;

stereo depth cameras to capture depth images of areas same as areas photographed by the stereo color cameras;

a mapping unit to map the captured depth images with respective corresponding color images, of the captured color images; and

a depth merging unit to generate corrected depth maps respectively corresponding to the captured color images, based on both disparity information associated with a disparity between the captured color images and primary depth maps respectively generated by the mapping of the mapping unit from the captured depth images.

2. The system of claim 1, wherein the depth merging unit comprises:

a first depth measuring unit to generate the primary depth maps respectively from the captured depth images;

a second depth measuring unit to generate secondary depth maps respectively corresponding to the captured color images, based on the disparity information; and

a weighted-average calculator to generate the corrected depth maps by weighted-averaging, using a predetermined weight, the primary depth maps and the secondary depth maps respectively corresponding to the captured color images.

3. The system of claim 1, wherein the depth merging unit comprises:

a first depth measuring unit to generate the primary depth maps respectively from the captured depth images; and

a second depth measuring unit to use information associated with the primary depth maps as a factor to calculate a disparity distance between the captured color images when stereo-matching of the captured color images is performed to generate the corrected depth maps.

4. The system of claim 1, further comprising:

a synchronizing unit to set the stereo color cameras to be synchronized with the stereo depth cameras.

5. The system of claim 1, further comprising:

a camera setting unit to determine a feature of each of the stereo color cameras and the stereo depth cameras, to set the stereo color cameras and the stereo depth cameras to respectively capture the color images and the depth images with a same size, and to set the stereo depth cameras to respectively capture same respective areas as areas captured by respective corresponding stereo color cameras.

6. The system of claim 1, further comprising:

a distortion correcting unit to correct a distortion that occurs in the captured color images and the captured depth images due to a feature of each of the stereo color cameras and the stereo depth cameras.

7. The system of claim 1, further comprising:

a stereo correcting unit to correct an error that occurs when the stereo color cameras and the stereo depth cameras perform capturing in different directions.

8. The system of claim 1, further comprising:

a color correcting unit to correct a color error in the captured color images, which occurs due to a feature of each of the stereo color cameras being different.

9. The system of claim 1, further comprising:

a 3D image file generating unit to generate a 3D image file including the captured color images and the corrected depth maps.

10. The system of claim 9, further comprising:

generating confidence maps to indicate respective confidences of the corrected depth maps.

11. A three-dimensional (3D) image generating method for a multi-view image, the method comprising:

receiving color images and depth images respectively captured from stereo color cameras and stereo depth cameras;

mapping the captured depth images with respective corresponding color images, of the captured color images; and

generating corrected depth maps respectively corresponding to the captured color images, based on both disparity information associated with a disparity between the captured color images and primary depth maps respectively generated from the mapping of the captured depth images.

12. The method of claim 11, wherein the generating of the corrected depth maps comprises:

generating the primary depth maps respectively from the captured depth images;

generating secondary depth maps respectively corresponding to the captured color images, based on the disparity information; and

generating the corrected depth maps by weighted-averaging, using a predetermined weight, the primary depth maps and the secondary depth maps respectively corresponding to the captured color images.

13. The method of claim 11, wherein the generating of the corrected depth maps comprises:

generating the primary depth maps respectively from the captured depth images; and

generating the corrected depth maps, using information associated with the primary depth maps as a factor to calculate a disparity distance between the captured color images when stereo-matching of the captured color images is performed to generate the corrected depth maps.

14. The method of claim 11, further comprising:

setting the stereo color cameras to be synchronized with the stereo depth cameras.

15. The method of claim 11, further comprising:

determining a feature of each of the stereo color cameras and the stereo depth cameras, to set the stereo color cameras and the stereo depth cameras to capture the color images and the depth images with a same size, and to set the stereo depth cameras to respectively capture same respective areas as areas captured by respective corresponding stereo color cameras.

16. The method of claim 11, further comprising:

correcting a distortion that occurs in the captured color images and the captured depth images due to a feature of each of the stereo color cameras and the stereo depth cameras.

17. The method of claim 11, further comprising:

correcting an error that occurs when the stereo color cameras and the stereo depth camera perform capturing in different directions.

18. The method of claim 11, further comprising:

correcting a color error in captured color images, which occurs due to a feature of each of the stereo color cameras being different.

19. The method of claim 11, further comprising:

generating a 3D image file including the captured color images and the corrected depth maps.

20. The method of claim 19, wherein the 3D image file further includes confidence maps to indicate respective confidences of the corrected depth maps.