3D image acquisition apparatus and method of calculating depth information in the 3D image acquisition apparatus

• US 9,418,425 B2
• Filed: 08/24/2012
• Issued: 08/16/2016
• Est. Priority Date: 10/25/2011
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Park, Yong-hwa, You, Jang-woo, Yoon, Hee-sun
• Primary Examiner(s): Seth, Manav
• Application Number:
  US13/594,094
• Publication Number:
  US 20130101176A1
• Time in Patent Office: 1,453 Days
• US Class Current: 1/1
• CPC Class Codes: G01S 17/36 with phase comparison between the received signal and the contemporaneously transmitted signal
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G01S 7/4816 of receivers alone
• CPC Class Codes: G01S 7/4912 Receivers
• CPC Class Codes: G01S 7/4915 Time delay measurement, e.g. operational details for pixel components signal extraction and conditioning G01S7/493; Phase measurement
• CPC Class Codes: G01S 7/4918 Controlling received signal intensity, gain or exposure of sensor
• CPC Class Codes: G01S 7/493 Extracting wanted echo signals
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 5/2226 Determination of depth image, e.g. for foreground/background separation determining depth by image analysis in general G06T7/50; segmentation by image analysis in general G06T7/10

Abstract

A 3-dimensional (3D) image acquisition apparatus and a method of calculating depth information in the 3D image acquisition apparatus, the 3D image acquisition apparatus including: an optical modulator for modulating light reflected from a subject by sequentially projected N (N is 3 or a larger natural number) light beams; an image sensor for generating N sub-images by capturing the light modulated by the optical modulator; and a signal processor for calculating depth information regarding a distance to the subject by using the N sub-images.

Claims

1. A 3-dimensional (3D) image acquisition apparatus comprising:

an optical modulator which modulates light reflected from a subject by sequentially projected N light beams;

an image sensor which generates N sub-images by capturing the light modulated by the optical modulator; and

a signal processor which calculates depth information corresponding to a distance to the subject by using the N sub-images,

wherein N is a natural number that is greater than or equal to 3, and

wherein the image sensor is exposed during a projecting time of each light beam of the N light beams to capture the modulated light and forms the N sub-images during at least a portion of a remaining time of the projecting time of each light beam of the N light beams.

2. The 3D image acquisition apparatus of claim 1, wherein the N light beams are discontinuously projected.

3. The 3D image acquisition apparatus of claim 1, wherein the N projected light beams are different from each other and are emitted by one or more light sources.

4. The 3D image acquisition apparatus of claim 3, wherein the one or more light sources sequentially project the N light beams with a predetermined time interval.

5. The 3D image acquisition apparatus of claim 1, wherein an operating time of the optical modulator is synchronized with the projecting time of each light beam of the N light beams.

6. The 3D image acquisition apparatus of claim 5, wherein the operating time of the optical modulator is shorter than the projecting time.

7. The 3D image acquisition apparatus of claim 5, wherein an exposure time of the image sensor is synchronized with the operating time of the optical modulator.

8. The 3D image acquisition apparatus of claim 1, wherein all pixels of the image sensor are exposed to the modulated light during the projecting time of each light beam of the N light beams.

9. The 3D image acquisition apparatus of claim 1, wherein the N light beams are periodic waves having a same period and at least one light beam from among the N light beams comprises a different intensity and a different phase from the other N light beams.

10. The 3D image acquisition apparatus of claim 9, wherein the optical modulator modulates the reflected light with the same modulation signal.

11. The 3D image acquisition apparatus of claim 1, wherein the N light beams have the same periodic waves.

12. The 3D image acquisition apparatus of claim 1, wherein the optical modulator modulates the reflected light with different modulation signals.

13. The 3D image acquisition apparatus of claim 1, wherein a phase difference between any two light beams projected at adjacent times from among the N light beams is a value obtained by dividing 360°

by N.

14. The 3D image acquisition apparatus of claim 1, wherein the reflected light includes N reflection light beams obtained by reflecting the N light beams from the subject.

15. The 3D image acquisition apparatus of claim 1, wherein the N sub-images generated by the image sensor to sequentially one-to-one match the N reflection light beams.

16. The 3D image acquisition apparatus of claim 1, wherein, if the N sub-images do not one-to-one match the N reflection light beams, the signal processor converts the N sub-images on a line by line basis and sequentially one-to-one matches the N line-based sub-images with the N reflection light beams.

17. The 3D image acquisition apparatus of claim 1, wherein the signal processor generates a first average image by averaging the N sub-images multiplied by first weighting factors, generates a second average image by averaging the N sub-images multiplied by second weighting factors, and calculates the depth information from the first average image and the second average image.

18. The 3D image acquisition apparatus of claim 17, wherein the depth information is calculated from an arctangent value of a ratio of the first average image to the second average image.

19. A method of calculating depth information, the method comprising:

modulating light reflected from a subject by sequentially projecting N light beams;

generating N sub-images by capturing the modulated light; and

calculating depth information regarding a distance to the subject by using the N sub-images,

wherein N is a natural number that is greater than or equal to 3, and

wherein an image sensor is exposed during a projecting time of each light beam of the N light beams to capture the modulated light and form the N sub-images during at least a portion of a remaining time of the projecting time of each light beam of the N light beams.

20. The method of claim 19, wherein the N light beams are discontinuously projected.

21. The method of claim 19, wherein the N projected light beams are different from each other and are emitted by one or more light sources.

22. The method of claim 21, wherein the N light beams are sequentially projected with a predetermined time interval.

23. The method of claim 19, wherein an operating time of an optical modulator for modulating the light is synchronized with the projecting time of each light beam of the N light beams.

24. The method of claim 23, wherein the operating time of the optical modulator is shorter than the projecting time.

25. The method of claim 23, wherein an exposure time of the image sensor which captures the light is synchronized with the operating time of the optical modulator.

26. The method of claim 25, wherein all pixels of the image sensor are exposed to the modulated light during the projecting time of each light beam of the N light beams.

27. The method of claim 19, wherein the N light beams are periodic waves having a same period and at least one light beam from among the N light beams comprises a different intensity and a different phase from the other N light beams, and the reflected light is modulated with the same modulation signal.

28. The method of claim 19, wherein the N light beams have the same periodic waves, and the reflected light is modulated with different modulation signals.

29. The method of claim 19, wherein a phase difference between any two light beams projected at adjacent times from among the N light beams is a value obtained by dividing 360°

by N.

30. The method of claim 19, wherein the generated N sub-images sequentially one-to-one match the N reflection light beams.

31. The method of claim 19, further comprising, if the N sub-images do not one-to-one match the N reflection light beams, converting the N sub-images on a line by line basis and sequentially one-to-one matching the N line-based sub-images with the N reflection light beams.

32. The method of claim 19, wherein a first average image is generated by averaging the N sub-images multiplied by first weighting factors, a second average image is generated by averaging the N sub-images multiplied by second weighting factors, and the depth information is calculated from the first average image and the second average image.

33. The method of claim 32, wherein the depth information is calculated from an arctangent value of a ratio of the first average image to the second average image.