3-dimensional image acquisition apparatus and method of extracting depth information in the 3D image acquisition apparatus

• US 8,902,411 B2
• Filed: 06/14/2011
• Issued: 12/02/2014
• Est. Priority Date: 12/23/2010
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Park, Yong-hwa, You, Jang-woo, Cho, Yong-chul
• Primary Examiner(s): Diacou, Ari M
• Application Number:
  US13/160,135
• Publication Number:
  US 20120162197A1
• Time in Patent Office: 1,267 Days
• Field of Search: 356/5.1
• US Class Current: 356/5.1
• CPC Class Codes: G01S 17/36 with phase comparison between the received signal and the contemporaneously transmitted signal
• CPC Class Codes: G01S 17/87 Combinations of systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G01S 7/487 Extracting wanted echo signals , e.g. pulse detection

Abstract

A 3 dimensional (3D) image acquisition apparatus and a method of extracting depth information in the 3D image acquisition apparatus are provided. The method of extracting depth information includes sequentially projecting N (N is a natural number equal to or greater than 3) different pieces of projection light onto a object; modulating N pieces of reflection light reflected from the object with an optical modulation signal having a gain waveform; generating N images by capturing the N pieces of modulated reflection light; generating a first averaged image by multiplying the N images by primary weights and generating a second averaged image by multiplying the N images by secondary weights; acquiring an average phase delay from the first and second averaged images; and calculating a distance to the object from the average phase delay.

Claims

1. A depth information extracting method comprising:

projecting N different pieces of projection light onto an object;

modulating N pieces of reflection light reflected from the object with an optical modulation signal;

generating N images by capturing the N pieces of modulated reflection light;

generating a first averaged image by multiplying the N images by primary weights and generating a second averaged image by multiplying the N images by secondary weights;

acquiring an average phase delay based on the first and second averaged images; and

calculating a distance to the object based on the average phase delay.

2. The depth information extracting method of claim 1, wherein each of the N different pieces of projection light are periodic waves having the same period, and each of the N different pieces of projection light has at least one of a different magnitude and a different phase as compared to a magnitude and a phase of each of the other N different pieces of projection light.

3. The depth information extracting method of claim 2, wherein the optical modulation signal is a periodic wave having a period that is equal to the period of the N different pieces of projection light.

4. The depth information extracting method of claim 2, wherein phase differences obtained by dividing 360 degrees by N exist between each of the N pieces of reflection light.

5. The depth information extracting method of claim 1, wherein the first averaged image is obtained by multiplying the N images by primary weights respectively corresponding to the N images and summing results of the multiplying of the primary weights, and the second averaged image is obtained by multiplying the N images by secondary weights respectively corresponding to the N images and summing results of the multiplying of the secondary weights.

6. The depth information extracting method of claim 5, wherein the primary and secondary weights are calculated by using at least one of magnitudes and phases of the N pieces of projection light.

7. The depth information extracting method of claim 6, wherein k^th primary and secondary weights corresponding to a k^th image of the N images are A_k and B_k, respectively, a magnitude of an i^th projection light of the N different pieces of projection light is a⁽ⁱ⁾, a phase of the i^th projection light is θ

⁽ⁱ⁾, and the k^th primary and secondary weights are defined by,

A_k=sum└

a⁽ⁱ⁾ cos θ

⁽ⁱ⁾−

a^(j) cos θ

^(j);

(i,j) is _N-1C₂<sup>(≠

k)</sup> of {1;

N}┘

B_k=sum[a⁽ⁱ⁾ sin θ

⁽ⁱ⁾−

a^(j) sin θ

^(j);

(i,j) is _N-1C₂<sup>(≠

k)</sup> of {1;

N}],

wherein i and j are any individual numbers excluding k from 1 to N and are defined as all combinations obtained by arranging (N−

1) numbers remaining after excluding k in an ascending order circulating from a number adjacent to k and selecting 2 of the (N−

1) numbers in the circulated ascending order.

8. The depth information extracting method of claim 6, wherein the primary and secondary weights are selected from a plurality of weights calculated for a plurality of combinations of magnitudes and phases of a plurality of pieces of projection light, including the N pieces of projection light.

9. The depth information extracting method of claim 8, wherein the generating of the first averaged image and the second averaged image uses weights corresponding to combinations of magnitudes and phases of the N pieces of projection light, from among the plurality of weights.

10. The depth information extracting method of claim 1, wherein the average phase delay is obtained from an arctangent value of a ratio of the first averaged image to the second averaged image.

11. The depth information extracting method of claim 1, further comprising:

additionally generating M new images by projecting M different pieces of projection light onto the object;

removing M oldest images of the N previous images;

generating first and second averaged images by multiplying the remaining N images by primary weights and secondary weights, respectively;

acquiring an average phase delay from the first and second averaged images; and

calculating a distance to the object from the average phase delay.

12. The depth information extracting method of claim 11, wherein N is a natural number equal to or greater than 3 and M is a natural number less than N.

13. The depth information extracting method of claim 1, wherein the projecting comprises sequentially projecting the N different pieces of projection light onto the object.

14. The depth information extracting method of claim 1, wherein N is a natural number equal to or greater than 3.

15. The depth information extracting method of claim 1, wherein the optical modulation signal has a gain waveform.

16. A depth information extracting method comprising:

projecting N identical pieces of projection light onto an object;

modulating N pieces of reflection light reflected from the object with N optical modulation signals having different gain waveforms, respectively;

generating N images by capturing the N pieces of modulated reflection light;

generating a first averaged image by multiplying the N images by primary weights and generating a second averaged image by multiplying the N images by secondary weights;

acquiring an average phase delay based on the first and second averaged images; and

calculating a distance to the object from the average phase delay.

17. The depth information extracting method of claim 16, wherein the N optical modulation signals are periodic waves having the same period, and each of the N optical modulation signals has at least one of a different magnitude and a different phase as compared to a magnitude and a phase of each of the other N optical modulation signals.

18. The depth information extracting method of claim 17, wherein the projection light is a periodic wave having a period that is equal to the period of the N optical modulation signals.

19. The depth information extracting method of claim 17, wherein phase differences obtained by dividing 360 degrees by N exist between each of the N optical modulation signals.

20. The depth information extracting method of claim 16, wherein the first averaged image is obtained by multiplying the N images by primary weights respectively corresponding to the N images and summing results of the multiplying of the primary weights, and the second averaged image is obtained by multiplying the N images by secondary weights respectively corresponding to the N images and summing results of the multiplying of the secondary weights.

21. The depth information extracting method of claim 20, wherein the primary and secondary weights are calculated by using at least one of magnitudes and phases of the N optical modulation signals.

22. The depth information extracting method of claim 16, wherein the average phase delay is obtained from an arctangent value of a ratio of the first averaged image to the second averaged image.

23. The depth information extracting method of claim 16, wherein the projecting comprises sequentially projecting the N identical pieces of projection light onto the object.

24. A 3 Dimensional (3D) image acquisition apparatus comprising:

a light source that projects at least three different pieces of projection light onto an object;

an optical modulator that modulates at least three pieces of light reflected from the object with an optical modulation signal;

an image pickup device that generates at least three images by capturing the light modulated by the optical modulator; and

a depth image processor that calculates a distance to the object by using the at least three images generated by the image pickup device,

wherein the depth image processor extracts depth information by generating a first averaged image by multiplying the at least three images by primary weights and generating a second averaged image by multiplying the at least three images by secondary weights;

acquiring an average phase delay based on the first and second averaged images; and

calculating the distance to the object based on the average phase delay.

25. The 3D image acquisition apparatus of claim 24, further comprising:

a light source driver that controls a waveform of the projection light by driving the light source;

an optical modulator driver that controls a gain waveform of the optical modulation signal by driving the optical modulator; and

a controller that controls operations of the light source driver, the optical modulator driver, and the image pickup device.

26. The 3D image acquisition apparatus of claim 24, further comprising:

a primary lens that concentrates the at least three pieces of light reflected from the object on a light incident surface of the optical modulator;

a filter disposed between the primary lens and the optical modulator that transmits only light having a predetermined wavelength to the optical modulator; and

a secondary lens disposed between the optical modulator and the image pickup device that concentrates the modulated light into an area of the image pickup device.

27. The 3D image acquisition apparatus of claim 24, wherein the image pickup device uses a Charge Coupled Device (CCD) image sensor, a Complementary Metal Oxide Semiconductor (CMOS) image sensor, a photo diode array having a two-dimensional or one-dimensional array, or a single photo diode for measuring a distance to a single point.

28. The 3D image acquisition apparatus of claim 24, further comprising a memory which stores a plurality of weights corresponding to at least one of a plurality of combinations of a plurality of pieces of projection light, and a plurality of magnitudes of the optical modulation signal.

29. The 3D image acquisition apparatus of claim 28, wherein when depth information is extracted, the depth image processor reads weights corresponding to combinations of magnitudes and phases of the at least three different pieces of projection light projected onto the object by the light source, or the optical modulation signal from the memory and uses the read weights to calculate the depth information.

30. A 3 Dimensional (3D) image acquisition apparatus comprising:

a light source that projects at least three identical pieces of projection light onto an object;

an optical modulator that modulates at least three pieces of light reflected from the object with at least three optical modulation signals having different gain waveforms;

an image pickup device that generates at least three images by capturing the at least three pieces of light modulated by the optical modulator; and

a depth image processor that calculates a distance to the object by using the image generated by the image pickup device,

wherein the depth image processor extracts depth information by generating a first averaged image by multiplying the at least three images by primary weights and generating a second averaged image by multiplying the at least three images by secondary weights;

acquiring an average phase delay based on the first and second averaged images; and

calculating the distance to the object from the average phase delay.

31. The 3D image acquisition apparatus of claim 30, wherein the at least three optical modulation signals are periodic waves having the same period, and each of the at least three optical modulation signals has at least one of a different magnitude and a different phase as compared to a magnitude and a phase of each of the other at least three optical modulation signals.

32. The 3D image acquisition apparatus of claim 31, wherein the at least three pieces of projection light are periodic waves having a period that is equal to the period of the at least three optical modulation signals.

33. The 3D image acquisition apparatus of claim 30, further comprising a memory which stores a plurality of weights corresponding to at least one of a plurality of combinations of a plurality of pieces of projection light, and a plurality of magnitudes and phases of optical modulation signals.