Apparatus and method for generating depth image

• US 9,310,488 B2
• Filed: 04/20/2011
• Issued: 04/12/2016
• Est. Priority Date: 09/17/2010
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Park, Yong-Hwa, Kim, Eung-Sun
• Primary Examiner(s): Chio, Tat Chi
• Assistant Examiner(s): TRUONG, NGUYEN T
• Application Number:
  US13/090,683
• Publication Number:
  US 20120069176A1
• Time in Patent Office: 1,819 Days
• Field of Search: 348/135
• US Class Current: 1/1
• CPC Class Codes: G01S 17/36 with phase comparison between the received signal and the contemporaneously transmitted signal
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G01S 7/4816 of receivers alone
• CPC Class Codes: G01S 7/4912 Receivers
• CPC Class Codes: G01S 7/4915 Time delay measurement, e.g. operational details for pixel components signal extraction and conditioning G01S7/493; Phase measurement

Abstract

An apparatus and method for generating a depth image are provided. The apparatus includes an optical modulator, an optical sensor, and a depth image processor. The optical modulator optically modulates light which is reflected from an object to which light modulated using a first pseudorandom (PN) sequence is irradiated. The optical modulator optically modulates the reflected light using second PN-sequences, where each of the second PN-sequences has a same PN sequence as the first PN-sequence, but the second PN-sequences have N different time shifts. The optical sensor senses the optically modulated reflective light. The depth image processor samples N intensity images from the optically modulated reflective light sensed by the optical sensor and generates a depth image of the object using the sampled N intensity images.

Claims

1. An apparatus for generating a depth image, the apparatus comprising:

an optical modulator which optically modulates reflective light which is reflected light from an object to which light modulated using a first pseudorandom (PN) sequence is irradiated, by use of second PN-sequences, each of which has a same PN sequence as the first PN-sequence, but the second PN-sequences having N different time shifts;

an optical sensor which senses the optically modulated reflective light; and

a depth image process unit which samples N intensity images from the optically modulated reflective light sensed by the optical sensor and generates a depth image of the object using the sampled N intensity images,

wherein the depth image process unit individually determines respective depth extraction errors of each of a plurality of depths when the plurality of depths are determined according to a plurality of times of flight of light that are predicted using the N sampled intensity images, and selects a depth of the plurality of depths having a smallest error deviation estimate of the errors as a final depth of the depth image,

wherein the depth image process unit calculates a time of flight of light that is irradiated from a light source and reaches the optical sensor based on the N sampled intensity images, and

wherein when N is a number that is equal to or greater than 3, the depth image process unit determines whether an intensity image I_M havin a minimum intensity among the three or more sampled intensity images is a first captured intensity image I₁ or a thirdly captured intensity image I₃, and calculates a depth according to the determination result.

2. The apparatus of claim 1, wherein the first PN-sequence and the second PN-sequences are bipolar m-sequences, and

a minimum information unit transmission time T_C and a length of sequence of the first PN-sequence are the same as a minimum information unit transmission time T_C and a length of sequence, respectively, of the second PN-sequences.

3. The apparatus of claim 1, wherein the first PN-sequence and the second PN-sequence are Walsh-codes or gold-sequences.

4. The apparatus of claim 1, wherein the N intensity images that are sampled correspond respectively to the N different time shifts of the second PN-sequences.

5. The apparatus of claim 1, wherein the depth image process unit models the sampled N intensity images using a function including the time of flight of light that is irradiated from the light source and reaches the optical sensor as an unknown quantity.

6. The apparatus of claim 5, wherein the function including the time of flight as an unknown quantity further includes ambient light and reflectivity of the reflective light as unknown quantities.

7. The apparatus of claim 5, wherein when four or more intensity images are sampled, the depth image process unit determines which intensity image I_M has a minimum intensity and which two intensity images have two top intensities which are different from the minimum intensity among the four or more intensity images, and calculates a depth according to the determination result.

8. The apparatus of claim 5, wherein the depth image process unit further determines the final depth by averaging the plurality of depths when the plurality of depths are determined according to the plurality of times of flight of light that are predicted using the N sampled intensity images.

9. The apparatus of claim 1, further comprising:

the light source which emits light to an object;

a light source driver which drives the light source;

an optical modulator driver which drives the optical modulator; and

a control unit which controls the light source driver to generate a light driving waveform of the light source using the first PN-sequence, and controls the optical modulator driver to generate an optical modulation driving waveform of the optical modulator using the second PN-sequence.

10. The apparatus of claim 9, wherein the depth image process unit increases a number of times of sampling the intensity images for expanding a capturing range of the depth image while the control unit maintains a constant interval between time shifts.

11. The apparatus of claim 9, wherein the control unit increases precision of the depth image by controlling the light source driver and the optical modulator driver such that an interval between the time shifts is shorter than a minimum information unit transmission time T_C of the first and second PN-sequences, and the depth image process unit samples the intensity images at the interval of the time shift.

12. The apparatus of claim 1, wherein the first PN-sequence and the second PN-sequence are different from those of a different apparatus for generating a depth image.

13. The apparatus of claim 1, wherein the first PN-sequence and the second PN-sequence are the same as those of a different apparatus for generating a depth image, but the first PN-sequence and the second PN-sequence are not synchronized with those of the different apparatus.

14. An apparatus for generating a depth image, the apparatus comprising:

an optical modulator which optically modulates reflective light which is reflected light from an object to which light modulated using N first pseudorandom (PN) sequences having, respectively, N different time shifts is irradiated, by use of a second PN-sequence which has a same PN sequence as that of the first PN-sequences but has a time shift different from those of the first PN-sequences;

an optical sensor which senses the optically modulated reflective light; and

a depth image process unit which samples N intensity images from light signals sensed by the optical sensor and generates a depth image of the object using the sampled N intensity images,

wherein the depth image process unit individually determines respective depth extraction errors of each of a plurality of depths when the plurality of depths are determined according to a plurality of times of flight of light that are predicted using the N sampled intensity images, and selects a depth of the plurality of depths having a smallest error deviation estimate of the errors as a final depth of the depth image,

wherein the depth image process unit calculates a time of flight of light that is irradiated from a light source and reaches the optical sensor based on the N sampled intensity images, and

wherein when N is a number that is equal to or greater than 3, the depth image process unit determines whether an intensity image I_M havin a minimum intensity among the three or more sampled intensity images is a first captured intensity image I₁ or a thirdly captured intensity image I₃, and calculates a depth according to the determination result.

15. The apparatus of claim 14, wherein the depth image process unit models the sampled N intensity images with a function including the time of flight of light that is irradiated from the light source and reaches the optical sensor as an unknown quantity.

16. A method of generating a depth image, the method comprising:

performing optical modulation on reflective light which is reflected light from an object to which light modulated using a first pseudorandom (PN) sequence is irradiated, by use of a second PN-sequence which has a same PN sequence as a first PN-sequence but the second PN-sequence has N different time shifts;

sensing the optically modulated reflective light using an optical sensor;

sampling N intensity images obtained from the sensed optically modulated reflective light; and

generating a depth image of the object using the N sampled intensity images,

wherein the generating comprises individually determining respective depth extraction errors of each of a plurality of depths when the plurality of depths are determined according to a plurality of times of flight of light that are predicted using the N sampled intensity images, and selecting a depth of the plurality of depths having a smallest error deviation estimate of the errors as a final depth of the depth image, and

wherein the method further comprises;

calculating a time of flight of light that is irradiated from a light source and reaches the optical sensor based on the N sampled intensity images, and

when N is a number that is equal to or greater than 3, determining whether an intensity inriageavin a minimum intensity among the three or more sampled intensity images is a first captured intensity image I₁ or a thirdly captured intensity image I₃, and calculating a depth according to the determination result.

17. The method of claim 16, wherein the first PN-sequence and the second PN-sequence are the same as those of a different apparatus for generating a depth image, but the first PN-sequence and the second PN-sequence are not synchronized with those of the different apparatus.

18. A method of generating a depth image, the method comprising:

irradiating light modulated using a first pseudorandom (PN) sequence toward an object;

optically modulating, using a plurality of second PN-sequences, light that is reflected off of the object, the second PN-sequences having a same PN-sequence as the first PN sequence;

sensing the optically modulated light using an optical sensor;

sampling N intensity images from the sensed optically modulated light; and

generating a depth image of the object using the N sampled intensity images,

wherein the second PN-sequences have N different time shifts, and

wherein the generating comprises individually determining respective depth extraction errors of each of a plurality of depths when the plurality of depths are determined according to a plurality of times of flight of light that are predicted using the N sampled intensity images, and selecting a depth of the plurality of depths having a smallest error deviation estimate of the errors as a final depth of the depth image, and

wherein the method further comprises;

calculating a time of flight of light that is irradiated from a light source and reaches the optical sensor based on the N sampled intensity images, and

wherein when N is a number that is equal to or greater than 3, determining whether an intensity image I_M having a minimum intensity among the three or more sampled intensity images is a first captured intensity image I₁ or a thirdly captured intensity image I₃, and calculating a depth according to the determination result.

19. The method of claim 18, wherein N is equal to or greater than four.