3D image acquisition apparatus and method of extracting depth information in 3D image acquisition apparatus

• US 9,123,164 B2
• Filed: 02/05/2013
• Issued: 09/01/2015
• Est. Priority Date: 02/07/2012
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Park, Yong-hwa, You, Jang-woo, Yoon, Hee-sun
• Primary Examiner(s): Hoang, Peter
• Application Number:
  US13/759,566
• Publication Number:
  US 20130201183A1
• Time in Patent Office: 938 Days
• Field of Search: 345/419, 348/46
• US Class Current: 1/1
• CPC Class Codes: G01S 17/08 for measuring distance only indirect measurement G01S17/46; active triangulation systems G01S17/48
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G06T 15/08 Volume rendering

Abstract

A method for extracting depth information, which may include extracting depth information relating to an object by using an actual nonlinear waveform instead of a mathematically ideal waveform and simultaneously may include effectively removing errors due to random noises that may be caused in a light source, an light modulator, an imaging device, or the like, is provided. The method may include performing simple multiplex and addition on captured images and performing a search of a look-up table. Thus, correct distance information may be extracted without being affected by a type of waveform being used, and an amount of memory which is used and calculated for removing random noise may be reduced.

Claims

1. A method for extracting depth information, the method comprising:

sequentially projecting a natural number N respective projection light beams toward an object;

modulating N reflection light beams which are correspondingly reflected from the object by using a light modulation signal;

generating N images by photographing the N reflection light beams;

generating a first weighted image U by performing a first multiplying operation which includes multiplying each respective one of the N generated images by respective first weights and adding the weighted images resulting from the first multiplying operation, and generating a second weighted image V by performing a second multiplying operation which includes multiplying each respective one of the N generated images by respective second weights that are respectively different from the corresponding first weights and adding the weighted images resulting from the second multiplying operation; and

calculating a distance relating to the object based on a ratio between the first weighted image U and the second weighted image V by using a look-up table that includes measurement information relating to the N respective projection light beams and the light modulation signal,

wherein the look-up table comprises at least one of;

a first look-up table which includes information relating to a relationship between a phase delay and an arctangent of the ratio between the first weighted image U and the second weighted image V;

a second look-up table which includes information relating to a relationship between the distance relating to the object and the arctangent of the ratio between the first weighted image U and the second weighted image V; and

a third look-up table which includes information relating to a relationship between the distance relating to the object and the ratio between the first weighted image U and the second weighted image V.

2. The method of claim 1, wherein the N respective projection light beams comprise periodic waves having a same period, wherein at least one of waveforms, amplitudes, and phases of the N respective projection light beams vary between respective ones of the N respective projection light beams.

3. The method of claim 2, wherein the light modulation signal has the same period as the N respective projection light beams.

4. The method of claim 1, wherein each of the N respective projection light beams and the light modulation signal has at least one of a sine wave, a triangular wave, a rectangular wave, a square wave, and a general waveform which is obtainable by combining at least two of a sine wave, a triangular wave, a rectangular wave, and a square wave, and which are expressible as a sum of a plurality of high-order sine terms and high-order cosine terms.

5. The method of claim 1, wherein the first weighted image U is generated by using the respective first weights A_l to A_N that respectively correspond to the N images, and the second weighted image V is generated by using the respective second weights B_l to B_N that respectively correspond to the N images.

6. The method of claim 5, wherein the respective first weights and the respective second weights are calculated based on measurements relating to the N respective projection light beams and the light modulation signal.

7. The method of claim 1, wherein each of a plurality of look-up tables corresponds to a respective one of a plurality of possible combinations of the N respective projection light beams and the light modulation signal, and

wherein the calculating the distance relating to the object is performed by using a respective one of the plurality of look-up tables which corresponds to a combination of the N respective projection light beams and the light modulation signal which has been used.

8. The method of claim 1, further comprising:

generating at least one new image by projecting at least one respective projection light beam toward the object, wherein the at least one new image comprises a number of new images which is not greater than (N-1);

removing one of the N images that were originally generated for each of the generated at least one new image;

generating new respective first weighted images and new respective second weighted images by respectively multiplying each respective one of the remaining N images by each of the respective first weights and by each of the respective second weights;

calculating a new ratio between the new respective first weighted images and the new respective second weighted images; and

calculating a new distance which relates to the object, based on the calculated new ratio, by using the look-up table.

9. A three-dimensional (3D) acquisition apparatus comprising:

a light source which projects projection light toward an object;

a light modulator which modulates reflection light that is reflected from the object by using a light modulation signal;

an imaging device which generates an image by photographing light that is modulated by the light modulator; and

a depth information image processor which calculates a distance relating to the object by using the image generated by the imaging device,

wherein the depth information image processor extracts depth information by using the method of claim 1.

10. The 3D acquisition apparatus of claim 9, further comprising:

a light source driver which drives the light source to control a waveform of projection light;

a light modulator driver which drives the light modulator to control a gain waveform of the light modulation signal; and

a controller which controls operations of the light source driver, the light modulator driver, and the imaging device.

11. The 3D acquisition apparatus of claim 9, wherein the imaging device uses a charge-doubled device (CCD) image sensor, a complementary metal-oxide semiconductor (CMOS), and at least one of a single photodiode and a photo diode array, which array includes at least one of a one-dimensional array and a two-dimensional array, for measuring a distance relating to a single point.

12. The 3D acquisition apparatus of claim 9, further comprising a memory which stores the look-up table, and which stores the respective first weights and the respective second weights that are calculated based on measurements relating to the projection light and the light modulation signal.

13. A method for extracting depth information, the method comprising:

sequentially projecting a natural number N identical projection light beams toward an object;

modulating N reflection light beams which are correspondingly reflected from the object by using N respective light modulation signals;

generating N images by photographing the N reflection light beams;

generating a first weighted image U by performing a first multiplying operation which includes multiplying each respective one of the N generated images by respective first weights and adding the weighted images resulting from the first multiplying operation, and generating a second weighted image V by performing a second multiplying operation which includes multiplying each respective one of the N generated images by respective second weights that are respectively different from the corresponding first weights and adding the weighted images resulting from the second multiplying operation; and

calculating a distance relating to the object based on a ratio between the first weighted image U and the second weighted image V by using a look-up table that includes measurement information relating to the N respective light modulation signals and one of the N identical projection light beams,

wherein the look-up table comprises at least one of;

a first look-up table which includes information relating to a relationship between a phase delay and an arctangent of the ratio between the first weighted image U and the second weighted image V;

a second look-up table which includes information relating to a relationship between the distance relating to the object and the arctangent of the ratio between the first weighted image U and the second weighted image V; and

a third look-up table which includes information relating to a relationship between the distance relating to the object and the ratio between the first weighted image U and the second weighted image V.

14. The method of claim 13, wherein each of a plurality of look-up tables corresponds to a respective one of a plurality of possible combinations of the one identical projection light beam and the N respective light modulation signals, and

wherein the calculating the distance relating to the object is performed by using a respective one of the plurality of look-up tables which corresponds to a combination of the one identical projection light beam and the N respective light modulation signals which has been used.

15. A 3D acquisition apparatus comprising:

a light source which projects projection light toward an object;

a light modulator which modulates reflection light that is reflected from the object by using an light modulation signal;

an imaging device which generates an image by photographing light that is modulated by the light modulator; and

a depth information image processor which calculates a distance relating to the object by using the image generated by the imaging device,

wherein the depth information image processor extracts depth information by using the method of claim 13.