Depth image acquisition apparatus and method of acquiring depth information

• US 10,205,933 B2
• Filed: 02/13/2015
• Issued: 02/12/2019
• Est. Priority Date: 08/29/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Park, Yonghwa
• Primary Examiner(s): Williams, Jeffery A
• Assistant Examiner(s): Young, Patricia I
• Application Number:
  US14/622,283
• Publication Number:
  US 20160065942A1
• Time in Patent Office: 1,460 Days
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G01S 7/4816 of receivers alone
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps

Abstract

A depth image acquisition apparatus and a method of acquiring depth information are provided. The method of acquiring depth information includes: sequentially projecting, to a subject, N different beams of light emitted from a light source for a time period including an idle time for each of the N different beams of transmitted light, where N is a natural number that is equal to or greater than 3; modulating, using a light modulation signal, beams of reflected light that are obtained by reflection of the N different beams from the subject; obtaining N phase images corresponding to the N different beams of light by capturing, using a rolling shutter method, the modulated beams of reflected light; and obtaining depth information by using the obtained N phase images.

Claims

1. A method of obtaining depth information, the method comprising:

sequentially projecting, to a subject, N different beams of light emitted from a light source for a time period comprising an exposure time and an idle time, subsequent to the exposure time, for each of the N different beams of light, where N is a natural number that is equal to or greater than 3;

modulating, using a light modulation signal, beams of reflected light that are obtained by reflection of the N different beams of light from the subject;

obtaining N phase images corresponding to the N different beams of light by capturing the modulated beams of reflected light with each scan line using a rolling shutter method during the exposure time; and

obtaining depth information of the subject by using the obtained N phase images,

wherein the idle time comprises a period of time equivalent to a readout time for all scan lines using the rolling shutter method.

2. A method of obtaining depth information, the method comprising:

projecting, to a subject, light emitted from a light source;

modulating, by sequentially using N light modulation signals having different phases for a time period comprising an exposure time and an idle time, subsequent to the exposure time, for each of the N light modulation signals, reflected light that is obtained by reflection of the light from the subject;

obtaining N phase images corresponding to the N light modulation signals by capturing the modulated reflected light with each scan line using a rolling shutter method during the exposure time; and

obtaining depth information of the subject by using the N phase images,

wherein the idle time comprises a period of time equivalent to a readout time for all scan lines using the rolling shutter method.

3. A method of obtaining depth information, the method comprising:

sequentially projecting, to a subject, N beams of light having different phases, where N is a natural number that is equal to or greater than 3;

modulating, using a light modulation signal, beams of reflected light that are obtained by reflection of the N beams of light from the subject;

reading out N phase images corresponding to the N beams of light by capturing the modulated beams of reflected light with each scan line using a rolling shutter method;

with respect to each scan line, obtaining a first relation equation, for each of the N beams of light, among a readout phase image corresponding to an x-th beam of the N beams of light, a readout time for a previous scan line, and a phase image generated from a single beam of light; and

obtaining, for each scan line, depth information of the subject by using the obtained N first relation equations to reduce distortion caused by the readout phase image comprising a portion of a phase image generated from the x-th beam of light and a portion of a phase image generated from an (x+1)-th beam of light,

wherein the readout phase image comprises the portion of the phase image generated from the x-th beam of light and the portion of the phase image generated from the (x+1)-th beam of light, according to a readout time for a previous scan line.

4. The method of claim 3, wherein the obtaining the depth information comprises:

obtaining a transformation matrix between the readout phase image and the phase image generated from the single beam of light, based on the N first relation equations;

obtaining the phase image generated from the single beam of light by applying, to the readout phase image, an inverse matrix of the obtained transformation matrix; and

obtaining a phase delay due to a time of flight (TOF) by using the phase image generated from the single beam of light.

5. The method of claim 4, wherein the N beams of light having different phases have a same amplitude and period, N is 4, and the different phases comprise 0 degrees, 90 degrees, 180 degrees, and 270 degrees.

6. The method of claim 5, wherein:

the obtaining the phase delay due to the TOF comprises obtaining the phase delay according to;

7. The method of claim 3, wherein the readout phase image has a form that is expressed by:

I_CIS<sup>(θ

)</sup>=r×

A×

cos(ϕ

_TOF+θ

)+r×

B,

wherein θ

denotes a phase of the light, r denotes surface reflectivity of the subject, A denotes a coefficient indicating sizes of the light and the light modulation signal, B denotes a coefficient indicating direct current (DC) components of the light and the light modulation signal, and ϕ

_TOF denotes a phase delay due to a TOF.

8. The method of claim 7, wherein the obtaining the depth information comprises:

obtaining N second relation equations that relate to a difference between the readout phase image corresponding to the x-th beam of the N beams of light and a readout phase image corresponding to the (x+1)-th beam of the N beams of light; and

selecting at least two of the N second relation equations and calculating the phase delay according to;

9. The method of claim 8, wherein the N beams of light having different phases have a same amplitude and period, N is 4, and the different phases comprise 0 degrees, 90 degrees, 180 degrees, and 270 degrees.

10. The method of claim 3, wherein periods of the N beams of light are the same as that of the light modulation signal.

11. The method of claim 3, wherein the reading out the N phase images comprises accumulating the modulated beams of reflected light for a predetermined exposure time with respect to each scan line to read out the N phase images corresponding to the N beams of light.

12. The method of claim 3, wherein the obtaining the depth information comprises:

obtaining a phase delay due to a TOF of the reflected light by using the readout N phase images, with respect to each scan line; and

obtaining a distance from the subject to a depth image acquisition apparatus, which captures an image of the subject, by using the obtained phase delay.

13. The method of claim 3, further comprising generating a depth image by using the obtained depth information.

14. A method of obtaining depth information, the method comprising:

projecting, to a subject, light emitted from a light source;

modulating, by sequentially using N light modulation signals having different phases, reflected light that is obtained by reflection of the transmitted light from the subject;

reading out N phase images corresponding to the N light modulation signals by capturing the modulated reflected light with each scan line using a rolling shutter method;

with respect to each scan line, obtaining a first relation equation, for each of the N light modulation signals, among a readout phase image corresponding to an x-th light modulation signal of the N light modulation signals, a readout time for a previous scan line, and a phase image generated from a single light modulation signal; and

obtaining, for each scan line, depth information of the subject by using the obtained N first relation equations to reduce distortion caused by the readout phase image comprising a portion of a phase image generated from the x-th light modulation signal and the portion of the phase image generated from the (x+1)-th light modulation signal,

wherein the readout phase image comprises the portion of the phase image generated from the x-th light modulation signal and the portion of the phase image generated from the (x+1)-th light modulation signal, according to a readout time for a previous scan line.

15. A depth image acquisition apparatus comprising:

a light projector configured to sequentially project, to a subject, N different beams of light emitted from a light source for a time period comprising an exposure time and an idle time, subsequent to the exposure time, for each of the N different beams of light, where N is a natural number that is equal to or greater than 3;

a light modulator configured to modulate, using a light modulation signal, beams of reflected light that are obtained by reflection of the N different beams of light from the subject;

an image capturer configured to obtain N phase images corresponding to the N different beams of light by capturing the modulated beams of reflected light with each scan line using a rolling shutter method during the exposure time; and

a depth image processor configured to obtain depth information of the subject by using the obtained N phase images,

wherein the idle time comprises a period of time equivalent to a readout time for all scan lines using the rolling shutter method.

16. A depth image acquisition apparatus comprising:

a light projector configured to sequentially project, to a subject, N beams of light having different phases, where N is a natural number that is equal to or greater than 3;

a light modulator configured to modulate, using a light modulation signal, beams of reflected light that are obtained by reflection of the N beams of light from the subject;

an image capturer configured to read out N phase images corresponding to the N beams of light by capturing the modulated beams of reflected light with each scan line using a rolling shutter method; and

a depth image processor configured to, with respect to each scan line, obtain a first relation equation, for each of the N beams of light, among a readout phase image corresponding to an x-th beam of the N beams of light, a readout time for a previous scan line, and a phase image generated from a single beam of light, and configured to obtain, for each scan line, depth information of the subject by using the obtained N first relation equations to reduce distortion caused by the readout phase image comprising a portion of a phase image generated from the x-th beam of light and a portion of a phase image generated from an (x+1)-th beam of light,

wherein the readout phase image comprises the portion of the phase image generated from the x-th beam of light and the portion of the phase image generated from the (x+1)-th beam of light, according to a readout time for a previous scan line.

17. The depth image acquisition apparatus of claim 16, wherein the depth image processor is configured to:

obtain a transformation matrix between the readout phase image and the phase image generated from the single beam of light, based on the N first relation equations;

obtain the phase image generated from the single beam of light by applying, to the readout phase image, an inverse matrix of the obtained transformation matrix; and

obtain a phase delay due to a TOF by using the phase image generated from the single beam of light.

18. The depth image acquisition apparatus of claim 16, wherein the readout phase image has a form that is expressed by:

I_CIS<sup>(θ

)</sup>=r×

A×

cos(ϕ

_TOF+θ

)+r×

B,

wherein θ

denotes a phase of the light, r denotes surface reflectivity of the subject, A denotes a coefficient indicating the sizes of the light and the light modulation signal, B denotes a coefficient indicating direct current (DC) components of the light and the light modulation signal, and ϕ

_TOF denotes a phase delay due to a TOF.

19. The depth image acquisition apparatus of claim 18, wherein the depth image processor is configured to obtain N second relation equations that relate to a difference between the readout phase image corresponding to an x-th beam of the N beams of light and a readout phase image corresponding to the (x+1)-th beam of the N beams of light, and to select at least two of the N second relation equations and calculate a phase delay according to: