DISTANCE MEASURING DEVICE AND METHOD OF CONTROLLING DISTANCE MEASURING DEVICE

• US 20180292206A1
• Filed: 07/10/2017
• Published: 10/11/2018
• Est. Priority Date: 08/30/2016
• Status: Active Grant

• Current Assignee: Sony Semiconductor Solutions Corporation (Sony Group Corp.)
• Original Assignee: Sony Semiconductor Solutions Corporation (Sony Group Corp.)
• Inventors: Ohki, Mitsuharu
• Granted Patent: US 10,641,606 B2
• Time in Patent Office: Days
• CPC Class Codes: G01C 3/06 Use of electric means to obtain final indication
• CPC Class Codes: G01S 17/36 with phase comparison between the received signal and the contemporaneously transmitted signal
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G01S 17/93 for anti-collision purposes
• CPC Class Codes: G01S 7/4914 of detector arrays, e.g. charge-transfer gates
• CPC Class Codes: G01S 7/493 Extracting wanted echo signals
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/20028 Bilateral filtering
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06T 7/557 from light fields, e.g. from plenoptic cameras
• CPC Class Codes: H04N 23/672 based on the phase difference signals
• CPC Class Codes: H04N 25/771 comprising storage means other than floating diffusion
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Image quality of a depth map is improved in a distance measuring device for receiving intermittent light.

In a pixel array unit, a plurality of pixels for receiving predetermined intermittent light is arranged in a two-dimensional lattice pattern. A continuous light image data generating unit generates continuous light image data in which a plurality of pieces of pixel data indicating a luminance of predetermined continuous light is arranged in a two-dimensional lattice pattern on the basis of received light data of each of the plurality of pixels. A depth map generating unit generates a depth map in which distance information corresponding to each of the plurality of pixels is arranged, from the received light data and the continuous light image data.

Claims

1. A distance measuring device comprising:

a pixel array unit including a plurality of pixels which receives predetermined intermittent light and is arranged in a two-dimensional lattice pattern;

a continuous light image data generating unit configured to generate continuous light image data in which a plurality of pieces of pixel data indicating a luminance of predetermined continuous light is arranged in a two-dimensional lattice pattern on the basis of received light data of each of the plurality of pixels; and

a depth map generating unit configured to generate a depth map in which distance information corresponding to each of the plurality of pixels is arranged on the basis of the received light data and the continuous light image data.

2. The distance measuring device according to claim 1, wherein

each piece of the received light data includes a plurality of charge signals, and

the continuous light image data generating unit generates the continuous light image data by integrating the plurality of charge signals for each of the plurality of pixels.

3. The distance measuring device according to claim 1, wherein

the depth map generating unit obtains a charge signal when a predetermined function, to which the plurality of charge signals and the continuous light image data have been input, is minimized to generate the depth map from the charge signals.

4. The distance measuring device according to claim 1, wherein

the depth map generating unit generates the depth map by using a predetermined filter.

5. The distance measuring device according to claim 4, wherein

the pixel array unit is divided into a plurality of pixel blocks, and

the depth map generating unit includes:

a low resolution charge signal generating unit which generates a plurality of charge signals for each of the plurality of pixel blocks by pixel addition to the received light data as low resolution charge signals;

a low resolution continuous light image data generating unit which integrates the low resolution charge signal for each of the plurality of pixel blocks and outputs low resolution continuous light image data in which data according to the integrated value is arranged;

a low resolution depth map generating unit which generates data in which distance information corresponding to each of the plurality of pixel blocks is arranged as a low resolution depth map on the basis of the low resolution charge signal; and

a high resolution depth map generating unit which inputs the continuous light image data, the low resolution continuous light image data, and the low resolution depth map to the predetermined filter to generate the depth map.

6. The distance measuring device according to claim 4, wherein

the pixel array unit is divided into a plurality of pixel blocks, and

the depth map generating unit includes:

a low resolution charge signal generating unit which generates a plurality of charge signals for each of the plurality of pixel blocks by pixel addition to the received light data as low resolution charge signals;

a low resolution continuous light image data generating unit which integrates the low resolution charge signal for each of the plurality of pixel blocks and outputs low resolution continuous light image data in which data according to the integrated value is arranged;

a high resolution charge signal generating unit which inputs the continuous light image data, the low resolution continuous light image data, and the low resolution charge signals to the predetermined filter to generate a new charge signal for each of the plurality of pixels as a high resolution charge signal; and

a depth converting unit which converts the high resolution charge signal into the distance information.

7. The distance measuring device according to claim 4, wherein

the depth map generating unit includes:

a depth converting unit which converts the plurality of charge signals into the distance information; and

a depth map noise removing unit which inputs the distance information and the continuous light image data to the predetermined filter to remove a noise of the distance information.

8. The distance measuring device according to claim 4, wherein

the depth map generating unit includes:

a charge signal noise removing unit which inputs the continuous light image data and the plurality of charge signals to the predetermined filter to remove a noise of the plurality of charge signals; and

a depth converting unit which converts the plurality of charge signals from which the noise has been removed into the distance information.

9. The distance measuring device according to claim 8, wherein

the plurality of charge signals may include a first, second, third, and fourth charge signals, and

the predetermined filter is a filter in which a weighting coefficient which gets smaller as an absolute difference value between a sum of the first and the second charge signals and a sum of the third and the fourth charge signals is larger is multiplied by a predetermined Gaussian function.

10. The distance measuring device according to claim 8, wherein

the plurality of charge signals may include a first, second, third, and fourth charge signals, and

the predetermined filter is a filter in which a weighting coefficient which gets larger as a sum of a square of a difference between the first and the second charge signals and a square of a difference between the third and the fourth charge signals is larger is multiplied by a predetermined Gaussian function.

11. The distance measuring device according to claim 8, wherein

the predetermined filter includes a Gaussian function in which a coefficient which gets larger as a difference between a phase of the intermittent light before a noise is removed and a phase of the intermittent light from which the noise has been removed is smaller is multiplied by a term of a standard deviation.

12. The distance measuring device according to claim 1, wherein

the pixel array unit receives the predetermined intermittent light in a predetermined period and receives the predetermined continuous light in a continuous light receiving period which is different from the predetermined period, and

the continuous light image data generating unit generates the continuous light image data from the received light data in the continuous light receiving period.

13. A method of controlling a distance measuring device, the method comprising:

a continuous light image data generating procedure of generating continuous light image data in which a plurality of pieces of pixel data indicating a received light amount of predetermined continuous light is arranged in a two-dimensional lattice pattern on the basis of received light data of each of a plurality of pixels in a pixel array unit in which the plurality of pixels for receiving predetermined intermittent light is arranged in a two-dimensional lattice pattern; and

a depth map generating procedure of generating a depth map in which distance information corresponding to each of the plurality of pixels is arranged from the received light data and the continuous light image data.