IMAGE CAPTURING APPARATUS, SYSTEM AND METHOD

• US 20180089848A1
• Filed: 03/31/2017
• Published: 03/29/2018
• Est. Priority Date: 09/28/2016
• Status: Active Grant

• Current Assignee: STMicroelectronics Incorporated (STMicroelectronics NV)
• Original Assignee: STMicroelectronics Incorporated (STMicroelectronics NV), Stmicroelectronics (Research & Development) Limited (STMicroelectronics NV)
• Inventors: YANG, Xiaoyong, DUTTON, Neale
• Granted Patent: US 10,521,921 B2
• Time in Patent Office: Days
• CPC Class Codes: G01S 17/04 Systems determining the presence of a target
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G01S 7/48 of systems according to group G01S17/00
• CPC Class Codes: G01S 7/4802 using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G01S 7/497 Means for monitoring or calibrating
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 40/166 using acquisition arrangements
• CPC Class Codes: G07C 9/37 using biometric data, e.g. fingerprints, iris scans or voice recognition
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means

Abstract

An apparatus includes time of flight single-photon avalanche diode (ToF SPAD) circuitry. The ToF SPAD circuitry generates indications of distance between the apparatus and an object within a field of view. A processor receives the indications of distance and controls at least one image sensor, such as a camera, to capture at least one image based on at least one indication of distance. The processor determines whether an image is a true representation of an expected object by comparing multiple indications of distance associated with the object to an expected object distance profile and comparing the image to at least one expected object image.

Claims

1. An apparatus, comprising:

time of flight single-photon avalanche diode (ToF SPAD) circuitry, which, in operation, generates multiple indications of distance between the apparatus and at least one object within a field of view of the apparatus, wherein the multiple indications of distance corresponding to respective regions within the field of view of the apparatus; and

processing circuitry, which, in operation,

controls at least one image sensor to capture at least one image based on at least one of the multiple indications of distance between the apparatus and the at least one object within the field of view of the apparatus; and

determines whether the at least one image comprises a true representation of an expected object, the determining including;

comparing the multiple indications of distance to an expected object distance profile; and

comparing the at least one image to at least one expected object image.

2. The apparatus as claimed in claim 1 wherein the processing circuitry, in operation:

determines whether the at least one object is within a field of view of the at least one image sensor based on at least one of the multiple indications of distance; and

controls the at least one image sensor to capture the at least one image based on the determination of whether the at least one object is within the field of view of the at least one image sensor, the at least one image comprising a representation of the object.

3. The apparatus as claimed in claim 2 wherein the processing circuitry, in operation, controls at least one illumination circuit based the determination of whether the at least one object is within the field of view of the at least one image sensor.

4. The apparatus as claimed in claim 1 wherein the processing circuitry, in operation:

controls at least one security function based on the comparison of the at least one image to the at least one expected object image.

5. The apparatus as claimed in claim 1 wherein the ToF SPAD circuitry, in operation, generates multiple indications of distance between the apparatus and the at least one object for a single region within the field of view, and the processing circuitry, in operation:

determines whether a translucent or transparent object is between the at least one image sensor and the at least one object in the field of view of the apparatus; and

controls an alert system based on the determining whether a translucent/transparent object is between the at least one image sensor and the at least one object in the field of view of the apparatus.

6. The apparatus as claimed in claim 1 wherein the ToF SPAD circuitry, in operation, provides at least one of the multiple indications of distance to the at least one image sensor to set a focal distance of the at least one image sensor.

7. The apparatus as claimed in claim 1 wherein,

the at least one image sensor comprises at least two image sensors, the at least two image sensors having at least a partially overlapping field of view with a field of view of the ToF SPAD circuitry;

and

the processing circuitry, in operation;

receives images from each of the at least two image sensors, the images comprising a representation of the object;

determines at least one indication of distance between the at least two image sensors and the at least one object within the field of view of the apparatus based on the images from each of the image sensors and parameters of the at least two image sensors; and

verifies the at least one indication of distance between the at least two image sensors and the at least one object based on at least one of the multiple indications of distance between the apparatus and the at least one object within the field of view of the apparatus.

8. The apparatus of claim 1 wherein the at least one image sensor comprises at least one camera.

9. A method, comprising:

generating, using time of flight single-photon avalanche diode (ToF SPAD) circuitry, multiple indications of distance between the ToF SPAD circuitry and at least one object within a field of view of the ToF SPAD circuitry, the multiple indications of distance corresponding to respective regions within the field of view of the ToF SPAD circuitry;

controlling, using processing circuitry, at least one image sensor to capture at least one image based on at least one of the multiple indications of distance between the ToF SPAD circuitry and the at least one object within the field of view of the ToF SPAD circuitry; and

determining, using the processing circuitry, whether the at least one image comprises a true representation of an expected object, the determining including;

comparing the multiple indications of distance to an expected object distance profile; and

comparing the at least one image to at least one expected object image.

10. The method as claimed in claim 9, comprising:

determining by the processing circuitry, whether the at least one object is within a field of view of the at least one image sensor based on at least one of the multiple indications of distance; and

controlling, by the processing circuitry, the at least one image sensor to capture the at least one image based on the determination of whether the at least one object is within the field of view of the at least one image sensor, the at least one image comprising a representation of the object.

11. The method as claimed in claim 10, comprising:

controlling, by the processing circuitry, at least one illumination circuit based the determination of whether the at least one object is within the field of view of the at least one image sensor.

12. The method as claimed in claim 9, comprising controlling at least one security function based on the comparison of the at least one image to the at least one expected object image.

13. The method of claim 9, comprising providing at least one of the multiple indications of distance to the at least one image sensor to set a focal distance of the at least one image sensor.

14. The method of claim 9 wherein the at least one image sensor comprises at least two image sensors, the at least two image sensors having at least a partially overlapping field of view with a field of view of the ToF SPAD circuitry, and the method comprises:

controlling, using the processing circuitry, the at least two image sensors to capture respective images comprising a representation of the at least one object;

determining, using the processor, at least one indication of distance between the at least two image sensors and the at least one object within the field of view based on the images from each of the at least two image sensors and parameters of the at least two image sensors; and

verifying the at least one indication of distance between the at least two image sensors and the at least one object within the field of view based on at least one of the multiple indications of distance between the apparatus and the at least one object in the field of view of the apparatus.

15. The method of claim 9 wherein the at least one image sensor comprises at least one camera.

16. A system, comprising:

image sensing circuitry, which, in operation, captures images of objects;

time of flight single-photon avalanche diode (ToF SPAD) circuitry, which, in operation, generates indications of distance associated with objects within a field of view, the indications of distance corresponding to respective regions within the field of view; and

processing circuitry, which, in operation;

controls capturing of images by the image sensing circuitry based on indications of distance associated with objects generated by the ToF SPAD circuitry; and

determines whether an image comprises a true representation of an expected object, the determining including;

comparing multiple indications of distance associated with the object to an expected object distance profile; and

comparing the image to at least one expected object image.

17. The system of claim 16, comprising illumination circuitry, wherein the processing circuitry, in operation, controls the illumination circuitry based on indications of distance associated with objects generated by the ToF SPAD circuitry.

18. The system of claim 16, comprising an integrated circuit, the integrated circuit including at least the ToF SPAD circuitry and the image sensing circuitry.

19. The system of claim 18 wherein the image sensing circuitry comprises at least two image sensors, the at least two image sensors having at least a partially overlapping field of view with a field of view of the ToF SPAD circuitry.

20. The system of claim 16, comprising smart phone circuitry.