Distance estimation using multi-camera device

• US 10,691,967 B2
• Filed: 10/02/2019
• Issued: 06/23/2020
• Est. Priority Date: 04/08/2013
• Status: Active Grant

• Current Assignee: Snap, Inc.
• Original Assignee: Snap, Inc.
• Inventors: Zak, Eyal, Egri, Ozi
• Primary Examiner(s): Rush, Eric
• Application Number:
  US16/591,292
• Publication Number:
  US 20200042815A1
• Time in Patent Office: 265 Days
• Field of Search: 382100, 382106, 382154, 382254, 382255, 382276, 382284, 382291, 382294, 382312, 382313, 348 42, 348 46, 348 50, 348135, 348140, 348142, 396 89, 396147-150, 396281, 396296, 396322-325, 396332
• CPC Class Codes: G01C 3/08 Use of electric radiation detectors
• CPC Class Codes: G01C 3/085 with electronic parallax measurement
• CPC Class Codes: G03B 35/02 by sequential recording
• CPC Class Codes: G03B 35/10 having single camera with stereoscopic-base-defining system
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30041 Eye; Retina; Ophthalmic
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 10/17 using hand-held instruments
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/51 Housings

Abstract

A method and apparatus for measuring a distance to an object, be a device (100) having at least two cameras (104, 106), is described. One or more first images including the object are acquired by a first camera of the device and one or more first reference images are acquired by a second camera of the device, while the device is in a first position. One or more second images including the object and one or more second reference images are acquired by cameras of the device, while the device is in a second position, different from the first position. Based on the first and second reference images, information on the displacement of at least one camera of the device between the first and second position are determined. The distance from the device to the object is calculated based on the first and second images including the object and the determined information on the displacement of the at least one camera.

Claims

1. A method of measuring a distance to an object, by a device having at least two cameras, comprising:

acquiring, by a first camera of the device while the device is in a first position, one or more first images that include the object;

acquiring, by a second camera of the device while the device is in the first position, one or more first reference images;

after acquiring the one or more first images, displaying, to a user, a stop instruction in response to a numerical score exceeding a dynamically adjustable threshold, the numerical score indicating an extent of matching between the one or more first reference images and a current image acquired by the second camera, and the dynamically adjustable threshold being according to an accuracy level for distance determination;

acquiring, by the first camera of the device while the device is in a second position, one or more second images including the object;

acquiring, by the second camera of the device while the device is in the second position, one or more second reference images;

determining, based on the one or more first and second reference images, information on a displacement of at least one camera of the device between the first and second positions; and

calculating the distance from the device to the object, based on the one or more first and second images including the object and the determined information on the displacement of the at least one camera, the calculation based on a fixed measurement.

2. The method of claim 1, wherein acquiring the one or more first images including the object comprises acquiring a plurality of images while the device is in the first position and generating a single image including the object from the plurality of acquired images.

3. The method of claim 2, wherein generating the single image including the object from the plurality of acquired images comprises selecting one of the plurality of acquired images.

4. The method of claim 2, wherein generating the single image including the object from the plurality of acquired images comprises generating a combination of some or all of the plurality of acquired images.

5. The method of claim 2, wherein generating the single image including the object from the plurality of acquired images comprises analyzing quality of the plurality of acquired images and discarding images of a quality below a given threshold.

6. The method of claim 1, comprising displaying to the user instructions directing to change a position of the device to the second position, after acquiring the one or more first images.

7. The method of claim 1, wherein the fixed measurement comprises a size of a body organ and wherein the body organ is included in the one or more first and second reference images.

8. The method of claim 7, wherein the body organ comprises a cornea.

9. The method of claim 1, wherein the fixed measurement comprises the distance between the first and second cameras of the at least two cameras.

10. The method of claim 1, wherein the first and second cameras of the at least two cameras are on opposite sides of the device.

11. The method of claim 1, wherein the one or more first images including the object and the one or more first reference images are acquired in pairs of reference and object images, the reference and object images of each pair being acquired concurrently.

12. A device, comprising:

a housing;

a display;

a first camera included in the housing;

a second camera included in the housing; and

a processor, included in the housing, configured to;

receive one or more first images including an object, acquired by the first camera while the device is in a first position;

receive one or more first reference images acquired by the second camera while the device is in the first position;

after acquiring the one or more first images, display, by the display of the device, a stop instruction in response to a numerical score exceeding a dynamically adjustable threshold, the numerical score indicating an extent of matching between the one or more first reference images and a current image acquired by the second camera, and the dynamically adjustable threshold being according to an accuracy level for distance determination;

receive one or more second images including the object acquired by the first camera while the device is in a second position;

receive one or more second reference images acquired by the second camera while the device is in the second position;

determine information on a displacement of at least one camera of the device between acquiring the one or more first images and acquiring the one or more second images, responsively to the one or more first and second reference images; and

calculate a distance from the device to the object based on the one or more first and second images and the determined information on the displacement of the at least one camera, the calculation based on a fixed measurement.

13. The device according to claim 12, wherein the first and second cameras are on opposite sides of the housing.

14. The device according to claim 12, wherein the processor is configured to control the first and second cameras to acquire the one or more first images and the one or more first reference images in pairs of concurrently acquired images and respective reference images.

15. The device according to claim 12, comprising a screen located on a same side of the housing as the second camera.

16. The device according to claim 12, wherein the processor is configured to provide, after receiving the one or more first images and the one or more first reference images, instructions on changing a position of the housing, for acquiring the one or more second images and the one or more second reference images.

17. The device according to claim 12, wherein the processor is configured to analyze an input stream provided by the second camera relative to the one or more first reference images, after receiving the one or more first images and the one or more first reference images, in order to determine when the housing is in a position suitable for acquiring the one or more second reference images and the one or more second images.

18. The device according to claim 12, comprising a screen and wherein the processor is configured to display on the screen, after receiving the one or more first images and the one or more first reference images, a stream of images acquired by the second camera, in order to allow a user to position the housing for acquiring the one or more second images and the one or more second reference images.

19. The device according to claim 12, wherein the processor is configured to determine an extent of relative rotation between the one or more first and second reference images and use the determined extent of relative rotation in determining the information on the displacement of the at least one camera between acquiring the one or more first images and acquiring the one or more second images.

20. A computer program product for distance estimation, comprising:

a computer readable non-transitory storage medium having computer readable program code embodied therewith, which when executed by a computer performs;

receiving one or more first images including an object acquired by a first camera of a device while the device is in a first position;

receiving one or more first reference images acquired by a second camera while the device is in the first position;

after acquiring the one or more first images, display, by a display of the device, a stop instruction in response to a numerical score exceeding a dynamically adjustable threshold, the numerical score indicating an extent of matching between the one or more first reference images and a current image acquired by the second camera, and the dynamically adjustable threshold being according to an accuracy level for distance determination;

receiving one or more second images including the object acquired by the first camera while the device is in a second position;

receiving one or more second reference images acquired by the second camera while the device is in the second position;

determining, based on the one or more first and second reference images, information on a displacement of at least one camera of the device between acquiring the one or more first and one or mom second reference images; and

calculating a distance from the device to the object, based on the one or more first and second images including the object and the determined information on the displacement of the at least one camera, the calculation based on a fixed measurement.