Determining sparse versus dense pattern illumination

• US 10,650,540 B2
• Filed: 07/17/2019
• Issued: 05/12/2020
• Est. Priority Date: 08/01/2017
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Gernoth, Thorsten, Fasel, Ian R., Guo, Haitao, Kumar, Atulit
• Primary Examiner(s): Chen, Xuemei G
• Application Number:
  US16/513,854
• Publication Number:
  US 20190347813A1
• Time in Patent Office: 300 Days
• CPC Class Codes: G01B 11/14 for measuring distance or clearance between spaced objects or spaced apertures G01B11/26 takes precedence; rangefinders G01C3/00
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/514 from specularities
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/141 Control of illumination
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: H04N 23/50 Constructional details
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

An estimate of distance between a user and a camera on a device is used to determine an illumination pattern density used for speckle pattern illumination of the user in subsequent images. The distance may be estimated using an image captured when the user is illuminated with flood infrared illumination. Either a sparse speckle (dot) pattern illumination pattern or a dense speckle pattern illumination pattern is used depending on the distance between the user'"'"'s face and the camera.

Claims

1. A method, comprising:

capturing a first image using a camera located on a device, the device comprising a computer processor and a memory, wherein the first image comprises an image captured while illuminating with flood infrared illumination from a first illuminator located on the device;

determining a distance between a user and the camera by assessing two-dimensional information in the first image; and

illuminating the user with patterned illumination having a first pattern density from a second illuminator located on the device in response to the determined distance being less than a threshold distance, wherein the second illuminator is configured to project one of the first pattern density or a second pattern density, the first pattern density being less than the second pattern density.

2. The method of claim 1, wherein the threshold distance is between about 20 cm and about 60 cm.

3. The method of claim 1, further comprising capturing a second image of the user, the second image comprising an image captured while illuminating the user with the patterned illumination having the first pattern density.

4. The method of claim 3, further comprising authenticating the user as an authorized user of the device using the first image and the second image.

5. The method of claim 1, further comprising illuminating the user with patterned illumination having the second pattern density from the second illuminator in response to the determined distance being greater than the threshold distance.

6. The method of claim 5, further comprising capturing a second image of the user, the second image comprising an image captured while illuminating the user with the patterned illumination having the second pattern density.

7. The method of claim 1, wherein the distance between the user and the camera is a distance between a face of the user and the camera.

8. The method of claim 1, wherein the distance between the user and the camera is determined by assessing two-dimensional information in the first image using machine learning located on the computer processor.

9. The method of claim 1, wherein the distance between the user and the camera is determined by encoding and decoding, using the computer processor, two-dimensional information in the first image.

10. The method of claim 1, further comprising:

capturing an additional image using the camera, wherein the additional image comprises an image of the user captured while illuminating with flood infrared illumination from the first illuminator;

determining a second distance between the user and the camera by assessing two-dimensional information in the additional image;

illuminating the user with patterned illumination having the first pattern density from the second illuminator in response to the second distance being less than the threshold distance; and

illuminating the user with patterned illumination having the second pattern density from the second illuminator in response to the second distance being greater than the threshold distance.

11. A device, comprising:

a camera;

a first illuminator providing flood infrared illumination;

a second illuminator providing patterned illumination;

circuitry coupled to the camera, the first illuminator, and the second illuminator, wherein the circuitry is programmed to;

capture a first image using the camera while illuminating with flood infrared illumination from the first illuminator;

determine a distance between a user and the camera by assessing two-dimensional information in the first image; and

illuminate the user with patterned illumination having a first pattern density from the second illuminator in response to the determined distance being less than a threshold distance, wherein the second illuminator is configured to project one of the first pattern density or a second pattern density, the first pattern density being less than the second pattern density.

12. The device of claim 11, wherein the threshold distance is between about 20 cm and about 60 cm.

13. The device of claim 11, wherein the camera comprises an infrared sensor.

14. The device of claim 11, wherein the circuitry is programmed to capture a second image of the user while illuminating the user with the patterned illumination.

15. The device of claim 14, wherein the circuitry is programmed to authenticate the user as an authorized user of the device using the first image and the second image.

16. The device of claim 11, wherein the circuitry is programmed to:

illuminate the user with patterned illumination having the second pattern density from the second illuminator in response to the determined distance being greater than the threshold distance; and

capture a second image of the user while illuminating the user with the patterned illumination having the second pattern density.

17. The device of claim 11, wherein the circuitry is programmed to operate machine learning to determine the distance between the user and the camera.

18. The device of claim 11, wherein the second illuminator comprises an array of emitters, the array of emitters comprising a first set of emitters and a second set of emitters that are controlled separately.

19. The device of claim 18, wherein circuitry is programmed to provide the patterned illumination having the first pattern density by driving the first set of emitters on the second illuminator, and wherein circuitry is programmed to provide the patterned illumination having the second pattern density by driving both the first set of emitters and the second set of emitters on the second illuminator.

20. The device of claim 19, wherein the first set of emitters comprises about 40% of a total number of emitters in the array of emitters.