Adaptive low-light identification

• US 9,723,224 B2
• Filed: 05/21/2014
• Issued: 08/01/2017
• Est. Priority Date: 03/31/2014
• Status: Expired due to Fees

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Inventors: Willis, Lawrence A., Pierce, William S., Slaby, Jiri
• Primary Examiner(s): Walker, Jared
• Application Number:
  US14/283,276
• Publication Number:
  US 20150281599A1
• Time in Patent Office: 1,168 Days
• Field of Search: 348164
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/667 Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 5/265 Mixing

Abstract

This document describes techniques and apparatuses for implementing adaptive low-light identification. These techniques and apparatuses enable a computing device to capture infrared (IR) imagery of a person and determine, based on previously-captured IR imagery of multiple people, an identity of the person. Visible-light imagery of the identified person can then be retrieved and combined with the IR imagery to provide composite imagery of the person, which is presented to a user. By so doing, the user can more-easily identify the person, particularly in low-light conditions.

Claims

1. A method comprising:

receiving, by an electronic device, previously captured infrared imagery of one or more people;

associating, by the electronic device, one or more respective identities with the one or more people included in the previously captured infrared imagery;

receiving, by the electronic device, using an infrared-enabled sensor, infrared imagery of a person;

comparing, by the electronic device, the infrared imagery of the person with the previously captured infrared imagery of the one or more people;

responsive to determining, by the electronic device, and based on the comparing, a match between at least one feature of the infrared imagery of the person and a corresponding at least one feature of the previously captured infrared imagery of the one or more people, determining, by the electronic device, based on the match and on the one or more respective identities associated with the one or more people included in the previously captured infrared imagery, an identity of the person;

receiving, by the electronic device, based on the identity of the person, previously captured visible-light imagery of the person, wherein the previously captured visible-light imagery of the person and the previously captured infrared imagery of the one or more people are captured prior to the electronic device receiving the infrared imagery of the person;

combining the infrared imagery of the person and the previously captured visible-light imagery of the person to provide composite imagery of the person; and

presenting, by the electronic device for display, the composite imagery of the person.

2. The method of claim 1, wherein determining the identity of the person is further based on a location of the electronic device or behavioral information known about the one or more people.

3. The method of claim 2, further comprising determining the location of the electronic device by one of:

determining, by the electronic device, the location using a global positioning system of the electronic device;

determining, by the electronic device, the location using an association with one or more known wireless networks;

determining, by the electronic device, the location using geo-tagged elements within the infrared imagery of the person;

or

determining, by the electronic device, the location based on a previously known location and by using a map and inertial data of the electronic device.

4. The method of claim 1, wherein the method is implemented in response to determining, by the electronic device using an ambient light sensor, that a level of ambient light is below a pre-determined threshold.

5. The method of claim 1, wherein the infrared-enabled sensor of the electronic device is configured to capture illuminated near-infrared light or a long-wavelength infrared thermal image.

6. The method of claim 1, wherein the electronic device comprises one of a smart phone, tablet computer, wearable computer, or vehicle-based computing system.

7. A method comprising:

receiving, by an electronic device using an infrared-enabled sensor, infrared imagery of both a location and a person at the location;

receiving, by the electronic device, based on location and orientation information associated with the electronic device, previously captured imagery of the location;

integrating, by the electronic device, objects of the previously captured imagery of the location with the infrared imagery of both the location and the person to provide composite imagery of the location;

receiving, by the electronic device, previously captured infrared imagery of one or more people;

comparing, by the electronic device, the infrared imagery of both the location and the person with the previously captured infrared imagery of the one or more people;

responsive to determining, by the electronic device, and based on the comparing, a match between at least one feature of the infrared imagery of both the location and the person and a corresponding at least one feature of the previously captured infrared imagery of the one or more people, determining, by the electronic device, based on the match and on one or more respective identities associated with the one or more people in the previously captured infrared imagery of the one or more people, an identity of the person;

receiving, by the electronic device, based on the identity of the person, previously captured visible-light imagery of the person, wherein the previously captured visible-light imagery of the person and the previously captured infrared imagery of the one or more people are captured prior to the electronic device receiving the infrared imagery of both the location and the person;

integrating, by the electronic device, the visible-light imagery of the person with the infrared imagery of both the location and the person to provide composite imagery of the person; and

presenting, by the electronic device and for display at the electronic device, the composite imagery of the location and the composite imagery of the person.

8. The method of claim 7, further comprising:

determining, by the electronic device using a global positioning system, the location associated with the electronic device; and

determining, by the electronic device using an accelerometer or a magnetometer, the orientation information associated with the electronic device.

9. The method of claim 7, wherein receiving the previously captured infrared imagery of the one or more people comprises receiving the previously captured imagery of the one or more people from a local repository or a remote repository.

10. The method of claim 7, wherein, prior to receiving the infrared imagery of both the location and the person, the electronic device receives at least one of:

the previously captured imagery of the location;

the previously captured infrared imagery of the one or more people;

or

the previously captured visible-light imagery of the person.

11. The method of claim 7, wherein for each of the one or more people, the respective previously captured infrared imagery comprises an infrared image template that includes at least one thermal image associated with a respective person and information to identify the respective person.

12. The method of claim 11, wherein receiving the visible-light imagery of the person comprises accessing a local memory or a remote location storing a visible-light image template that includes at least one visible-light image associated with the person and information associating the at least one visible-light image with the person.

13. The method of claim 7, wherein the electronic device comprises one of a smart phone, tablet computer, wearable computer, or vehicle-based computing system.

14. An apparatus comprising:

one or more processors;

an infrared-enabled sensor configured to capture infrared imagery;

a display configured to present content; and

at least one non-transitory computer-readable storage medium storing instructions that are executable by the one or more processors to;

receive, via the infrared-enabled sensor, infrared imagery of both a location and a person at the location;

receive previously captured imagery of a location of the apparatus;

integrate the previously captured imagery of the location with the infrared imagery of both the location and the person to provide composite imagery of the location;

receive previously captured infrared imagery of one or more people;

compare the infrared imagery of both the location and the person with the previously captured infrared imagery of the one or more people;

responsive to determining, based on the comparing, a match between at least one feature of the infrared imagery of both the location and the person and a corresponding at least one feature of the previously captured infrared imagery of the one or more people, determine, based on the match and on one or more respective identities associated with the one or more people in the previously captured infrared imagery of the one or more people, an identity of the person;

receive, based on the identity of the person, previously captured visible-light imagery of the person, wherein the previously captured visible-light imagery of the person and the previously captured infrared imagery of the one or more people are captured prior to the electronic device receiving the infrared imagery of both the location and the person;

integrate the infrared imagery of both the location and the person with the previously captured visible-light imagery of the person to provide composite imagery of the person; and

present, via the display, the composite imagery of the location and the composite imagery of the person.

15. The apparatus of claim 14, further comprising a global positioning system module, and wherein the instructions stored on the at least one non-transitory computer-readable storage medium are further executable by the one or more processors to determine the location of the apparatus using the global positioning system module.

16. The apparatus of claim 14, wherein the previously captured visible-light imagery of the person comprises a visible-light image of the person, and wherein the instructions stored on the at least one non-transitory computer-readable storage medium are further executable by the one or more processors to:

receive, prior to receiving the infrared imagery of the person, the visible-light image of the person;

or

retrieve, from an online repository, the visible-light image of the person.

17. The apparatus of claim 14, wherein the at least one feature of the previously captured infrared imagery of the one or more people includes at least one of body shape, body size, walking gait, jewelry, clothing, or facial biometrics.

18. The apparatus of claim 14, further comprising a sensor configured to detect a level of ambient light, and wherein the instructions stored on the at least one non-transitory computer-readable storage medium are further executable by the one or more processors to disable, in response to determining that the level of ambient light is below a predefined threshold, a visible-light view mode of the apparatus and present the composite imagery of the person.

19. The apparatus of claim 14, wherein the apparatus is configured in whole, or in part, as one of a smart phone, tablet computer, wearable computer, or vehicle-based computing system.

20. An apparatus comprising:

one or more processors;

an infrared-enabled sensor configured to capture infrared imagery;

a display configured to present content; and

at least one non-transitory computer-readable storage medium storing instructions that are executable by the one or more processors to;

receive previously captured infrared imagery of one or more people;

associate one or more respective identities with the one or more people included in the previously captured infrared imagery;

receive, using the infrared-enabled sensor, infrared imagery of a person;

compare the infrared imagery of the person with the previously captured infrared imagery of the one or more people;

responsive to determining, based on the comparing, a match between at least one feature of the infrared imagery of the person and a corresponding at least one feature of the previously captured infrared imagery of the one or more people, determine, based on the match and on the one or more respective identities associated with the one or more people included in the previously captured infrared imagery, an identity of the person;

receive, based on the identity of the person, previously captured visible-light imagery of the person, wherein the previously captured visible-light imagery of the person and the previously captured infrared imagery of the one or more people are captured prior to the apparatus receiving the infrared imagery of the person;

combine the infrared imagery of the person and the previously captured visible-light imagery to provide composite imagery of the person; and

present, via the display, the composite imagery of the person.