Enhanced face recognition in video

• US 9,355,301 B2
• Filed: 11/24/2014
• Issued: 05/31/2016
• Est. Priority Date: 06/10/2011
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Karakotsios, Kenneth M., Fu, Kah Kuen, Ivanchenko, Volodymyr V., Huang, Mingjing
• Primary Examiner(s): Desire, Gregory M
• Application Number:
  US14/552,378
• Publication Number:
  US 20150078623A1
• Time in Patent Office: 554 Days
• US Class Current: 1/1
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/166 using acquisition arrangements
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction

Abstract

The computational resources needed to perform processes such as image recognition can be reduced by determining appropriate frames of image information to use for the processing. In some embodiments, infrared imaging can be used to determine when a person is looking substantially towards a device, such that an image frame captured at that time will likely be adequate for facial recognition. In other embodiments, sound triangulation or motion sensing can be used to assist in determining which captured image frames to discard and which to select for processing based on any of a number of factors indicative of a proper frame for processing.

Claims

1. A computer-implemented method of selecting at least one image frame for facial recognition, comprising:

under the control of an electronic device comprising at least one infrared (IR) imaging mechanism and configured with executable instructions,

detecting at least one feature in at least one image, the feature being indicative of a person'"'"'s head;

activating the at least one IR imaging mechanism of the electronic device operable to emit IR light;

detecting features corresponding to pupils in reflected IR light;

analyzing the detected features to determine at least one attribute of the detected features in the reflected IR light; and

selecting a frame of the at least one image when at least one attribute of the detected features meets a specified criterion.

2. The computer-implemented method of claim 1, further comprising determining that detected IR reflections are indicative of the detected features corresponding to human eyes.

3. The computer-implemented method of claim 2, further comprising:

determining an amount of separation between the human eyes; and

analyzing the separation between the human eyes to determine whether a person is looking towards the electronic device.

4. The computer-implemented method of claim 3, further comprising:

determining that the person is looking towards the electronic device; and

selecting the frame of the at least one image.

5. The computer-implemented method of claim 1, the at least one IR imaging mechanism operable to emit pulses of IR light at a specified frequency of emission.

6. The computer-implemented method of claim 1, further comprising:

detecting motion near the electronic device before initiating a capture of the at least one image.

7. The computer-implemented method of claim 1, further comprising:

detecting at least one feature from supplemental information indicative of a person'"'"'s head.

8. The computer-implemented method of claim 1, wherein the specified criterion includes a threshold amount of lighting or contrast in the frame of the at least one image, positions of the detected features relative to the person'"'"'s head, or sizes and shapes of the detected features.

9. A computer-implemented method of selecting at least one image frame for facial recognition, comprising:

under the control of an electronic device comprising at least one infrared (IR) imaging mechanism and configured with executable instructions,

detecting at least one feature in image data, the feature being indicative of a person'"'"'s head;

activating the at least one IR imaging mechanism of the electronic device operable to emit IR light;

detecting features corresponding to a person'"'"'s corneas in reflected IR light;

initiating a capture of at least one image on the electronic device; and

selecting a frame of the at least one image based on a specified selection criterion.

10. The computer-implemented method of claim 9, further comprising determining that detected IR reflections are indicative of the detected features corresponding to human eyes.

11. The computer-implemented method of claim 10, further comprising:

determining an amount of separation between the human eyes; and

analyzing the separation between the human eyes to determine whether a person is looking towards the electronic device.

12. The computer-implemented method of claim 11, further comprising:

determining that the person is looking towards the electronic device; and

selecting the frame of the at least one image.

13. The computer-implemented method of claim 10, further comprising:

detecting at least one feature from supplemental information indicative of a person'"'"'s head.

14. The computer-implemented method of claim 9, the at least one IR imaging mechanism operable to emit pulses of IR light at a specified frequency of emission.

15. The computer-implemented method of claim 9, further comprising:

detecting motion near the electronic device before initiating a capture of the at least one image.

16. A computer system, comprising:

one or more processors;

a memory device including instructions that, when executed by the one or more processors, cause the computing system to;

detecting at least one feature in at least one image frame, the feature being indicative of a person'"'"'s head;

activate at least on infrared (IR) imaging mechanism of an electronic device operable to illuminate a person with at least one wavelength of IR light and detect IR light reflected back from the person;

detect features corresponding to the person in the reflected IR light; and

select a frame of the at least one image when the detected features meet a specified selection criterion.

17. The computer system of claim 16, wherein the memory device further includes instructions that determine that the detected IR reflections are indicative of the detected features corresponding to human eyes.

18. The computer system of claim 17, wherein the memory device further includes instructions that:

determine an amount of separation between the human eyes; and

analyze the separation between the human eyes to determine whether the person is looking towards the electronic device.

19. The computer system of claim 18, wherein the memory device further includes instructions that:

determine that the person is looking towards the electronic device; and

select the frame of the at least one image.

20. The computer system of claim 16, wherein the memory device further includes instructions that:

detect motion near the electronic device before initiating a capture of the at least one image.

21. The computer system of claim 16, wherein the memory device further includes instructions that:

detect at least one feature from supplemental information indicative of a person'"'"'s head.