ENHANCED FACE RECOGNITION IN VIDEO

• US 20120314914A1
• Filed: 06/10/2011
• Published: 12/13/2012
• Est. Priority Date: 06/10/2011
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Karakotsios, Kenneth M., Huang, Mingjing, Fu, Kah Kuen, Ivanchenko, Volodymyr V.
• Granted Patent: US 8,705,812 B2
• Time in Patent Office: Days
• US Class Current: 382/118
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/166 using acquisition arrangements
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction

Abstract

The computational resources needed to perform processes such as image recognition can be reduced by determining appropriate frames of image information to use for the processing. In some embodiments, infrared imaging can be used to determine when a person is looking substantially towards a device, such that an image frame captured at that time will likely be adequate for facial recognition. In other embodiments, sound triangulation or motion sensing can be used to assist in determining which captured image frames to discard and which to select for processing based on any of a number of factors indicative of a proper frame for processing.

Claims

1. A computer-implemented method of selecting a video frame for facial recognition, comprising:

under control of one or more computing systems configured with executable instructions,

initiating a capture of video information on an electronic device;

detecting at least one feature in the video information indicative of a person'"'"'s head;

in response to detecting the at least one feature, activating at least one infrared (IR) imaging mechanism of the electronic device operable to illuminate the person with at least one wavelength of IR light and detect IR light reflected back from the person;

detecting features corresponding to the person'"'"'s pupils in the reflected IR light;

analyzing the detected features to determine at least one of an orientation, a measurement, or a level of focus of the detected features in the reflected IR light;

selecting a current frame of video information when at least one of the orientation, measurement, or level of focus of the detected features meets a specified selection criterion; and

performing facial recognition on the selected frame.

2. The computer-implemented method of claim 1, wherein selecting a current frame of video information is further based on a determination that detected IR reflections are indicative of the detected features corresponding to human eyes.

3. The computer-implemented method of claim 1, wherein the at least one IR imaging mechanism is operable to emit pulses of IR light at a specified frequency of emission.

4. The computer-implemented method of claim 1, further comprising:

detecting motion near the electronic device before initiating a capture of video information.

5. A computer-implemented method of selecting an image frame for processing, comprising:

under control of one or more computing systems configured with executable instructions,

obtaining a series of image frames captured using at least one image capture element;

executing at least one orientation determining process around the time of obtaining the series of image frames to determine a relative orientation of a person in at least a portion of the series of image frames;

selecting one of the image frames from the series in response to determining, via the at least one orientation determining process, that the person is substantially facing toward the at least one image capture element at a time substantially corresponding to a time of capture of the selected image frame; and

providing the selected frame for processing.

6. The computer-implemented method of claim 5, wherein the at least one orientation process includes:

emitting at least one wavelength of IR radiation in a capture direction of the at least one image capture element;

detecting reflected IR radiation; and

locating one or more features in the reflected IR radiation corresponding to human eyes.

7. The computer-implemented method of claim 6, wherein the at least one orientation process includes:

capturing audio information using at least two audio capture elements; and

determining at least a direction of a source of the audio information,

wherein selecting one of the image frames is further based at least in part upon the determined direction.

8. The computer-implemented method of claim 6, wherein the at least one orientation process includes:

determining at least one of a motion or pressure with respect to a portion of an electronic device including the at least one capture element, the motion or pressure corresponding to an input from a person to be identified to the electronic device.

9. The computer-implemented method of claim 5, wherein selecting one of the image frames includes selecting an image frame corresponding to a time in which the at least one image capture element is determined to be substantially stationary with respect to a person being imaged.

10. The computer-implemented method of claim 5, wherein selecting one of the image frames includes selecting an image frame corresponding to a time in which a motion sensor detects less than a maximum threshold of movement of objects capable of being captured in the image frame with respect to the at least one image capture element.

11. The computer-implemented method of claim 5, wherein the at least one image capture element is configured to capture the series of images in response to at least one of a detected motion, a detected heat signature, a detected type of IR reflection, or a threshold amount of audio information.

12. The computer-implemented method of claim 5, further comprising:

performing at least one head detection process before executing the at least one orientation determining process.

13. The computer-implemented method of claim 12, wherein the at least one orientation determining process is initiated in response to locating at least one feature in the series of images matching a shape or contour of a human face.

14. The computer-implemented method of claim 5, wherein processing includes at least one of performing facial recognition or identity authentication on the selected frame.

15. The computer-implemented method of claim 5, wherein selecting one of the image frames from the series further depends at least in part upon a determined amount of blur detected in the at least one orientation determining process.

16. The computer-implemented method of claim 5, wherein the at least one orientation determining process includes at least one of three-dimensional, stereoscopic, or multi-camera imaging.

17. The computer-implemented method of claim 5, further comprising:

providing secure access to determined functionality in response to authenticating an identity of a user captured in the selected frame.

18. A computer-implemented method of selecting a video frame for facial recognition, comprising:

under control of one or more computing systems configured with executable instructions,

detecting motion near an electronic device;

in response to detecting motion, activating at least one infrared (IR) imaging mechanism of the electronic device operable to illuminate a person within view of the IR imaging mechanism with at least one wavelength of IR light and detect IR light reflected back from the person;

detecting features corresponding to the person'"'"'s corneas in the reflected IR light;

in response to detecting the features, initiating a capture of video information on the electronic device;

selecting a current frame of video information when at least one of an orientation, a measurement, or level of focus of the detected features meets a specified selection criterion; and

performing facial recognition on the selected frame.

19. The computer-implemented method of claim 18, wherein selecting a current frame of video information is further based on a determination that the reflected IR is indicative of the detected features corresponding to actual human eyes and not images of human eyes.

20. The computer-implemented method of claim 18, wherein the at least one IR imaging mechanism includes a first sensor substantially adjacent to an infrared source and a second sensor positioned a distance away from the infrared source on the electronic device.

21. A computing device, comprising:

a processor;

at least one image capture element; and

a memory device including instructions that, when executed by the processor, cause the computing device to;

obtain a series of image frames captured using at least one image capture element;

execute at least one orientation determining process around the time of obtaining the series of image frames to determine a relative orientation of a person in at least a portion of the series of image frames;

select one of the image frames from the series in response to determining, via the at least one orientation determining process, that the person is substantially facing toward the computing device at a time substantially corresponding to a time of capture of the selected image frame; and

perform facial recognition on the selected frame.

22. The computing device of claim 21, further comprising:

at least one infrared (IR) emitter; and

at least one IR receiver,

wherein the at least one orientation process includes;

emitting at least one wavelength of IR radiation using at least one IR emitter;

detecting reflected IR radiation using at least one IR detector of the electronic device; and

locating one or more features in the reflected IR radiation corresponding to human eyes.

23. The computing device of claim 21, further comprising:

at least two audio capture elements,

wherein the at least one orientation process includes;

capturing audio information using at least two audio capture elements; and

determining at least a direction of a source of the audio information,

wherein selecting one of the image frames is further based at least in part upon the determined direction.

24. The computing device of claim 21, further comprising:

at least one of a touch-sensitive material or a pressure sensitive material on at least a portion of a casing of the computing device,

wherein the at least one orientation process includes determining at least one of a motion or pressure with respect to a portion of an electronic device including the at least one capture element, the motion or pressure corresponding to an input from a person to be identified to the electronic device.

25. The computing device of claim 21, further comprising:

at least one motion-determining element,

wherein the selecting one of the image frames from the series is further based upon the computing device being substantially stationary as determined by the at least one motion-determining element.

26. A non-transitory computer-readable storage medium storing instructions for selecting an image frame for processing, the instructions when executed by a processor causing the processor to:

obtain a series of image frames captured using at least one image capture element;

execute at least one orientation determining process around the time of obtaining the series of image frames to determine a relative orientation of a person in at least a portion of the series of image frames;

select one of the image frames from the series in response to determining, via the at least one orientation determining process, that the person is substantially facing toward the computing device at a time substantially corresponding to a time of capture of the selected image frame; and

perform facial recognition on the selected frame.

27. The non-transitory computer-readable storage medium of claim 26, wherein the instructions when executed further cause the processor to:

emit at least one wavelength of IR radiation in a capture direction of the at least one image capture element;

detect reflected IR radiation; and

locate one or more features in the reflected IR radiation corresponding to human eyes.