Low threshold face recognition

• US 8,326,001 B2
• Filed: 06/29/2010
• Issued: 12/04/2012
• Est. Priority Date: 06/29/2010
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Free, Robert Mikio
• Primary Examiner(s): Ahmed, Samir
• Assistant Examiner(s): LE, TOTAM HA
• Application Number:
  US12/826,581
• Publication Number:
  US 20110317872A1
• Time in Patent Office: 889 Days
• Field of Search: 382/103, 382/118, 382/199
• US Class Current: 382/118
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

Methods, systems , and apparatus, including computer programs encoded on a computer storage medium, are disclosed for reducing the impact of lighting conditions and biometric distortions, while providing a low-computation solution for reasonably effective (low threshold) face recognition. In one aspect, the methods include processing a captured image of a face of a user seeking to access a resource by conforming a subset of the captured face image to a reference model. The reference model corresponds to a high information portion of human faces. The methods further include comparing the processed captured image to at least one target profile corresponding to a user associated with the resource, and selectively recognizing the user seeking access to the resource based on a result of said comparing.

Claims

1. A method performed by an image processor, the method comprising:

processing a captured image of a face of a user seeking to access a resource by conforming a subset of the captured face image to a reference model, the reference model corresponding to a high information portion of human faces, and the high information portion including eyes and a mouth of a face depicted in a reference image, where the processing of the captured image comprises

detecting a face within the captured image by identifying the eyes in an upper one third of the captured image and the mouth in a lower third of the captured image, and

matching the eyes of the detected face with the eyes of the face depicted in the reference image to obtain a normalized image of the detected face;

comparing the processed image to at least one target profile corresponding to a user associated with the resource, wherein the comparing of the processed image comprises

obtaining a difference image of the detected face by subtracting the normalized image of the detected face from a normalized image of a target face associated with a target profile, and

calculating scores of respective pixels of the difference image based on a weight defined according to proximity of the respective pixels to high information portions of the human faces, wherein the weight decreases from a maximum weight value at a mouth-level to a minimum weight value at an eyes-line; and

selectively recognizing the user seeking access to the resource based on a result of the comparing.

2. The method of claim 1, wherein the high information portion further includes a tip of a nose.

3. The method of claim 1, wherein the processing of the captured image further comprises vertically scaling a distance between an eyes-line and the mouth of the detected face to equal a corresponding distance for the face depicted in the reference image in order to obtain the normalized image of the detected face.

4. The method of claim 3, wherein the processing of the captured image further comprises matching the mouth of the detected face to the mouth of the face depicted in the reference image in order to obtain the normalized image of the detected face.

5. The method of claim 1, wherein the weight decreases continuously from the maximum weight valued to the minimum weight value.

6. The method of claim 1, wherein the weight decreases discretely from the maximum weight valued to the minimum weight value.

7. The method of claim 1, wherein the selectively recognizing the user comprises presenting to the user a predetermined indication according to a user'"'"'s profile.

8. The method of claim 1, wherein the resource represents an appliance, the method further comprising capturing the image using an image capture device of the appliance.

9. The method of claim 8, wherein the selectively recognizing the user comprises turning on a display of the appliance, wherein the display had been off prior to the comparing.

10. The method of claim 8, wherein the said processing of the captured image comprises:

applying an orange-distance filter to the captured image; and

segmenting a skin-tone orange portion of the orange-distance filtered image to represent a likely presence of a face in front of the image capture device.

11. The method of claim 10, wherein the processing of the captured image further comprises determining changes in area and in location of the skin-tone orange portion of the captured image relative to a previously captured image to represent likely movement of the face in front of the image capture device.

12. The method of claim 11, wherein the processing of the captured image further comprises detecting a face within the skin-tone orange portion of the orange-distance filtered image when the determined changes are less than predetermined respective variations.

13. An appliance comprising:

a data storage device configured to store profiles of users associated with the appliance;

an image capture device configured to acquire color frames;

one or more data processors configured to perform operations including;

applying an orange-distance filter to a frame acquired by the image capture device;

determining respective changes in area and location of a skin-tone orange portion of the acquired frame relative to a previously acquired frame;

inferring, based on the determined changes, a presence of a face substantially at rest when the frame was acquired;

detecting a face corresponding to the skin-tone orange portion of the acquired frame in response to the inference, the detecting including finding eyes and a mouth within the skin-tone orange portion;

normalizing the detected face based on locations of eyes and a mouth of a face in a reference image;

analyzing weighted differences between normalized target faces and the normalized detected face, such that weight values assigned to differences between portions of the normalized target faces and corresponding portions of the normalized detected face decrease from a maximum weight value at a mouth-level to a minimum weight value at an eyes-line wherein the target faces are associated with respective users of the appliance;

matching the face detected in the acquired frame with one of the target faces based on a result of the analyzing; and

acknowledging the match of the detected face in accordance with a profile stored on the data storage device and associated with a user of the appliance having the matched face.

14. The appliance of claim 13, wherein the data storage device is further configured to store:

rules for analyzing the weighted differences including weighting rules and scoring rules; and

rules for matching the detected face against target faces.

15. A system comprising:

one or more processors;

a storage system storing

a reference model corresponding to a high information portion of human faces, where the high information portion includes eyes and a mouth of a face depicted in a reference image, and

at least one target profile corresponding to a user associated with the system; and

a non-transitory computer readable medium encoding instructions that when executed by the one or more processors cause the system to execute operations comprising;

processing a captured image of a face of a user seeking to access the system by conforming a subset of the captured face image to the reference model, where the processing of the captured image comprises;

detecting a face within the captured image by identifying the eyes in an upper one third of the captured image and the mouth in a lower third of the captured image, and

matching the eyes of the detected face with the eyes of the face depicted in the reference image to obtain a normalized image of the detected face,

comparing the processed image to the at least one target profile stored on the storage system, wherein the comparing of the processed image comprises;

obtaining a difference image of the detected face by subtracting the normalized image of the detected face from a normalized image of a target face associated with the at least one target profile, and

calculating scores of respective pixels of the difference image based on a weight defined according to proximity of the respective pixels to high information portions of the human faces, wherein the weight decreases from a maximum weight value at a mouth-level to a minimum weight value at an eyes-line, and

selectively recognizing the user seeking access to the system based on a result of the comparing.

16. The system of claim 15, wherein the high information portion further includes a tip of a nose.

17. The system of claim 15, wherein the operation of processing the captured image further comprises vertically scaling a distance between an eyes-line and the mouth of the detected face to equal a corresponding distance for the face depicted in the reference image in order to obtain the normalized image of the detected face.

18. The system of claim 17, wherein the operation of processing the captured image further comprises matching the mouth of the detected face to the mouth of the face depicted in the reference image in order to obtain the normalized image of the detected face.

19. The system of claim 15, wherein the weight decreases continuously from the maximum weight valued to the minimum weight value.

20. The system of claim 15, wherein the weight decreases discretely from the maximum weight valued to the minimum weight value.

21. The system of claim 15, wherein the operation of selectively recognizing the user comprises presenting to the user a predetermined indication according to a user'"'"'s profile.

22. The system of claim 15, further comprising:

an image capture device,

wherein the image is captured using the image capture device.

23. The system of claim 22, further comprising:

a display,

wherein the operation of selectively recognizing the user comprises turning on the display, wherein the display had been off prior to the comparing.

24. The system of claim 22, wherein the operation of processing the captured image further comprises:

applying an orange-distance filter to the captured image, and

segmenting a skin-tone orange portion of the orange-distance filtered image to represent a likely presence of a face in front of the image capture device.

25. The system of claim 24, wherein the operation of processing the captured image further comprises determining changes in area and in location of the skin-tone orange portion of the captured image relative to a previously captured image to represent likely movement of the face in front of the image capture device.

26. The system of claim 25, wherein the operation processing the captured image further comprises detecting a face within the skin-tone orange portion of the orange-distance filtered image when the determined changes are less than predetermined respective variations.