Method, system, and computer program for identification and sharing of digital images with face signatures

• US 8,750,574 B2
• Filed: 12/30/2008
• Issued: 06/10/2014
• Est. Priority Date: 12/31/2007
• Status: Active Grant

• Current Assignee: 11967134 Canada Limited
• Original Assignee: Applied Recognition Inc.
• Inventors: Ganong, Ray, Waugh, Donald, Man Ro, Yong, Plataniotis, Konstantinos, Studholme, Chris
• Primary Examiner(s): Thirugnanam, Gandhi
• Application Number:
  US12/811,240
• Publication Number:
  US 20100287053A1
• Time in Patent Office: 1,988 Days
• Field of Search: 382/115, 382/117
• US Class Current: 382/115
• CPC Class Codes: G06F 15/16 Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs coordinating program control therefor G06F9/52; in regulating and control system G05B
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: G06V 10/32 Normalisation of the pattern dimensions
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

The present invention solves the problem of automatically recognizing multiple known faces in photos or videos on a local computer storage device (on a home computer). It further allows for sophisticated organization and presentation of the photos or videos based on the graphical selection of known faces (by selecting thumbnail images of people). It also solves the problem of sharing or distributing photos or videos in an automated fashion between ‘friends’ who are also using the same software that enables the invention. It further solves the problem of allowing a user of the invention to review the results of the automatic face detection, eye detection, and face recognition methods and to correct any errors resulting from the automated process.

Claims

1. A method for recognizing one or more faces in a digital image, the method comprising:

a. generating one or more face coordinates corresponding to one or more candidate regions for one or more candidate faces, the face coordinates generating comprising;

initiating a first texture-based detection routine on the digital image to detect one or more candidate face regions, each candidate face region defined by respective face coordinates;

applying a skin color detection test to the one or more candidate face regions to eliminate one or more of the candidate faces that fail the skin color detection test, if any;

reducing the size defined by the face coordinates of the of the one or more candidate face regions to a predefined size;

initiating a second texture-based detection routine on each size-reduced candidate face region to define a set of positively identified face objects and uncertain face objects; and

defining further positively identified face objects, if any, by rotating the digital images associated with the uncertain face objects;

b. generating eye coordinates based on the face coordinates of each positively identified face object;

c. detecting each face using one or more projection images defined by the face coordinates of each positively identified face object and the eye coordinates; and

d. comparing each projection image with one or more known projection images, wherein a similarity threshold is provided for defining a best match between the projection image and the known projection images.

2. The method claimed in claim 1 further characterized by, if the digital image is a color image:

a. detecting in the digital image, the ratio of skin color to non-skin color; and

b. if the ratio exceeds a threshold, determining that the digital image does not consist of a face.

3. The method claimed in claim 1 further characterized by rotating each image bounded by the eye coordinates to correspond to the rotation of the uncertain face objects.

4. The method claimed in claim 1 further characterized by:

a. cropping an eye image corresponding to the portion of the digital image bounded by each eye coordinates;

b. optionally resizing the eye image to a predetermined size;

c. reducing reflected light in the eye image; and

d. isolating a plurality of pupils in the eye image corresponding to dark locations of the eye image.

5. The method claimed in claim 1 further characterized in that the projection image is generated by:

a. translating, rotating, and scaling the candidate region to a normalized image having a predetermined size wherein the eye coordinates are linked to predetermined locations;

b. masking the normalized image to define a masked image wherein the face is isolated;

c. applying a histogram equalization to a greyscale depiction of the masked image; and

d. generating a principal component analysis (“

PCA”

) vector of the equalized image.

6. The method of claim 1 wherein the rotating comprises:

performing a first rotation on each uncertain face object;

re-applying the face generating on the respective uncertain face object rotated by the first rotation;

performing a second rotation on each uncertain face object;

re-applying the face generating on the respective uncertain face object rotated by the second rotation; and

defining the positively identified face objects where a face is detected in the respective uncertain face object by both re-applications of the face generating.

7. The method of claim 6 wherein the first rotation comprises a first rotation angle, and the second rotation comprises a second rotation angle incremented with respect to the first rotation angle, the method comprising repeating the first rotation performing, second rotation performing, both face generating re-applying, and further positively identified face objects defining over a predetermined range of first rotation angles and second rotation angles.

8. The method of claim 1 comprising flattening each positively identified face object and applying a two-dimensional linear regression on pixel intensity in the respective face region for ensuring spatial uniformity of pixel intensity.

9. The method of claim 1 comprising applying a histogram image equalization in the greyscale domain to each positively identified face object.

10. The method of claim 1 comprising creating a principal component analysis (“

PCA”

) vector of each positively identified face object;

the projection image comparing comprising comparing the PCA vector of each positively identified face object with one or more PCA vectors associated with one or more known projection images, wherein the similarity threshold defines a best match between the PCA vector of the respective positively identified face object and the PCA vectors associated with the known projection images.