Method, system and computer program for identification and sharing of digital images with face signatures

• US 9,928,407 B2
• Filed: 08/21/2015
• Issued: 03/27/2018
• Est. Priority Date: 12/31/2007
• Status: Active Grant

• Current Assignee: 11967134 Canada Limited
• Original Assignee: Applied Recognition Inc.
• Inventors: Ganong, Ray, Waugh, Donald Craig, Studholme, Chris, Plataniotis, Kostas, Ro, Yong Man
• Primary Examiner(s): Thirugnanam, Gandhi
• Application Number:
  US14/832,433
• Publication Number:
  US 20160086019A1
• Time in Patent Office: 949 Days
• CPC Class Codes: G06F 15/16 Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs coordinating program control therefor G06F9/52; in regulating and control system G05B
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: G06V 10/32 Normalisation of the pattern dimensions
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

Methods and systems are provided for sharing a digital image depicting one or more faces. The method may include linking a plurality of computer terminals to a computer network, each computer terminal associated with an individual; receiving a digital image at at least one of the computer terminals; executing a face recognition routine on the digital image, the face recognition routine detecting at least one face in the digital image, each detected face corresponding to a person, the face recognition routine recognizing at least one of the persons as being one of the individuals; and for each individual recognized in the digital image by the face recognition routine, initiating dissemination of the digital image to the computer terminal associated with respective individual whose face is recognized in the digital image.

Claims

1. A method for recognizing one or more faces in a digital image, the method comprising:

receiving a digital image at at least one computer device;

executing, at the at least one computer device, a face detection routine on the digital image involving;

generating one or more face coordinates corresponding to one or more candidate regions for one or more faces;

generating eye coordinates based on the face coordinates;

executing, at the at least one computer device, a face recognition routine involving, for each of the one or more candidate regions for the one or more faces, generating a face signature using one or more projection images defined by the face coordinates and the eye coordinates; and

comparing the face signature with one or more known face signatures to determine a distance value for each comparison, computing an aggregation of the distance values, determining a best match between the face signature and the known face signatures using the aggregated distance values and comparing the best match to a similarity threshold, the best match determining an identity corresponding to at least one of the one or more faces; and

making available results of the face recognition routine wherein the face signature is generated by;

translating, rotating, and scaling the candidate region to a normalized image having a predetermined size wherein the eye coordinates are linked to predetermined locations;

masking the normalized image to define a masked image wherein the face is isolated;

applying a histogram equalization to a greyscale depiction of the masked image; and

generating the face signature using principal component analysis of the one or more projection images.

2. The method claimed in claim 1 wherein the face coordinates are generated by:

initiating a first texture-based detection routine to detect one or more candidate faces, each candidate face having face coordinates;

applying a skin color detection test to the one or more candidate faces to define a subset of the one or more candidate faces;

reducing the size defined by the face coordinates of the subset to a predefined size to define face objects;

initiating a second texture-based detection routine to the face objects to define a set of true positive face objects and a set of uncertain face objects; and

rotating the uncertain face objects to define further true positive face objects.

3. The method claimed in claim 2 further comprising, if the digital image is a color image, detecting in the digital image, the ratio of skin color to non-skin color;

and if the ratio exceeds a threshold, determining that the digital image does not consist of a face.

4. The method claimed in claim 2 further characterized by rotating each image bounded by the eye coordinates to correspond to the rotation of the uncertain face objects.

5. The method claimed in claim 1 further characterized by:

cropping an eye image corresponding to the portion of the digital image bounded by each eye coordinates;

optionally resizing the eye image to a predetermined size;

reducing reflected light in the eye image; and

isolating a plurality of pupils in the eye image corresponding to dark locations of the eye image.

13. The method of claim 1, further comprising:

registering a user of the at least one computing device; and

downloading and installing a computer product on the at least one computing device to configure the computing device to execute the face detection routine and the face recognition routine.

14. The method of claim 1, further comprising:

providing an interface on the at least one computer device for collecting registration data for a user of the least one computer device to register the user prior to executing the face detection routine and the face recognition routine.

15. The method of claim 1, wherein executing the face detection routine and the face recognition routine involves transmitting the digital image to a remote server and receiving the results of the face recognition routine from the remote server.

23. The method of claim 1, further comprising identifying an output option and making available the results of the face recognition routine according to the output option.

6. A method for sharing a digital image depicting one or more faces, the method comprising:

receiving a digital image at at least computer device;

enabling the at least one computer device to initiate a face detection routine and a face recognition routine on the digital image, the face detection routine involving generating one or more face coordinates corresponding to one or more candidate regions for one or more faces;

generating eye coordinates based on the face coordinates, the face recognition routine producing a list of one or more persons whose faces are depicted in the digital image, by for each of the one or more candidate regions, generating a face signature using one or more projection images defined by the face coordinates and the eye coordinates, comparing the face signature with one or more known face signatures to determine a distance value for each comparison, determining a best match between the face signature and the known face signatures and comparing the best match to a similarity threshold, the best match determining an identity of a person in the list of one or more persons corresponding to at least one of the one or more faces; and

enabling the at least one computer device to initiate a sharing routine for disseminating the digital image to computer terminals associated with the one or more persons, wherein the face signature is generated by;

translating, rotating, and scaling the candidate region to a normalized image having a predetermined size wherein the eye coordinates are linked to predetermined locations;

masking the normalized image to define a masked image wherein the face is isolated;

applying a histogram equalization to a greyscale depiction of the masked image; and

generating the face signature using principal component analysis of the one or more projection images.

7. The method claimed in claim 6, wherein the at least one computer device is linked to a processing engine, wherein the processing engine:

scans one or more local or remote file folders such that the processing engine determines when the digital image is linked to one of the file folders;

initiates the face detection routine and the face recognition routine on the digital image; and

links a result of the face recognition routine to a database.

8. The method claimed in claim 7, wherein another or the same computer terminal is also linked to a user interface enabling an individual to link the digital image to one of the file folders.

9. The method claimed in claim 8, wherein the user interface is accessible from a mobile device.

10. The method claimed in claim 7, wherein the user interface enables a user to create one or more albums linking a plurality of digital images.

11. The method claimed in claim 6, further comprising:

the individuals being linked to one or more peer groups; and

the digital image being disseminated only to persons linked to one of the peer groups also linked to the individual that is associated with the computer terminal linked to the digital image.

16. The method of claim 6, wherein the dissemination comprises using a social network service or a photo storage site to share the digital image with the one or more computer terminals associated with the one or more persons.

20. The method of claim 6, further comprising the at least one computer device receiving a list of known persons and an electronic address for each person in the list of known persons, the face recognition routine recognizing comprising identifying the one or more persons whose faces are depicted in the digital image as one of the known persons in the list of known persons and using the electronic address for the known person of the list of known persons for the sharing routine.

21. The method of claim 6, further comprising defining a peer group, recognizing a face of a member of the peer group within the digital image, queuing the digital image to be transmitted in a secure fashion to the member of the peer group.

22. The method of claim 6, further comprising loading a list of known persons from a social network or photo storage site for disseminating the digital image to a person of the list of known persons when the person is in the list of the one or more persons whose faces are depicted in the digital image.

12. A computer device for recognizing one or more faces in a digital image, the device comprising:

a storage device receiving and storing a digital image;

at least one processor configured to execute a face detection routine and a face recognition routine, the face detection routine involving generating one or more face coordinates corresponding to one or more candidate regions for one or more faces;

generating eye coordinates based on the face coordinates, the face recognition routine involving, for each of the one or more candidate regions, generating a face signature using one or more projection images defined by the face coordinates and the eye coordinates, comparing the face signature with one or more known face signatures to determine a distance value for each comparison, determining a best match between the face signature and the known face signatures and comparing the best match to a similarity threshold, the best match determining an identity corresponding to at least one of the one or more faces, and making available results of the face recognition routine, wherein the face signature is generated by;

translating, rotating, and scaling the candidate region to a normalized image having a predetermined size wherein the eye coordinates are linked to predetermined locations;

masking the normalized image to define a masked image wherein the face is isolated;

applying a histogram equalization to a greyscale depiction of the masked image; and

generating the face signature using principal component analysis of the one or more projection images.

17. The device of claim 12 further comprising a registration interface for registering a user of the at least one computing device and initiating downloading and installing of a computer product on the device to configure the computer device to execute the face detection routine and the face recognition routine.

18. The device of claim 12 further configured to execute the face detection routine and the face recognition routine by transmitting the digital image to a remote server and receiving the results of the face recognition routine from the remote server.

19. The device of claim 12 wherein the device is a mobile device.