Classification system for consumer digital images using automatic workflow and face detection and recognition

• US 8,199,979 B2
• Filed: 07/20/2009
• Issued: 06/12/2012
• Est. Priority Date: 01/22/2004
• Status: Expired due to Term

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Corcoran, Peter, Prilutsky, Yury, Bigioi, Petronel, Ciuc, Mihai, Ciurel, Stefanita, Vertran, Constantin
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US12/506,124
• Publication Number:
  US 20100014721A1
• Time in Patent Office: 1,058 Days
• Field of Search: 382115-118, 340/5.1, 340/5.2, 340/5.52, 340/5.53, 340 58- 586, 902/3, 713/186
• US Class Current: 382/118
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

A processor-based system operating according to digitally-embedded programming instructions includes a face detection module for identifying face regions within digital images. A normalization module generates a normalized version of the face region. A face recognition module automatically extracts a set of face classifier parameter values from the normalized face region that are referred to as a faceprint. A workflow module automatically compares the extracted faceprint to a database of archived faceprints previously determined to correspond to known identities. The workflow module determines based on the comparing whether the new faceprint corresponds to any of the known identities, and associates the new faceprint and normalized face region with a new or known identity within a database. A database module serves to archive data corresponding to the new faceprint and its associated parent image according to the associating by the workflow module within one or more digital data storage media.

Claims

1. A processor-based device operating according to digitally-embedded programming instructions and communicating with one or more digital data storage media for classifying and archiving images including face regions that are acquired with an image acquisition device, the programming instructions comprising:

a face detection module for identifying a group of pixels corresponding to a face region that is rotated left or right or tilted up or down, or a combination thereof, within digital image data acquired by the image acquisition device;

a normalization module for generating a normalized version of the face region that is neither rotated left or right nor tilted up or down, nor a combination thereof;

a face recognition module for extracting a set of face classifier parameter values from said normalized version of the face region, said set of face classifier parameter values being collectively known as a faceprint associated with said normalized face region; and

a workflow module for comparing said extracted faceprint to one or more extracted faceprints recently acquired by the same acquisition device and associated with a same identity, and for determining based on the comparing whether a new faceprint corresponds closely enough to one or more other faceprints to be used as a reference faceprint for said identity; and

for storing, locally or in a remote database, or both, said reference faceprint in association with said identity, and

wherein said determining by the workflow module comprises determining proximities of the values of the face classifier parameters of the new face print image with values corresponding to one or more archived fingerprints wherein the determining by the workflow module comprises automatically determining that the new faceprint corresponds to a known identity based on one or more geometric distance proximities being within a predetermined proximity threshold, and wherein the predetermined proximity threshold comprises a first threshold, and the determining by the workflow module comprises confirming whether the normalized face region associated with the new faceprint corresponds to a known identity when a geometric distance proximity is outside the first threshold and within a second threshold greater than the first threshold.

2. The device of claim 1, wherein the generating a normalized version of the face region by the normalization module comprises:

mapping the face region to a 3D model,

rotating the 3D model to obtain a 3D frontal face region, and

mapping the 3D frontal face region to a normalized 2D face region.

3. The device of claim 2, wherein the workflow module is further configured to receive an indication from a user once a sufficient number of corresponding faceprints are determined to enable provision of a reference faceprint for a particular associated identity.

4. The device of claim 1, wherein the workflow module is further configured for merging, averaging or otherwise processing a group of two or more faceprints which correspond closely to each other in order to provide the reference faceprint for the associated identity.

5. The device of claim 4, wherein more than one reference faceprint is provided for a particular associated identity.

6. The device of claim 1, where the workflow module is further configured for determining a statistical range of values for one or more parameter values of said reference faceprint to provide a statistical reference faceprint.

7. A method of classifying and archiving images including face regions that are acquired with an image acquisition device, comprising:

identifying a group of pixels corresponding to a face region that is rotated left or right or tilted up or down, or a combination thereof, within digital image data acquired by the image acquisition device;

generating a normalized version of the face region that is neither rotated left or right nor tilted up or down, nor a combination thereof;

extracting a set of face classifier parameter values from said normalized version of the face region, said set of face classifier parameter values being collectively known as a faceprint associated with said normalized face region;

comparing said extracted faceprint to one or more extracted faceprints recently acquired by the same acquisition device and associated with a same identity,

determining based on the comparing whether a new faceprint corresponds closely enough to one or more other faceprints to be used as a reference faceprint for said identity;

determining proximities of the values of the face classifier parameters of the new face print image with values corresponding to the one or more archived faceprints;

determining that the new faceprint corresponds to a known identity based on one or more geometric distance proximities being within a predetermined proximity threshold that comprises a first threshold;

confirming whether the normalized face region associated with the new faceprint corresponds to a known identity when a geometric distance proximity is outside the first threshold and within a second threshold greater than the first threshold; and

storing, locally or in a remote database, or both, said reference faceprint in association with said identity.

8. The method of claim 7, wherein the generating a normalized version of the face region comprises:

mapping the face region to a 3D model,

rotating the 3D model to obtain a 3D frontal face region, and

mapping the 3D frontal face region to a normalized 2D face region.

9. The method of claim 8, further comprising receiving an indication from a user once a sufficient number of corresponding faceprints are determined to enable provision of a reference faceprint for a particular associated identity.

10. The method of claim 7, further comprising merging, averaging or otherwise processing a group of two or more faceprints which correspond closely to each other in order to provide the reference faceprint for the associated identity.

11. The method of claim 10, wherein more than one reference faceprint is provided for a particular associated identity.

12. The method of claim 7, further comprising determining a statistical range of values for one or more parameter values of said reference faceprint to provide a statistical reference faceprint.

13. One or more non-transitory computer readable media having code embedded therein for programming a processor to perform a method of classifying and archiving images including face regions that are acquired with an image acquisition device, wherein the method comprises:

identifying a group of pixels corresponding to a face region that is rotated left or right or tilted up or down, or a combination thereof, within digital image data acquired by the image acquisition device;

generating a normalized version of the face region that is neither rotated left or right nor tilted up or down, nor a combination thereof;

extracting a set of face classifier parameter values from said normalized version of the face region, said set of face classifier parameter values being collectively known as a faceprint associated with said normalized face region;

comparing said extracted faceprint to one or more extracted faceprints recently acquired by the same acquisition device and associated with a same identity,

determining based on the comparing whether a new faceprint corresponds closely enough to one or more other faceprints to be used as a reference faceprint for said identity;

determining proximities of the values of the face classifier parameters of the new face print image with values corresponding to the one or more archived faceprints;

determining that the new faceprint corresponds to a known identity based on one or more geometric distance proximities being within a predetermined proximity threshold that comprises a first threshold;

confirming whether the normalized face region associated with the new faceprint corresponds to a known identity when a geometric distance proximity is outside the first threshold and within a second threshold greater than the first threshold; and

storing, locally or in a remote database, or both, said reference faceprint in association with said identity.

14. The one or more non-transitory computer readable media of claim 13, wherein the generating a normalized version of the face region comprises:

mapping the face region to a 3D model,

rotating the 3D model to obtain a 3D frontal face region, and

mapping the 3D frontal face region to a normalized 2D face region.

15. The one or more non-transitory computer readable media of claim 14, wherein the method further comprises receiving an indication from a user once a sufficient number of corresponding faceprints are determined to enable provision of a reference faceprint for a particular associated identity.

16. The one or more non-transitory computer readable media of claim 13, wherein the method further comprises merging, averaging or otherwise processing a group of two or more faceprints which correspond closely to each other in order to provide the reference faceprint for the associated identity.

17. The one or more non-transitory computer readable media of claim 16, wherein more than one reference faceprint is provided for a particular associated identity.

18. The one or more non-transitory computer readable media of claim 13, wherein the method further comprises determining a statistical range of values for one or more parameter values of said reference faceprint to provide a statistical reference faceprint.