Classification and organization of consumer digital images using workflow, and face detection and recognition

• US 8,897,504 B2
• Filed: 07/31/2012
• Issued: 11/25/2014
• Est. Priority Date: 01/22/2004
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Corcoran, Peter, Prilutsky, Yury, Bigioi, Petronel, Ciuc, Mihai, Ciurel, Stefanita, Vertan, Constantin
• Primary Examiner(s): STREGE, JOHN B
• Application Number:
  US13/563,730
• Publication Number:
  US 20130050460A1
• Time in Patent Office: 847 Days
• Field of Search: 382/118
• US Class Current: 382/118
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 18/232 Non-hierarchical techniques
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06V 10/763 Non-hierarchical techniques, e.g. based on statistics of modelling distributions
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks

Abstract

A processor-based system operating according to digitally-embedded programming instructions performs a method including identifying a group of pixels corresponding to a face region within digital image data acquired by an image acquisition device. A set of face analysis parameter values is extracted from said face region, including a faceprint associated with the face region. First and second reference faceprints are determined for a person using reference images captured respectively in predetermined face-portrait conditions and using ambient conditions. The faceprints are analyzed to determine a baseline faceprint and a range of variability from the baseline associated with the person. Results of the analyzing are stored and used in subsequent recognition of the person in a subsequent image acquired under ambient conditions.

Claims

1. One or more non-transitory processor-readable media having stored thereon a processor-readable database of faceprint image data corresponding to detected face regions within images acquired with an image acquisition device and digitally-stored, wherein the faceprint image data are stored within the media for access by a processor, the database comprising:

an image data component including

acquired digital image data including content data corresponding to acquired digital images therein; and

unique identifiers corresponding to the acquired digital images or face regions therein, or both;

an identity data component including an identification listing of known identities to which identified face regions detected within the acquired digital image data have been determined to correspond; and

a face recognition data component, comprising for an individual known identity;

one or more identity tables corresponding to one or more identities to which the individual known identity is linked,

wherein each identity table comprises one or more face class entries, wherein each face class entry is linked to and defined by a group of one or more faceprints;

wherein each faceprint corresponds to values of parameters produced by applying at least one transformation to an image that corresponds to the faceprint.

2. The one or more processor-readable media of claim 1, wherein the image data component further comprises an image list of acquired digital image data.

3. The one or more processor-readable media of claim 1, wherein the known identities correspond to handles identifying a known person.

4. The one or more processor-readable media of claim 1, wherein the identity data component further comprises one or more database links to face recognition data of the face recognition data component.

5. The one or more processor-readable media of claim 1, wherein the identity data component further comprises one or more database links to personal data associated with one or more known identities.

6. The one or more processor-readable media of claim 1, wherein the identity data component comprises a table of personal data associated with at least one of the known identities.

7. The one or more processor-readable media of claim 1, wherein the face recognition data component further comprises one or more face class tables corresponding to the one or more face class entries of the one or more identity tables, wherein each face class table comprises one or more faceprint image entries corresponding to faceprints from the acquired digital image data.

8. The one or more processor-readable media of claim 1, wherein the identified face regions comprise spatially normalized face regions, which are normalized with respect to size.

9. The one or more processor-readable media of claim 8, wherein the normalized face regions comprise spatially normalized face regions, which are normalized with respect to size, orientation or pose, or luminance normalized face regions, or combinations thereof, and

wherein said normalized face regions are normalized with respect to a standard size based on one or more distances between eyes, nose, mouth, or one or more other facial features, or combinations thereof, and

wherein the identity data component further comprises a set of links to a relationship list or a group membership list or both.

10. The one or more processor-readable media of claim 8, wherein the normalized face regions comprise spatially normalized face regions, which are normalized with respect to size, orientation or pose, luminance normalized face regions, or combinations thereof, and

wherein the image data component on the image acquisition device further comprises multiple tables of image classification events, occasions, locations, or places, or other categories, or combinations thereof, each to which groups of multiple images of the image data are determined to belong.

11. The one or more processor-readable media of claim 8, wherein the normalized face regions comprise spatially normalized face regions, which are normalized with respect to size, orientation or pose, luminance normalized face regions, or combinations thereof, and wherein said normalized face regions are normalized with respect to a standard size based on one or more distances between eyes, nose, mouth, or one or more other facial features, or combinations thereof, and

wherein the image data component further includes additional image data associated with circumstances of acquisition of a parent image and associated face region corresponding to a group of image data.

12. The one or more processor-readable media of claim 8, wherein the normalized face regions comprise spatially normalized face regions, which are normalized with respect to size, orientation or pose, luminance normalized face regions, or combinations thereof, and

wherein the one or more groups of image data further include image metadata including focusing distance of the lens at time of acquisition, or effective digital camera sensor size, or both.

18. The one or more processor-readable media of claim 1, wherein said transformation transforms an entire face region into a non-spatial domain.

19. The one or more processor-readable media of claim 1, wherein said parameters comprise basis vectors determined by transforming the image with a statistical analysis technique.

20. The one or more processor-readable media of claim 1, wherein said face class entry comprises a grouping of one or more faceprints determined by a clustering algorithm.

21. The one or more processor-readable media of claim 20, wherein a cluster radius determined by said clustering algorithm varies.

22. The one or more processor-readable media of claim 21, wherein a new faceprint associated with a new or known identity updates the cluster radius associated with at least one face class entry.

13. A portable digital image acquisition device, comprising:

a lens,

an image sensor,

a processor, and

one or more non-transitory processor-readable media having stored thereon;

a processor-readable a database of faceprint image data corresponding to detected face regions within images acquired with an image acquisition device and digitally-stored, wherein the faceprint image data are stored within the media for access by a processor, the database comprising;

an image data component including

acquired digital image data including content data corresponding to acquired digital images therein; and

unique identifiers corresponding to the acquired digital images or face regions therein, or both;

an identity data component including an identification listing of known identities to which identified face regions detected within the acquired digital image data have been determined to correspond; and

a face recognition data component, comprising for an individual known identity;

one or more identity tables corresponding to one or more identities to which the individual known identity is linked,

wherein each identity table comprises one or more face class entries, wherein each face class entry is linked to and defined by a group of one or more faceprints;

wherein each faceprint corresponds to values of parameters produced by applying at least one transformation to an image that corresponds to the faceprint.

14. The device of claim 13, wherein the face recognition data component further comprises one or more face class tables corresponding to the one or more face class entries of the one or more identity tables, wherein each face class table comprises one or more faceprint image entries corresponding to faceprints from the acquired digital image data.

15. The device of claim 14, wherein the image data component on the image acquisition device further comprises multiple tables of image classification events, occasions, locations, or places, or other categories, or combinations thereof, each to which groups of multiple images of the image data are determined to belong.

16. The device of claim 14, wherein the image data component further includes additional image data associated with circumstances of acquisition of a parent image and associated face region corresponding to a group of image data.

17. The device of claim 14, wherein the one or more groups of image data further include image metadata including focusing distance of the lens at time of acquisition, or effective digital camera sensor size, or both.