Classification and organization of consumer digital images using workflow, and face detection and recognition

• US 10,346,677 B2
• Filed: 09/29/2017
• Issued: 07/09/2019
• Est. Priority Date: 01/22/2004
• Status: Expired due to Term

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Corcoran, Peter, Prilutsky, Yury, Bigioi, Petronel, Ciuc, Mihai, Ciurel, Stefanita, Vertan, Constantin
• Primary Examiner(s): Strege, John B
• Application Number:
  US15/721,387
• Publication Number:
  US 20180025218A1
• Time in Patent Office: 648 Days
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06V 10/763 Non-hierarchical techniques, e.g. based on statistics of modelling distributions
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks

Abstract

Techniques for identifying a person in a target image are described. According to one of the techniques, identifying a person in a target image involves displaying, within a graphical user interface, an image that depicts one or more faces. One or more faces are automatically detected within the image. A user provides input that selects a face of the one or more faces to be a currently-selected face. A set of images are selected from a collection of images, where the set of images includes images that closely match the currently-selected face. Concurrently with display of the currently-selected face, each image in the set of images is displayed. Within the graphical user interface, a control is provided. The control enables a user to select a target image from the set of images. In response to detecting that the user has selected a target image using the control, the currently-selected face is associated with a person to which the target image corresponds.

Claims

1. A processor-based image acquisition and processing system, comprising:

an image acquisition component including a lens and image sensor for acquiring a digital image, and

a processing component including a processor and a memory having processor-readable code embedded therein for programming the processor to perform a method of classifying and archiving images including face regions that are acquired with the image acquisition component, the method comprising;

displaying, within a graphical user interface, an image that depicts one or more faces;

detecting one or more faces within the image;

receiving user input that selects a face of the one or more faces to be a currently-selected face;

selecting a set of images, from a collection of images, that include face regions which closely match the currently-selected face, by first normalizing the face regions in the set of images and then extracting face classifier parameters from the face regions in the set of images;

concurrently with display of the currently-selected face, displaying each image in the set of images;

selecting a target image from the set of images by first determining a subset of N most probable matches, based on whether a majority of members of the set have like identity, and then selecting the target image based on whether the geometric distance between the faceprint of the currently-selected face and the faceprint of the target image lie within a certain geometric distance of each other in facespace; and

associating the currently-selected face with a person to which the target image corresponds, wherein the method is performed by one or more computing devices.

2. The method of claim 1, wherein the detecting step includes displaying a visual indication of a boundary around the one or more faces.

3. The method of claim 2, further comprising concurrently displaying an originally-acquired image with the image that depicts one or more faces with a visual indication of a boundary around the one or more faces.

4. The method of claim 1, wherein an image in the set of images comprises a normalized face region that is normalized by:

based on, at least in part, one or more distances between eyes, a nose, and a mouth depicted in a face region of the image,

determining a profile type of a face depicted in the face region; and

based on, at least in part, the profile type, determining a transformation, and applying the transformation to the face region to generate the normalized face region.

5. The method of claim 1, wherein displaying each image in the set of images includes displaying an indication of degree of match between each image and the currently-selected face.

6. The method of claim 1, wherein displaying each image in the set of images includes displaying each image in an order that reflects a degree of match between each image and the currently-selected face.

7. The method of claim 1 wherein each image of the set of images comprises a face image, created from an original digitally acquired image including a face by performing the steps of:

identifying a group of pixels that correspond to a face within the original digitally acquired image;

selecting a portion of the original image to include the group of pixels;

automatically generating a new image with values of pixels based on the selected portion.

8. The method of claim 1, further comprising generating an option to create a new face for identifying a new person based on the currently-selected face.

9. The method of claim 8, further comprising in response to receiving input that selects the option to create the new face, allowing a user to enter a handle to identify the new person.

10. The method of claim 1, wherein the selecting step includes selecting set of images of known persons within a database.

11. One or more non-transitory computer-readable media storing instructions which, when executed by one or more hardware processors, cause performance of a method comprising:

displaying, within a graphical user interface, an image that depicts one or more faces;

detecting one or more faces within the image;

receiving user input that selects a face of the one or more faces to be a currently-selected face;

selecting a set of images, from a collection of images, that include face regions which closely match the currently-selected face, by first normalizing the face regions in the set of images and then extracting face classifier parameters from the face regions in the set of images;

concurrently with display of the currently-selected face, displaying each image in the set of images;

selecting a target image from the set of images by first determining a subset of N most probable matches, based on whether a majority of members of the set have like identity, and then selecting the target image based on whether the geometric distance between the faceprint of the currently-selected face and the faceprint of the target image lie within a certain geometric distance of each other in facespace; and

associating the currently-selected face with a person to which the target image corresponds.

12. The one or more non-transitory computer-readable media of claim 11, wherein the detecting step includes displaying a visual indication of a boundary around the one or more faces.

13. The one or more non-transitory computer-readable media of claim 12, further comprising concurrently displaying an originally-acquired image with the image that depicts one or more faces with a visual indication of a boundary around the one or more faces.

14. The one or more non-transitory computer-readable media of claim 11, wherein an image in the set of images comprises a normalized face region that is normalized by:

based on, at least in part, one or more distances between eyes, a nose, and a mouth depicted in a face region of the image,

determining a profile type of a face depicted in the face region; and

based on, at least in part, the profile type, determining a transformation, and applying the transformation to the face region to generate the normalized face region.

15. The one or more non-transitory computer-readable media of claim 11, wherein displaying each image in the set of images includes displaying an indication of degree of match between each image and the currently-selected face.

16. The one or more non-transitory computer-readable media of claim 11, wherein displaying each image in the set of images includes displaying each image in an order that reflects a degree of match between each image and the currently-selected face.

17. The one or more non-transitory computer-readable media of claim 11, wherein each image of the set of images comprises a face image, created from an original digitally acquired image including a face by performing the steps of:

identifying a group of pixels that correspond to a face within the original digitally acquired image;

selecting a portion of the original image to include the group of pixels;

automatically generating a new image with values of pixels based on the selected portion.

18. The one or more non-transitory computer-readable media of claim 11, further comprising generating an option to create a new face for identifying a new person based on the currently-selected face.

19. The one or more non-transitory computer-readable media of claim 18, further comprising in response to receiving input that selects the option to create the new face, allowing a user to enter a handle to identify the new person.

20. The one or more non-transitory computer-readable media of claim 11, wherein the selecting step includes selecting set of images of known persons within a database.