Grouping and ranking images based on facial recognition data

• US 9,773,156 B2
• Filed: 04/29/2014
• Issued: 09/26/2017
• Est. Priority Date: 04/29/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Chan, Chunkit Jacky, Lee, David, Light, Allison
• Primary Examiner(s): Hasan, Syed
• Application Number:
  US14/264,619
• Publication Number:
  US 20150310040A1
• Time in Patent Office: 1,246 Days
• Field of Search: None
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

Technologies for grouping images, and ranking the images and the groupings, based on entities shown in the images. Images may be grouped based on faces shown in the images. Different images with faces that indicate the same entity (e.g., Adam) may be automatically grouped together. Different images with faces that indicate the same multiple entities (e.g., the people in my family) may also be automatically grouped together. Such automatic grouping may be based on facial recognition technologies. Further, images and groups of images may be automatically ranked based on the faces shown and entities represented. Such rankings may also be influenced by adjacent data that indicates family and friends and the like, and that can be used to identify such entities in the images.

Claims

1. A method performed on a computing device, the method comprising:

determining, by the computing device for each image in a set of images, facial recognition data that, for each face detected by the computing device in the each image, comprises a face identifier that uniquely identifies the each face, a set of facial feature descriptors, a face score that is based on an open/closed state of the each face'"'"'s eyes and mouth and that indicates an overall quality of the each face, and a face signature that, across the images in the set, uniquely identifies an entity that the each face represents; and

grouping, by the computing device based at least on the face signatures and the face scores, images of the set into one or more groups, where each group comprises one or more images that each show a detected face that represents a same entity as that represented by any other detected face shown in any image in the each group.

2. The method of claim 1 where the grouping is further based on the face scores.

3. The method of claim 1 where a face with a face score below a particular threshold is not included in the grouping.

4. The method of claim 1 where at least one of the groups is a multi-entity group.

5. The method of claim 1 further comprising ranking, based on the face scores, the images in the set or the groups.

6. The method of claim 1 further comprising ranking the groups according to a number of entities of the groups.

7. The method of claim 1 further comprising ranking the groups according to entities of the groups matching adjacent data that indicates family and friends.

8. A system comprising a computing device and at least one program module that together are configured for performed actions comprising:

determining, by the computing device for each image in a set of images, facial recognition data that, for each face detected by the computing device in the each image, comprises a face identifier that uniquely identifies the each face, a set of facial feature descriptors, a face score that is based on an open/closed state of the each face'"'"'s eyes and mouth and that indicates an overall quality of the face, and a face signature that, across the images in the set, uniquely identifies an entity that the each face represents; and

grouping, by the computing device based at least on the face signatures and the face scores, images of the set into one or more groups, where each group comprises one or more images that each show a detected face that represents a same entity as that represented by any other detected face shown in any image in the each group.

9. The system of claim 8 where the grouping is further based on the face scores.

10. The system of claim 8 where a face with a face score below a particular threshold is not included in the grouping.

11. The system of claim 8 where at least one of the groups is a multi-entity group.

12. The system of claim 8, the actions further comprising ranking, based on the face scores, the images in the set or the groups.

13. The system of claim 8, the actions further comprising ranking the groups according to a number of entities of the groups.

14. The system of claim 8, the actions further comprising ranking the groups according to entities of the groups matching adjacent data that indicates family and friends.

15. At least one computer-readable media storing computer-executable instructions that, when executed by a computing device, cause the computing device to perform actions comprising:

determining, by the computing device for each image in a set of images, facial recognition data that, for each face detected by the computing device in the image, comprises a face identifier that uniquely identifies the each face, a set of facial feature descriptors, a face score that is based on an open/closed state of the each face'"'"'s eyes and mouth and that indicates an overall quality of the each face, and a face signature that, across the images in the set, uniquely identifies an entity that the each face represents; and

grouping, by the computing device based at least on the face signatures and the face scores, images of the set into one or more groups, where each group comprises one or more images that each show a detected face that represents a same entity as that represented by any other detected face shown in any image in the each group.

16. The at least one computer-readable media of claim 15 where the grouping is further based on the face scores, or where a face with a face score below a particular threshold is not included in the grouping.

17. The at least one computer-readable media of claim 15 where at least one of the groups is a multi-entity group.

18. The at least one computer-readable media of claim 15, the actions further comprising ranking, based on the face scores, the images in the set or the groups.

19. The at least one computer-readable media of claim 15, the actions further comprising ranking the groups according to a number of entities of the groups.

20. The at least one computer-readable media of claim 15, the actions further comprising ranking the groups according to entities of the groups matching adjacent data that indicates family and friends.