Method of person identification using social connections

• US 8,416,997 B2
• Filed: 01/27/2010
• Issued: 04/09/2013
• Est. Priority Date: 01/27/2010
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Winters, Dustin L., Spaulding, Kevin E., Gallagher, Andrew C.
• Primary Examiner(s): Johns, Andrew W
• Assistant Examiner(s): Dunphy, David F
• Application Number:
  US12/694,305
• Publication Number:
  US 20110182482A1
• Time in Patent Office: 1,168 Days
• Field of Search: None
• US Class Current: 382/118
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06V 30/242 Division of the character sequences into groups prior to recognition; Selection of dictionaries
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

A method for automatically identifying persons in digital media assets associated with a database of individual user accounts, comprising: providing access to a database of individual user accounts, wherein the database includes connections between the individual user accounts; receiving a digital media asset associated with a first individual user account; analyzing the received digital media asset to detect an unidentified person; designating collections of digital media assets associated with the first individual user account and other connected individual user accounts; training person identifiers for the previously identified persons by analyzing digital media assets containing the previously identified persons; using a processor to identify the detected unidentified person using the trained person identifiers; and associating metadata providing an indication of the one or more identified persons with the received digital media asset.

Claims

1. A method for automatically identifying persons in digital media assets associated with a database of individual user accounts, comprising:

providing access to the database of individual user accounts, wherein the database includes connections between the individual user accounts;

designating a collection of digital media assets, wherein each digital media asset is associated with one or more of the individual user accounts and wherein one or more of the digital media assets have been tagged with the identity of one or more previously identified persons;

receiving a digital media asset associated with a first individual user account;

analyzing the received digital media asset to detect an unidentified person contained in the received digital media asset;

extracting appearance features for the unidentified person;

using the database of individual user accounts to produce a list of candidate individuals corresponding to previously identified persons;

determining weights for each candidate individual in the list of candidate individuals responsive to a connection path between the candidate individual and the first individual user account, wherein the weights can take on any of three or more possible values corresponding to three or more different connection paths;

using a processor to perform an identification process to identify the detected unidentified person from the list of candidate individuals responsive to trained person identifiers for the candidate individuals, the extracted appearance features, and the determined weights;

associating metadata with the received digital media asset providing an indication of the identity of the detected unidentified person.

2. The method of claim 1 wherein the previously identified persons were identified by an individual tagging a particular digital media asset with the identity of persons contained in the particular digital media asset.

3. The method of claim 2 wherein the individual further tagged the particular digital media asset with a bounding box identifying the location of the previously identified persons within the particular digital media asset.

4. The method of claim 1 further including the step of querying an owner of the first individual user account to verify the identity of the one or more identified persons.

5. The method of claim 1 wherein the received digital media asset is a digital image or a digital video.

6. The method of claim 1 wherein the database of individual user accounts is an on-line social network, each individual user account having an associated internet site that is linked to internet sites for other individual user accounts.

7. The method of claim 6 wherein the collections of digital media assets are media assets that have been uploaded by the owners of the individual user accounts to their respective interne sites.

8. The method of claim 1 wherein the identification process includes:

determining match scores between the detected unidentified person and the previously identified persons;

determining weighted match scores by applying the weights to the match scores to determined weighted match scores; and

identifying the detected unidentified person to be the previously identified person corresponding to the largest weighted match score.

9. The method of claim 8 wherein some of the connection paths are personal connection paths and others of the connection paths are group connection paths, and wherein higher weights are assigned for candidate individuals having personal connection paths than for for candidate individuals having group connection paths.

10. The method of claim 1 wherein notification is provided to the one or more identified persons that they have been identified in the received digital media asset.

11. The method of claim 1 wherein the one or more identified persons are given the opportunity to verify that they were correctly identified.

12. The method of claim 1 wherein the one or more identified persons are given the opportunity to request that metadata providing an indication of their identity not be associated with the received digital media asset.

13. The method of claim 1 wherein the person identifiers include determining ratios of distances between facial features.

14. The method of claim 13 wherein the person identifier for a particular previously identified person is trained by determining characteristic ratios of distances between facial features from one or more images containing the particular previously identified persons.

15. The method of claim 1 wherein the weights are responsive to a connection distance associated with the connection paths.

16. The method of claim 15 wherein the weights include a first weight corresponding to a direct connection, a second weight corresponding to a first indirect connection, and a third weight corresponding to a second indirect connection having a connection distance larger than the first indirect connection.

17. The method of claim 15 wherein the weights provide a higher preference for candidate individuals having a smaller connection distance.

18. The method of claim 1 wherein the weights are responsive to connection types associated with the connection paths.

19. A method for identifying person in a digital image comprising steps of;

providing access to a database of individual user accounts associated with a plurality of individual users including a first user account associated with a first user;

providing one or more connections between the first user account and one or more second user accounts;

providing one or more connections between the one or more second user accounts and one or more third user accounts, the third user accounts being different than any of the first or second user accounts;

designating collections of digital media assets associated with the individual user accounts, wherein the collections of digital media assets include digital media assets containing previously identified persons, and wherein at least some of the previously identified persons are associated with corresponding individual user accounts;

receiving a first digital media asset from the first user account including an unidentified person;

using a processor to compare the unidentified person to the previously identified persons corresponding to the second user accounts determine an identity match;

using a processor to compare the unidentified person to the previously identified persons corresponding to the third user accounts determine an identity match if an identity match with the previously identified persons corresponding to the second user accounts was not found; and

associating the first digital media asset with an individual user account associated with the identity match.

20. A method for identifying an individual in a digital image comprising the steps of;

providing access to a database of individual user accounts associated with a plurality of individual users including a first user account associated with a first user;

providing one or more connections between the first user account and one or more second user accounts;

providing a plurality of user groups, each user group being associated with a plurality of user accounts;

identifying one or more third user accounts associated with user groups to which the first user account is associated;

designating collections of digital media assets associated with the individual user accounts, wherein the collections of digital media assets include digital media assets containing previously identified persons, and wherein at least some of the previously identified persons are associated with corresponding individual user accounts;

receiving a first digital media asset from the first user account including an unidentified person;

using a processor to compare the unidentified person to the previously identified persons corresponding to the second user accounts determine an identity match;

using a processor to compare the unidentified person to the previously identified persons corresponding to the third user accounts determine an identity match if an identity match with the previously identified persons corresponding to the second user accounts was not found; and

associating the first digital media asset with an individual user account associated with the identity match.