User identification with voiceprints on online social networks

• US 10,607,148 B1
• Filed: 12/21/2016
• Issued: 03/31/2020
• Est. Priority Date: 12/21/2016
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Niewczas, Mateusz Marek
• Primary Examiner(s): Hicks, Austin
• Application Number:
  US15/386,393
• Time in Patent Office: 1,196 Days
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: G10L 17/06 Decision making techniques; Pattern matching strategies
• CPC Class Codes: G10L 17/08 Use of distortion metrics or a particular distance between probe pattern and reference templates
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 63/0861 using biometrical features, e.g. fingerprint, retina-scan cryptographic mechanisms or cryptographic arrangements for entity authentication using biological data H04L9/3231
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/306 User profiles
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal
• CPC Class Codes: H04L 67/53 using third party service providers
• CPC Class Codes: H04W 12/02 Protecting privacy or anonymity, e.g. protecting personally identifiable information [PII]
• CPC Class Codes: H04W 12/63 Location-dependent; Proximity-dependent
• CPC Class Codes: H04W 4/21 for social networking applications
• CPC Class Codes: H04W 4/80 Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication

Abstract

In one embodiment, a method includes, by one or more computing devices of an online social network, receiving, from a client system of a first user of the online social network, a first audio input from an unknown user, identifying one or more candidate users, wherein each candidate user is a user of the online social network within a threshold degree of separation of a known user, calculating, for each candidate user, a probability score representing a probability that the unknown user is the candidate user, wherein the probability score is based on a comparison of the first audio input to a voiceprint of the candidate user stored by the online social network, wherein each voiceprint comprises audio data for auditory identification of the candidate user, and identifying one of the candidate users as being the unknown user based on the calculated probability scores of the candidate users.

Claims

1. A method comprising, by one or more computing devices of an online social network:

receiving, from a client system of a first user of the online social network, a first audio input from an unknown user;

identifying one or more candidate users, wherein each candidate user is a user of the online social network within a threshold degree of separation of a known user;

determining, for each candidate user, a proximity of the candidate user to the known user;

calculating, for each candidate user, a probability score representing a probability that the unknown user is the candidate user, wherein the probability score is based on the proximity of the candidate user and a comparison of the first audio input to a voiceprint of the candidate user stored by the online social network, wherein each voiceprint comprises audio data for auditory identification of the candidate user; and

identifying one of the candidate users as being the unknown user based on the calculated probability scores of the candidate users.

2. The method of claim 1, wherein the client system of the first user comprises one or more of a mobile device, a media device, or a beacon.

3. The method of claim 1, wherein the known user is the first user, the candidate users comprise the first user, and the probability score is proportional to a number of times the first audio input matches a particular voiceprint.

4. The method of claim 1, wherein the known user is the first user, the candidate users comprise one or more users within the threshold degree of separation of the first user, and the probability score is lower than a threshold probability score.

5. The method of claim 1, further comprising:

receiving, at the client system of the first user, a second audio input; and

increasing the probability score in response to the second audio input matching a voiceprint of the known user and the second audio input comprising a name of the candidate user spoken by the known user.

6. The method of claim 1, wherein the determining that the candidate user is proximate to the known user comprises detecting that the candidate user has checked in at a location proximate to the known user.

7. The method of claim 1, further comprising:

receiving, by a listening device in a known location, a second audio input; and

increasing the probability score in response to detecting that the candidate user has checked in at a location proximate to the listening device.

8. The method of claim 7, further comprising increasing the probability score in response to detecting that the known user is located proximate to the listening device.

9. The method of claim 7, wherein detecting that the known user is located proximate to the listening device comprises one or more of:

detecting that the known user has checked in at a location proximate to the listening device,

detecting that the second audio input matches a voiceprint of the known user, or

detecting that the second audio input comprises a name of the candidate user spoken by the known user.

10. The method of claim 1, wherein the first user is one or more of a user logged into the client system, a user using an application on the client system, or an owner of the client system.

11. The method of claim 1, further comprising:

receiving identity information for the unknown user;

generating a new voiceprint based on the first audio input; and

storing the new voiceprint in association with the identity information for subsequent access by the online social network.

12. The method of claim 1, wherein the probability score is determined based on a behavioral pattern of the first user.

13. The method of claim 12, wherein the probability score is increased when the behavioral pattern comprises the first user periodically checking in at a location, and a listening device detects the first user'"'"'s voice at the location.

14. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

receive, from a client system of a first user of an online social network, a first audio input from an unknown user;

identify one or more candidate users, wherein each candidate user is a user of the online social network within a threshold degree of separation of the first a known user;

determine, for each candidate user, a proximity of the candidate user to the known user;

calculate, for each candidate user, a probability score representing a probability that the unknown user is the candidate user, wherein the probability score is based on the proximity of the candidate user and a comparison of the first audio input to a voiceprint of the candidate user stored by the online social network, wherein each voiceprint comprises audio data for auditory identification of the candidate user; and

identify one of the candidate users as being the unknown user based on the calculated probability scores of the candidate users.

15. The media of claim 14, wherein the client system of the first user comprises one or more of a mobile device, a media device, or a beacon.

16. The media of claim 14, wherein the known user is the first user, the candidate users comprise the first user, and the probability score is proportional to a number of times the first audio input matches a particular voiceprint.

17. A system comprising:

one or more processors; and

a non-transitory memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to;

receive, from a client system of a first user of an online social network, a first audio input from an unknown user;

identify one or more candidate users, wherein each candidate user is a user of the online social network within a threshold degree of separation of the first a known user;

determine, for each candidate user, a proximity of the candidate user to the known user;

calculate, for each candidate user, a probability score representing a probability that the unknown user is the candidate user, wherein the probability score is based on the proximity of the candidate user and a comparison of the first audio input to a voiceprint of the candidate user stored by the online social network, wherein each voiceprint comprises audio data for auditory identification of the candidate user; and

identify one of the candidate users as being the unknown user based on the calculated probability scores of the candidate users.

18. The system of claim 17, wherein the client system of the first user comprises one or more of a mobile device, a media device, or a beacon.

19. The system of claim 17, wherein the known user is the first user, the candidate users comprise the first user, and the probability score is proportional to a number of times the first audio input matches a particular voiceprint.