Method and apparatus for recognizing a speaker in lawful interception systems

• US 8,219,404 B2
• Filed: 08/09/2007
• Issued: 07/10/2012
• Est. Priority Date: 08/09/2007
• Status: Active Grant

• Current Assignee: Cyberbit Ltd.
• Original Assignee: Nice Systems Limited (Nice Ltd)
• Inventors: Weinberg, Adam, Opher, Irit, Benaroya, Eyal, Gutman, Renan
• Primary Examiner(s): AZAD, ABUL K
• Application Number:
  US11/836,213
• Publication Number:
  US 20090043573A1
• Time in Patent Office: 1,797 Days
• Field of Search: 704246-250, 704/273
• US Class Current: 704/273
• CPC Class Codes: G10L 17/06 Decision making techniques; Pattern matching strategies
• CPC Class Codes: H04L 63/302 gathering intelligence information for situation awareness or reconnaissance

Abstract

A method and apparatus for identifying a speaker within a captured audio signal from a collection of known speakers. The method and apparatus receive or generate voice representations for each known speakers and tag the representations according to meta data related to the known speaker or to the voice. The representations are grouped into one or more groups according to the indices. When a voice to be recognized is introduced, characteristics are determined according to which the groups are prioritized, so that the representations participating only in part of the groups are matched against the voice to be identified, thus reducing identification time and improving the statistical significance.

Claims

1. A method for associating a voice of a first human speaker with at least one of a multiplicity of speakers, each of the multiplicity of speakers associated with an acoustic model and with data, the method comprising the steps of:

capturing an audio signal of the first human speaker communicating with a second human speaker and extracting the voice of the first human speaker;

receiving or extracting the data associated with each of the multiplicity of speakers;

generating tags associated with each of the acoustic models of the multiplicity of speakers;

wherein the tags represent an at least one parameter associated with the acoustic model of the speaker and with at least one parameter associated with the acoustic models of persons who are known to be connected directly or indirectly via one or more middle persons to a known speaker in the interaction;

constructing one or more groups comprising acoustic models of speakers based on tag value similarity;

determining an order for the groups by giving high priority to groups with persons who have direct connection with the first speaker or the second speaker and lower priority to groups with persons having indirect connection to the first speaker or the second speaker;

selecting at least one group to be matched against the voice of the first speaker;

determining an at least one non-acoustic score between data related to the first speaker, and the tags of the at least one matched group or the tags of an at least one acoustic model from the at least one matched group;

determining an at least one acoustic score between the voice of the first speaker and an at least one acoustic model from the at least one matched group;

obtaining a total score by combining the non-acoustic score with the acoustic score;

determining according to the total score whether an identification criteria was met; and

if the identification criteria was met, associating the first speaker with the at least one model from the matched group.

2. The method of claim 1 further comprising the step of constructing the acoustic model.

3. The method of claim 1 wherein the data associated with each of the multiplicity of speakers is meta data related to any at least one of the multiplicity of speakers.

4. The method of claim 1 wherein the data associated with each of the multiplicity of speakers relates to the acoustic model.

5. The method of claim 1 wherein another parameter relates to the acoustic model of the speaker.

6. The method of claim 1 wherein generating tags is performed according to a level of connectivity between the second speaker the first speaker was communicating with, and an at least one of the multiplicity of speakers.

7. The method of claim 1 wherein generating tags is performed according to a time of communication between the second speaker the first speaker was communicating with, and an at least one of the multiplicity of speakers.

8. The method of claim 1 wherein generating tags is performed according to a predetermined group of speakers.

9. The method of claim 1 wherein the at least one parameter relates to any one or more of the group consisting of:

characteristics of a second speaker that communicated with the speaker from the multiplicity of speakers;

identity of the second speaker;

age of the second speaker;

accent of the second speaker;

language spoken by the second speaker;

a feature of the acoustic model of the second speaker;

data extracted from a voice sample of the second speaker;

level of connectivity between the speaker and the second speaker;

levels of connectivity between the second speakers and persons they are interacting with;

an at least one word used by the second speaker;

an at least one name mentioned by the second speaker;

a location associated with the second speaker;

a phone number or part thereof associated with the second speaker;

a pronunciation of an at least one phoneme by the second speaker;

a characteristic of a channel used by the second speaker; and

a time of an at least one communication of the second speaker.

10. The method of claim 1 wherein the data related to the first human speaker relates to any one or more of the group consisting of:

identity of the first human speaker;

age of the first human speaker;

accent of the first human speaker;

language spoken by the first human speaker;

a characteristic of the at least one voice model;

data extracted from the voice sample;

level of connectivity between the first speaker and the second human speaker the first speaker was communicating with;

levels of connectivity between the second human speakers and persons they interact with;

an at least one word used by the first human speaker;

an at least one name mentioned by the first human speaker;

a location associated with the first human speaker;

a phone number or part thereof associated with first human speaker;

a pronunciation of one or more phonemes by the first human speaker;

a characteristic of a channel used by the first human speaker; and

a time of an at least one communication of the first human speaker.

11. The method of claim 1 wherein the audio signal is in a format selected from the group consisting of:

PCM, a-law, mu-law, GSM, CDMA, TDMA, ADPCM and VOIP.

12. The method of claim 1 wherein constructing one or more groups is performed according to geographic proximity of the speakers to a location from which the first human speaker is communicating.

13. The method of claim 1 wherein constructing one or more groups is performed according to words used by the speakers.

14. The method of claim 1 wherein constructing one or more groups is performed according to geographic proximity of the speakers to a location mentioned by the first human speaker.

15. An apparatus for associating a voice of a first human speaker, with at least one of a multiplicity of speakers, each of the multiplicity of speakers associated with an acoustic model and with data, the apparatus comprising:

a storage device for storing the acoustic model and associated meta data;

a capturing or logging component for receiving a voice sample of the first human speaker extracted from a captured audio signal of a communication between the first human speaker and a second human speaker;

a tagging component for generating tags associated with the acoustic models of the multiplicity of speakers;

wherein the tags represent an at least one parameter associated with the acoustic model of the speaker and with at least one parameter associated with the acoustic models of persons who are known to be connected directly or indirectly via one or more middle persons to a known speaker in the interaction; and

for constructing one or more groups comprising acoustic models of speakers based on tag value similarity;

a selection component for determining an order for the groups by giving high priority to groups with persons who have direct connection with the first speaker or the second speaker and lower priority to groups with persons having indirect connection to the first speaker or the second speaker; and

selecting a group comprising an at least one acoustic model for matching with the voice sample of the first speaker;

a non-acoustic score determination component, for determining a non-acoustic score between data related to the first speaker, and the tags of the at least one matched group or the tags of an at least one acoustic model from the at least one matched group;

an acoustic score determination component for determining an acoustic score between the voice of the first speaker and an at least one acoustic model from the at least one matched group;

a combining component for combining the acoustic score and the non-acoustic score into a total score; and

a criteria evaluation component for determining whether the total score meets an at least one criteria.

16. The apparatus of claim 15 further comprising a group determination order for determining a matching order between at least two groups.

17. The apparatus of claim 15 further comprising a model determination order for determining a matching order between at least two models belonging to the same group.

18. The apparatus of claim 15 further comprising a model generation component for generating an acoustic model from a voice sample.

19. The apparatus of claim 15 further comprising a data extraction component for extracting data related to a voice sample or to a speaker thereof.

20. The apparatus of claim 15 further comprising an alert generation device for generating an alert when the first speaker is identified as at least one of the multiplicity of speakers.

21. The apparatus of claim 15 further comprising a result reporting component for reporting a result related to matching the first speaker and the at least one matched model.

22. The apparatus of claim 15 wherein the data associated with each of the multiplicity of speakers is meta data related to any at least one of the multiplicity of speakers.

23. The apparatus of claim 15 wherein the captured audio signal represents any one or more items selected from the group consisting of:

a phone conversation;

a voice over IP conversation;

an audio part of a video conference;

a radio broadcast;

an audio part of a television broadcast; and

a captured microphone.

24. The apparatus of claim 15 wherein the captured audio signal is in a format selected from the group consisting of:

PCM, a-law, mu-law, GSM, CDMA, TDMA, ADPCM and VOIP.

25. The apparatus of claim 15 wherein the associated meta data relates to an at least one level of connectivity between the second human speaker the first human speaker was communicating with and an at least one speaker associated with the voice models.

26. The apparatus of claim 15 wherein the at least one parameter relates to any one or more of the group consisting of:

characteristics of the second speaker that communicated with the speaker from the multiplicity of speakers;

identity of the second speaker;

age of the second speaker;

accent of the second speaker;

language spoken by the second speaker;

a feature of the acoustic model of the second speaker;

data extracted from a voice sample of the second speaker;

level of connectivity between the speaker and the second speaker;

levels of connectivity between the second speakers and persons they communicate with;

one or more words used by the second speaker;

one or more names mentioned by the second speaker;

a location associated with the second speaker;

a phone number or part thereof associated with the second speaker;

a pronunciation of one or more phonemes by the second speaker;

a characteristic of a channel used by the second speaker and a time of an at least one communication of the second speaker.

27. The apparatus of claim 15 wherein the data related to the first speaker relates to any one or more of the group consisting of:

identity of the first speaker;

age of the first speaker;

accent of the first speaker;

language spoken by the first speaker;

a feature of the at least one voice model;

data extracted from the second voice sample;

level of connectivity between the first speaker and another speaker;

an at least one word used by the first speaker;

an at least one name mentioned by the first speaker;

a location associated with the first speaker;

a phone number or part thereof associated with the first speaker;

a pronunciation of an at least one phoneme by the first speaker;

a characteristic of a channel used by the first speaker; and

a time of an at least one communication of the first speaker.

28. The apparatus of claim 15 wherein the tagging is performed according to a level of connectivity between the second speaker the first speaker was communicating with, and an at least one of the multiplicity of speakers.

29. The apparatus of claim 15 wherein the tagging is performed according to a time of communication between the second speaker the first speaker was communicating with, and an at least one of the multiplicity of speakers.

30. The apparatus of claim 15 wherein the tagging is performed according to a predetermined group of speakers.

31. A method for associating a voice of a first human speaker with at least one of a multiplicity of speakers, each of the multiplicity of speakers associated with an acoustic model and with data, the method comprising the steps of:

capturing an audio signal of the first human speaker communicating with a second human speaker and extracting the voice of the first human speaker and the second human speaker;

receiving or extracting the data associated with each of the multiplicity of speakers;

generating tags associated with each of the acoustic models of the multiplicity of speakers;

wherein the tags represent an at least one parameter associated with the acoustic model of the speaker and with at least one parameter associated with the acoustic models of persons who are known to be connected directly or indirectly via one or more middle persons to a known speaker in the interaction;

constructing one or more groups comprising acoustic models of speakers based on tag value similarity;

determining an order for the groups by giving high priority to groups with persons who have direct connection with the first speaker or the second speaker and lower priority to groups with persons having indirect connection to the first speaker or the second speaker;

selecting at least one group to be matched against the voice of the first speaker or voice of the second speaker according to the group order; and

associating the first speaker or the second speaker with the at least one model from the matched group.

32. The method of claim 31 wherein the associating is performed by determining an at least one non-acoustic score between data related to the first speaker, and the tags of the at least one matched group or the tags of an at least one acoustic model from the at least one matched group;

determining an at least one acoustic score between the voice of the first speaker and an at least one acoustic model from the at least one matched group;

obtaining a total score by combining the non-acoustic score with the acoustic score;

determining according to the total score whether an identification criteria was met; and

if the identification criteria was met, associating the first speaker with the at least one model from the matched group.