Multi-party conversation analyzer and logger

• US 10,084,920 B1
• Filed: 01/26/2015
• Issued: 09/25/2018
• Est. Priority Date: 06/24/2005
• Status: Active Grant

• Current Assignee: Securus Technologies Incorporated (Securus Special Investments LLC)
• Original Assignee: Securus Technologies Incorporated (Securus Special Investments LLC)
• Inventors: Gainsboro, Jay, Weinstein, Lee
• Primary Examiner(s): Bezuayehu, Solomon
• Application Number:
  US14/605,915
• Time in Patent Office: 1,338 Days
• Field of Search: 379 671, 379 8801- 8819
• CPC Class Codes: G10L 15/1807 using prosody or stress
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 21/12 by displaying time domain information
• CPC Class Codes: G10L 25/63 for estimating an emotional state
• CPC Class Codes: H04M 2201/41 using speaker recognition speaker recognition per se G10L17/00
• CPC Class Codes: H04M 3/2281 Call monitoring, e.g. for law enforcement purposes; Call tracing; Detection or prevention of malicious calls
• CPC Class Codes: H04M 3/42221 Conversation recording systems at the subscriber's set H04M1/656
• CPC Class Codes: H04M 3/568 audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30

Abstract

A multi-party conversation analyzer and logger uses a variety of techniques including spectrographic voice analysis, absolute loudness measurements, directional microphones, and telephonic directional separation to determine the number of parties who take part in a conversation, and segment the conversation by speaking party. In one aspect, the invention monitors telephone conversations in real time to detect conditions of interest (for instance, calls to non-allowed parties or calls of a prohibited nature from prison inmates). In another aspect, automated prosody measurement algorithms are used in conjunction with speaker segmentation to extract emotional content of the speech of participants within a particular conversation, and speaker interactions and emotions are displayed in graphical form. A conversation database is generated which contains conversation recordings, and derived data such as transcription text, derived emotions, alert conditions, and correctness probabilities associated with derived data. Investigative tools allow flexible queries of the conversation database.

Claims

1. A method for identifying participants in a recorded voice conversation, the method comprising:

receiving a plurality of audio signals that comprise the recorded voice conversation;

diarizing the plurality of audio signals using directional separation of the audio signals based on the origin of each audio signal;

separating the diarized audio signals into a plurality of separate channels; and

sorting speech segments identified in a first channel of the plurality of separate channels based on audio features identified in the speech segments, wherein the audio features comprise spectral shape characteristics of the speech segments, and wherein the audio features further comprise compression algorithms identified as encoding one or more of the speech segments of the first channel.

2. The method of claim 1, wherein the audio features comprise an absolute loudness envelope identified in one or more speech segments of the first channel.

3. The method of claim 1, wherein the directional separation used to diarize the plurality of audio signals is telephonic directional separation.

4. The method of claim 1, further comprising:

grouping the sorted speech segments of the first channel, wherein a group of speech segments within the first channel indicates multiple participants within the first channel.

5. The method of claim 1, wherein the spectral shape characteristics are indicative of distinct phone extensions.

6. The method of claim 1, wherein the audio features comprise distinct voices identified within the one or more speech segments of the first channel.

7. The method of claim 6, wherein the distinct voices are identified within the voices of each person known to be participating in the voice conversation.

8. The method of claim 1, wherein the identified compression algorithms indicate a type of communication device used to encode the one or more speech segments.

9. The method of claim 8, wherein the type of communication device is a cell phone.

10. A system for identifying participants in a recorded voice conversation, the system comprising:

one or more processors; and

a memory coupled to the one or more processors, the memory storing computer-readable instructions that, upon execution by the one or more processors, cause the system to;

receive a plurality of audio signals that comprise the recorded voice conversation;

diarize the plurality of audio signals using directional separation of the audio signals based on the origin of each audio signal;

separate the diarized audio signals into a plurality of separate channels; and

sort speech segments identified in a first channel of the plurality of separate channels based on audio features identified in the speech segments, wherein the audio features comprise spectral shape characteristics of the speech segments, and wherein the audio features further comprise compression algorithms identified as encoding one or more of the speech segments of the first channel.

11. The system of claim 10, wherein the audio features comprise an absolute loudness envelope in one or more speech segments of the first channel.

12. The system of claim 10, wherein the directional separation used to diarize the plurality of audio signals is telephonic directional separation.

13. The system of claim 10, the memory device storing additional computer-readable instructions that, upon execution by the one or more processors, further cause the system to:

group the sorted speech segments of the first channel, wherein a group of speech segments within the first channel indicates multiple participants within the first channel.

14. The system of claim 10, wherein the spectral shape characteristics are indicative of distinctive phone extensions.

15. The system of claim 10, wherein the audio features comprise distinct voices identified within the one or more speech segments of the first channel.

16. The system of claim 15, wherein the distinct voices are identified within the voices of each person known to be participating in the voice conversation.

17. The system of claim 10, wherein the identified compression algorithms indicate a type of communication device used to encode the one or more speech segments.

18. The system of claim 17, wherein the type of communication device is a cell phone.