Multi-Party Conversation Analyzer and Logger

• US 20160217807A1
• Filed: 03/28/2016
• Published: 07/28/2016
• Est. Priority Date: 06/24/2005
• Status: Active Grant

• Current Assignee: Securus Technologies Incorporated (Securus Special Investments LLC)
• Original Assignee: Securus Technologies Incorporated (Securus Special Investments LLC)
• Inventors: Gainsboro, Jay Loring, Weinstein, Lee Davis
• Granted Patent: US 10,127,928 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G10L 15/1807 using prosody or stress
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 21/12 by displaying time domain information
• CPC Class Codes: G10L 25/63 for estimating an emotional state
• CPC Class Codes: H04M 2201/41 using speaker recognition speaker recognition per se G10L17/00
• CPC Class Codes: H04M 3/2281 Call monitoring, e.g. for law enforcement purposes; Call tracing; Detection or prevention of malicious calls
• CPC Class Codes: H04M 3/42221 Conversation recording systems at the subscriber's set H04M1/656
• CPC Class Codes: H04M 3/568 audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30

Abstract

A multi-party conversation analyzer and logger uses a variety of techniques including spectrographic voice analysis, absolute loudness measurements, directional microphones, and telephonic directional separation to determine the number of parties who take part in a conversation, and segment the conversation by speaking party. In one aspect, the invention monitors telephone conversations in real time to detect conditions of interest (for instance, calls to non-allowed parties or calls of a prohibited nature from prison inmates). In another aspect, automated prosody measurement algorithms are used in conjunction with speaker segmentation to extract emotional content of the speech of participants within a particular conversation, and speaker interactions and emotions are displayed in graphical form. A conversation database is generated which contains conversation recordings, and derived data such as transcription text, derived emotions, alert conditions, and correctness probabilities associated with derived data. Investigative tools allow flexible queries of the conversation database.

Claims

1. A method for monitoring the emotional states of conversation participants, the method comprising:

monitoring a conversation between a plurality of participants;

determining an emotional state of one or more of the participants during one or more portions of the monitored conversation;

displaying a timeline of a segment of the monitored conversation, wherein the timeline indicates the determined emotional state of the one or more participants during the segment of the monitored conversation.

2. The method of claim 1, wherein the determined emotional state comprises a type of emotion.

3. The method of claim 2, wherein the determined type of emotion is indicated on the timeline by a color hue.

4. The method of claim 1, wherein the determined emotional state comprises an intensity of emotion.

5. The method of claim 4, wherein the determined emotional intensity is indicated on the timeline by a color saturation.

6. The method of claim 1, wherein the emotional state is determined based on the detection of prosodic indicators in the monitored conversation.

7. The method of claim 6, wherein the prosodic indicators comprise a loudness, rate of speaking, and pauses in the monitored conversation.

8. The method of claim 1, wherein the displayed timeline comprises separate timelines for the display of the determined emotion states for each of the one or more participants.

9. The method of claim 1, wherein the determined emotional state is indicated using an emotionally indicative icon.

10. The method of claim 9, wherein the color hue and color intensity of the emotionally indicative icon are varied in response to the determined emotional state.

11. A system for monitoring the emotional states of conversation participants, the system comprising:

a processor; and

a memory coupled to the processor, the memory storing computer-readable instructions that, upon execution by the processor, cause the system to;

monitor a conversation between a plurality of participants;

determine an emotional state of one or more of the participants during one or more portions of the monitored conversation;

display a timeline of a segment of the monitored conversation, wherein the timeline indicates the determined emotional state of the one or more participants during the segment of the monitored conversation.

12. The system of claim 11, wherein the determined emotional state comprises a type of emotion.

13. The system of claim 12, wherein the determined type of emotion is indicated on the timeline by a color hue.

14. The system of claim 11, wherein the determined emotional state comprises an intensity of emotion.

15. The system of claim 14, wherein the determined emotional intensity is indicated on the timeline by a color saturation.

16. The system of claim 11, wherein the emotional states are determined based on prosodic indicators in the monitored conversation.

17. The system of claim 16, wherein the prosodic indicators comprise a loudness, rate of speaking, and pauses in the monitored conversation.

18. The system of claim 11, wherein the displayed timeline comprises separate timelines for the display of the determined emotion states for each of the one or more participants.

19. The system of claim 11, wherein the determined emotional state is indicated using an emotionally indicative icon.

20. The system of claim 19, wherein the color hue and color intensity of the emotionally indicative icon are varied to indicate the determined emotional state.