Detecting emotion in voice signals in a call center

• US 8,965,770 B2
• Filed: 03/29/2011
• Issued: 02/24/2015
• Est. Priority Date: 08/31/1999
• Status: Expired due to Fees

• Current Assignee: Accenture Global Services Limited (Accenture PLC)
• Original Assignee: Accenture Global Services Limited (Accenture PLC)
• Inventors: Petrushin, Valery A.
• Primary Examiner(s): Armstrong, Angela A
• Application Number:
  US13/074,844
• Publication Number:
  US 20110178803A1
• Time in Patent Office: 1,428 Days
• Field of Search: 704/270, 704/270.1, 704/205, 704/207
• US Class Current: 704/270
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices

Abstract

A computer system monitors a conversation between an agent and a customer. The system extracts a voice signal from the conversation and analyzes the voice signal to detect a voice characteristic of the customer. The system identifies an emotion corresponding to the voice characteristic and initiates an action based on the emotion. The action may include communicating the emotion to an emergency response team, or communicating feedback to a manager of the agent, as examples.

Claims

1. A method comprising:

monitoring, with a computer system comprising a processor, a conversation;

extracting, with the processor, a voice signal from the conversation;

ranking multiple voice parameters according to how the multiple voice parameters influence accuracy of emotion recognition;

selecting, from the ranking of the multiple voice parameters, a specific voice parameter;

analyzing, with the processor, the voice signal to obtain an extracted parameter value for the specific voice parameter associated with the voice signal;

identifying, with the processor, an emotion associated with the voice signal by;

accessing a database comprising probability statistics of possible emotions associated with the specific voice parameter; and

determining the emotion associated with the voice signal from the possible emotions using the extracted parameter value and based on a predetermined probability criteria; and

initiating, by the processor, an action based on the emotion.

2. The method of claim 1, where the multiple voice parameters comprise at least two of a maximum value of the fundamental frequency, a standard deviation of the fundamental frequency, a range of the fundamental frequency, a mean of the fundamental frequency, a mean of the bandwidth of the first formant, a mean of the bandwidth of the second format, or a standard deviation of energy.

3. The method of claim 1, where the action comprises recording a subsequent portion of the conversation.

4. The method of claim 1, where analyzing further comprises:

determining a speech rate of words spoken in the conversation.

5. The method of claim 1, where initiating comprises:

initiating the action when only when the emotion is a negative emotion.

6. The method of claim 1, wherein the probability statistics of possible emotions comprise performance confusion statistics indicating an accuracy ratio between a portrayed emotion and how the portrayed emotion was evaluated;

and

wherein determining the emotion comprises using the performance confusion statistics.

7. The method of claim 1, wherein the probability statistics of possible emotions comprise self-recognition statistics indicative of an ability of a person to accurately identify an emotion portrayed by that person;

and

wherein determining the emotion comprises using the self-recognition statistics.

8. A system for processing voice signals, the system comprising:

a processor; and

a memory in communication with the processor, the memory comprising software instructions that when executed by the processor cause the processor to;

monitor a conversation;

extract a voice signal from the conversation;

analyze the voice signal to obtain an extracted parameter value of a voice parameter associated with the voice signal, the voice parameter comprising at least one of a slope of the fundamental frequency and a maximum value of a first formant;

identify an emotion associated with the voice signal by;

accessing a database comprising probability statistics of possible emotions associated with the voice parameter; and

determining the emotion associated with the voice signal from the possible emotions using the extracted parameter value and based on a predetermined probability criteria; and

initiate an action based on the emotion.

9. The system of claim 8, wherein the probability statistics of possible emotions comprise performance confusion statistics indicating an accuracy ratio between a portrayed emotion and how the portrayed emotion was evaluated;

and

wherein determining the emotion comprises using the performance confusion statistics.

10. The system of claim 8, where the action comprises recording a subsequent portion of the conversation.

11. The system of claim 8, where the software instructions, when executed, further cause the processor to analyze the voice signal by:

detecting a speech rate of words spoken in the conversation.

12. The system of claim 8, where the software instructions, when executed, cause the processor to:

initiate the action when only when the emotion is a negative emotion.

13. The system of claim 8, wherein the probability statistics of possible emotions comprise self-recognition statistics indicative of an ability of a person to accurately identify an emotion portrayed by that person;

and

wherein determining the emotion comprises using the self-recognition statistics.

14. A product comprising:

a tangible computer readable medium; and

software instructions stored on the medium, that when executed by a processor in a computer system, cause the processor to;

monitor a conversation;

extract a voice signal from the conversation;

rank multiple voice parameters according to how the multiple voice parameters influence accuracy of emotion recognition;

select, from the ranking of the multiple voice parameters, a specific voice parameter;

analyze the voice signal to obtain an extracted parameter value of the specific voice parameter associated with the voice signal;

identify an emotion associated with the voice signal by;

accessing a database comprising probability statistics of possible emotions associated with the specific voice parameter; and

determining the emotion associated with the voice signal from the possible emotions using the extracted parameter value and based on a predetermined probability criteria; and

initiate an action based on the emotion.

15. The product of claim 14, wherein the probability statistics of possible emotions comprise performance confusion statistics indicating an accuracy ratio between a portrayed emotion and how the portrayed emotion was evaluated;

and

wherein determining the emotion comprises using the performance confusion statistics.

16. The product of claim 14, where the multiple parameters comprise at least two of a maximum value of the fundamental frequency, a standard deviation of the fundamental frequency, a range of the fundamental frequency, a mean of the fundamental frequency, a mean of the bandwidth of the first formant, a mean of the bandwidth of the second formant, or a standard deviation of energy.

17. The product of claim 14, where the action comprises recording a subsequent portion of the conversation.

18. The product of claim 14, where the software instructions, when executed, further cause the processor to analyze the voice signal by:

detecting a speech rate of words spoken in the conversation.

19. The product of claim 14, where the software instructions, when executed, cause the processor to:

initiate the action when only when the emotion is a negative emotion.

20. The product of claim 14, wherein the probability statistics of possible emotions comprise self-recognition statistics indicative of an ability of a person to accurately identify an emotion portrayed by that person;

and

wherein determining the emotion comprises using the self-recognition statistics.