Voice messaging system that organizes voice messages based on detected emotion

• US 6,697,457 B2
• Filed: 08/31/1999
• Issued: 02/24/2004
• Est. Priority Date: 08/31/1999
• Status: Expired due to Term

• Current Assignee: Accenture Global Services Limited (Accenture PLC)
• Original Assignee: Accenture LLP (Accenture PLC)
• Inventors: Petrushin, Valery A.
• Primary Examiner(s): Weaver, Scott L.
• Assistant Examiner(s): Foster, Roland G.
• Application Number:
  US09/387,166
• Publication Number:
  US 20020002460A1
• Time in Patent Office: 1,638 Days
• Field of Search: 379/67.1, 379/88.01, 379/88.04, 379/88.07, 379/88.08, 379/88.11, 379/88.22, 379/88.23, 379/88.25, 704/231, 704/246, 704/270, 704/275, 704/276, 704/200, 704/205, 704/207, 704/209
• US Class Current: 379/88.08
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices
• CPC Class Codes: H04M 2201/41 using speaker recognition speaker recognition per se G10L17/00
• CPC Class Codes: H04M 2203/2061 Language aspects
• CPC Class Codes: H04M 2203/301 Management of recordings
• CPC Class Codes: H04M 3/382 using authorisation codes or passwords
• CPC Class Codes: H04M 3/533 Voice mail systems
• CPC Class Codes: H04M 3/5335 Message type or catagory, e.g. priority, indication

Abstract

A system, method and article of manufacture are provided for managing voice messages based on emotion characteristics of the voice messages. First, a plurality of voice messages that are transferred over a telecommunication network are received. Thereafter, such voice messages are stored on a storage medium. An emotion associated with voice signals of the voice messages is then determined. The voice messages are organized based on the determined emotion. Access to the organized voice messages is then permitted.

Claims

1. A method for managing voice messages based on emotion characteristics of the voice messages comprising the steps of:

(a) receiving a plurality of voice messages transferred over a telecommunication network, wherein the voice messages include at least one voice signal;

(b) storing the voice messages on a storage medium;

(c) extracting at least one feature from the voice signal selected from a group of features consisting of a maximum value of a fundamental frequency, a standard deviation of the fundamental frequency, a range of the fundamental frequency, a mean of the fundamental frequency, a mean of a bandwidth of a first formant, a mean of a bandwidth of a second formant, a standard deviation of energy, a speaking rate, a slope of the fundamental frequency, a maximum value of the first formant, a maximum value of the energy, a range of the energy, a range of the second formant, and a range of the first formant;

(d) determining an emotion associated with the voice signals of the voice messages based on said feature of the voice signals;

(e) organizing the voice messages based on the determined emotion; and

(f) allowing access to the organized voice messages.

2. A method as recited in claim 1, wherein the voice messages follow a telephone call.

3. A method as recited in claim 1, wherein the voice messages of a similar emotion are organized together.

4. A method as recited in claim 1, wherein the voice messages are organized in real time immediately upon receipt over the telecommunication network.

5. A method as recited in claim 1, and further comprising the step of identifying a manner in which the voice messages are organized for facilitating access to the organized voice messages.

6. A computer program embodied on a computer readable medium for managing voice messages based on emotion characteristics of the voice messages comprising:

(a) a code segment that receives a plurality of voice messages transferred over a telecommunication network, wherein the voice messages include at least one voice signal;

(b) a code segment that stores the voice messages on a storage medium;

(c) a code segment that extracts at least one feature from the voice signal selected from a group of features consisting of a maximum value of a fundamental frequency, a standard deviation of the fundamental frequency, a range of the fundamental frequency, a mean of the fundamental frequency, a mean of a bandwidth of a first formant, a mean of a bandwidth of a second formant, a standard deviation of energy, a speaking rate, a slope of the fundamental frequency, a maximum value of the first formant, a maximum value of the energy, a range of the energy, a range of the second formant, and a range of the first formant;

(d) a code segment that determines an emotion associated with voice signals of the voice messages;

(e) a code segment that organizes the voice messages based on the determined emotion; and

(f) a code segment that allows access to the organized voice messages.

7. A computer program as recited in claim 6, wherein the voice messages follow a telephone call.

8. A computer program as recited in claim 6, wherein the voice messages of a similar emotion are organized together.

9. A computer program as recited in claim 6, wherein the voice messages are organized in real time immediately upon receipt over the telecommunication network.

10. A computer program as recited in claim 6, and further comprising a code segment that identifies a manner in which the voice messages are organized for facilitating access to the organized voice messages.

11. A system for managing voice messages based on emotion characteristics of the voice messages comprising:

(a) logic that receives a plurality of voice messages transferred over a telecommunication network, wherein the voice messages include at least one voice signal;

(b) logic that stores the voice messages on a storage medium;

(c) logic that extracts at least one feature from the voice signal selected from a group of features consisting of a maximum value of a fundamental frequency, a standard deviation of the fundamental frequency, a range of the fundamental frequency, a mean of the fundamental frequency, a mean of a bandwidth of a first formant, a mean of a bandwidth of a second formant, a standard deviation of energy, a speaking rate, a slope of the fundamental frequency, a maximum value of the first formant, a maximum value of the energy, a range of the energy, a range of the second formant, and a range of the first formant;

(d) logic that determines an emotion associated with voice signals of the voice messages;

(e) logic that organizes the voice messages based on the determined emotion; and

(f) logic that allows access to the organized voice messages.

12. A system as recited in claim 11, wherein the voice messages follow a telephone call.

13. A system as recited in claim 11, wherein the voice messages of a similar emotion are organized together.

14. A system as recited in claim 11, wherein the voice messages are organized in real time immediately upon receipt over the telecommunication network.

15. A system as recited in claim 11, and further comprising logic that identifies a manner in which the voice messages are organized for facilitating access to the organized voice messages.

16. A method for managing voice messages based on emotion characteristics of the voice messages, comprising the steps of:

(a) receiving a plurality of voice messages transferred over a telecommunication network;

(b) extracting at least one segment of audio frequency from each said voice message;

(c) extracting at least one audio feature from the segment of audio frequency,

wherein the at least one audio feature is selected from a group of audio features consisting of a maximum value of a fundamental frequency, a standard deviation of the fundamental frequency, a range of the fundamental frequency, a mean of the fundamental frequency, a mean of a bandwidth of a first formant, a mean of a bandwidth of a second formant, a standard deviation of energy, a speaking rate, a slope of the fundamental frequency, a maximum value of the first formant, a maximum value of the energy, a range of the energy, a range of the second formant, and a range of the first formant; and

(d) determining an emotion associated with said voice message using the at least one audio feature of the voice message.

17. The method of claim 16, further comprising the step of organizing the voice messages based on the determined emotion.

18. The method of claim 17, further comprising the step of allowing access to the organized voice messages.

19. The method of claim 16, wherein said emotion is determined by using the at least one audio feature as an input to a neural network containing at least one algorithm that is used to determine emotion.

20. The method of claim 16, wherein said emotion is determined by using the at least one audio feature as an input to an ensemble of classifiers that is used to determine emotion.

21. The method of claim 16, wherein voice messages of a similar emotion are organized together.

22. The method of claim 16, further comprising the step of notifying a third-party based on the detection of a predetermined emotion in said voice message.