EMOTION RECOGNITION APPARATUS

• US 20090313019A1
• Filed: 05/21/2007
• Published: 12/17/2009
• Est. Priority Date: 06/23/2006
• Status: Active Grant

• Current Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Hirose, Yoshifumi, Kamai, Takahiro, Nakatoh, Yoshihisa, Kato, Yumiko
• Granted Patent: US 8,204,747 B2
• Time in Patent Office: Days
• US Class Current: 704/254
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units

Abstract

An emotion recognition apparatus is capable of performing accurate and stable speech-based emotion recognition, irrespective of individual, regional, and language differences of prosodic information. The emotion recognition apparatus is an apparatus for recognizing an emotion of a speaker from an input speech, and includes: a speech recognition unit (106) which recognizes types of phonemes included in the input speech; a characteristic tone detection unit (104) which detects a characteristic tone that relates to a specific emotion, in the input speech; a characteristic tone occurrence indicator computation unit (111) which computes a characteristic tone occurrence indicator for each of the phonemes, based on the types of the phonemes recognized by the speech recognition unit (106), the characteristic tone occurrence indicator relating to an occurrence frequency of the characteristic tone; and an emotion judgment unit (113) which judges an emotion of the speaker in a phoneme at which the characteristic tone occurs in the input speech, based on the characteristic tone occurrence indicator computed by the characteristic tone occurrence indicator computing unit (111).

Claims

1. An emotion recognition apparatus that recognizes an emotion of a speaker from an input speech, comprising:

a characteristic tone detection unit operable to detect, in the input speech, a characteristic tone that relates to a specific emotion;

a speech recognition unit operable to recognize types of phonemes included in the input speech, based on the characteristic tone detected by said characteristic tone detection unit;

a characteristic tone occurrence indicator computation unit operable to compute a characteristic tone occurrence indicator for each of the phonemes, based on the types of the phonemes recognized by said speech recognition unit, the characteristic tone occurrence indicator indicating a likelihood of the phoneme being spoken with the characteristic tone; and

an emotion judgment unit operable to judge an emotion of the speaker in a phoneme at which the characteristic tone occurs in the input speech, using the characteristic tone occurrence indicator computed by said characteristic tone occurrence indicator computing unit, in accordance with a rule that the specific emotion is stronger when the characteristic tone occurrence indicator is smaller.

2. The emotion recognition apparatus according to claim 1, further comprising

an emotion strength judgment unit operable to judge a strength of the emotion of the speaker in the phoneme at which the characteristic tone occurs, in accordance with a computation rule that a strength of the specific emotion is greater when the characteristic tone occurrence indicator is smaller.

3. The emotion recognition apparatus according to claim 2,

wherein said emotion strength judgment unit is operable to:

check a time position at which the characteristic tone detected by said characteristic tone detection unit occurs in the input speech, and the characteristic tone occurrence indicator computed for each of the phonemes by said characteristic tone occurrence indicator computation unit; and

judge the strength of the emotion of the speaker in the phoneme at which the characteristic tone occurs, in accordance with the computation rule that the strength of the specific emotion is greater when the characteristic tone occurrence indicator is smaller.

4. The emotion recognition apparatus according to claim 1,

wherein said characteristic tone detection unit is operable to detect a tone of a voice whose vowel part has a fluctuation in a sound source, as the characteristic tone.

5. The emotion recognition apparatus according to claim 1, further comprising

at least a feature parameter database storing a feature parameter of a voice which includes the characteristic tone, for each phoneme type,

wherein said speech recognition unit is operable to recognize the types of the phonemes included in the input speech, based on said feature parameter database.

6. The emotion recognition apparatus according to claim 5,

wherein said feature parameter database includes:

a first database storing a feature parameter of a voice which includes at least one characteristic tone including the characteristic tone, for each phoneme type; and

a second database storing a feature parameter of a voice which includes no characteristic tone, for each phoneme type,

said emotion recognition apparatus further comprises

a selection unit operable to select a database corresponding to a result of the detection by said characteristic tone detection unit, from said first database and said second database, and

said speech recognition unit is operable to recognize the types of the phonemes included in the input speech, based on the database selected by said selection unit.

7. The emotion recognition apparatus according to claim 1, further comprising:

an acoustic feature parameter database storing an acoustic feature parameter for each phoneme type; and

a language feature parameter database storing a language feature parameter which represents a word dictionary including at least one of a reading and a pronunciation symbol,

wherein said speech recognition unit is operable to recognize the types of the phonemes included in the input speech based on said acoustic feature parameter database and said language feature parameter database by, for a word in which the characteristic tone is detected, decreasing a weight of the acoustic feature parameter included in said acoustic feature parameter database and increasing a weight of the language feature parameter included in said language feature parameter database.

8. An emotion recognition apparatus that recognizes an emotion of a speaker from an input speech, comprising:

a characteristic tone detection unit operable to detect, in the input speech, a characteristic tone that relates to a specific emotion;

a phoneme input unit operable to input types of phonemes included in the input speech;

a characteristic tone occurrence indicator computation unit operable to compute a characteristic tone occurrence indicator for each of the phonemes, by using at least the types of the phonemes inputted by said phoneme input unit as parameters, the characteristic tone occurrence indicator indicating a likelihood of the phoneme being spoken with the characteristic tone; and

an emotion judgment unit operable to judge an emotion of the speaker in a phoneme at which the characteristic tone occurs in the input speech, using the characteristic tone occurrence indicator computed by said characteristic tone occurrence indicator computation unit, in accordance with a correspondence rule that the specific emotion is stronger when the characteristic tone occurrence indicator is smaller.

9. An emotion recognition apparatus that recognizes an emotion of a speaker from an input speech, comprising:

a speech recognition unit operable to recognize types of phonemes included in the input speech;

a characteristic tone detection unit operable to extract, from the input speech, a pressed speech portion whose vowel part has an amplitude fluctuation or a frequency fluctuation in a sound source; and

an emotion judgment unit operable to judge, for each of the phonemes recognized by said speech recognition unit, that an emotion of the speaker is strong anger in a case where the pressed speech portion detected by said characteristic tone detection unit is a phoneme including at least one of an unvoiced labial plosive, an unvoiced dental affricate, and an unvoiced labio-dental fricative.

10. An emotion recognition method for recognizing an emotion of a speaker from an input speech, comprising:

a characteristic tone detection step of detecting, in the input speech, a characteristic tone that relates to a specific emotion;

a speech recognition step of recognizing types of phonemes included in the input speech, based on the characteristic tone detected in said characteristic tone detection step;

a characteristic tone occurrence indicator computation step of computing a characteristic tone occurrence indicator for each of the phonemes, based on the types of the phonemes recognized in said speech recognition step, the characteristic tone occurrence indicator indicating a likelihood of the phoneme being spoken with the characteristic tone; and

an emotion judgment step of judging an emotion of the speaker in a phoneme at which the characteristic tone occurs in the input speech, using the characteristic tone occurrence indicator computed in said characteristic tone occurrence indicator computing step, in accordance with a rule that the specific emotion is stronger when the characteristic tone occurrence indicator is smaller.

11. The emotion recognition method according to claim 10, further comprising an emotion strength judgment step of judging a strength of the emotion of the speaker in the phoneme at which the characteristic tone occurs, in accordance with a computation rule that a strength of the specific emotion is greater when the characteristic tone occurrence indicator is smaller.

12. A program for recognizing an emotion of a speaker from an input speech, said program causing a computer to execute:

a characteristic tone detection step of detecting, in the input speech, a characteristic tone that relates to a specific emotion;

a speech recognition step of recognizing types of phonemes included in the input speech, based on the characteristic tone detected in said characteristic tone detection step;

a characteristic tone occurrence indicator computation step of computing a characteristic tone occurrence indicator for each of the phonemes, based on the types of the phonemes recognized in said speech recognition step, the characteristic tone occurrence indicator indicating a likelihood of the phoneme being spoken with the characteristic tone; and

an emotion judgment step of judging an emotion of the speaker in a phoneme at which the characteristic tone occurs in the input speech, using the characteristic tone occurrence indicator computed in said characteristic tone occurrence indicator computing step, in accordance with a rule that the specific emotion is stronger when the characteristic tone occurrence indicator is smaller.

13. The program according to claim 12, further causing the computer to execute

an emotion strength judgment step of judging a strength of the emotion of the speaker in the phoneme at which the characteristic tone occurs, in accordance with a computation rule that a strength of the specific emotion is greater when the characteristic tone occurrence indicator is smaller.