Emotion recognition method and device

• US 7,451,079 B2
• Filed: 07/12/2002
• Issued: 11/11/2008
• Est. Priority Date: 07/13/2001
• Status: Expired due to Fees

• Current Assignee: Sony France SA (Sony Group Corp.)
• Original Assignee: Sony France SA (Sony Group Corp.)
• Inventors: Oudeyer, Pierre-Yves
• Primary Examiner(s): Hudspeth; David R.
• Assistant Examiner(s): Sked; Matthew J.
• Application Number:
  US10/194,848
• Publication Number:
  US 20030055654A1
• Time in Patent Office: 2,314 Days
• Field of Search: None
• US Class Current: 704/205
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 13/033 Voice editing, e.g. manipulating the voice of the synthesiser
• CPC Class Codes: G10L 13/04 Details of speech synthesis systems, e.g. synthesiser structure or memory management
• CPC Class Codes: G10L 13/10 Prosody rules derived from text; Stress or intonation
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices

Abstract

Emotion recognition is performed by extracting a set comprising at least one feature derived from a signal, and processing the set of extracted feature(s) to detect an emotion therefrom. The voice signal is low pass filtered prior to extracting therefrom at least one feature of the set. The cut-off frequency for the low pass filtering is typically centered around 250 Hz. The features are e.g. statistical quantities extracted from sampling a signal of the intensity or pitch of the voice signal.

Claims

1. A method of detecting an emotion conveyed in a voice signal, comprising the sequential steps of:

low-pass filtering said voice signal,

extracting a set comprising at least one prosodic feature of sound properties derived entirely from said filtered signal, wherein said low-pass filtering step comprises filtering the intensity of said voice signal with a cut-off frequency (Fco) in the range of 150 to 400 Hz, and

processing said set of extracted feature(s) to detect an emotion therefrom.

2. Method according to claim 1, wherein said cut-off frequency (Fco) is in the range of 250 to 300 Hz.

3. Method according to claim 2, wherein said cut-off frequency (Fco) is equal to 250 Hz.

4. Method according to claim 1, wherein the feature extracting step comprises the steps of selectively extracting one or more among 160 features obtained from the following combination of parameters I, II, and III:

Parameters I;

i) the intensity of the signal derived after said low-pass filtering step, ii) the intensity of the signal without low-pass filtering, iii) the pitch of the signal without low-pass filtering, and iv) the pitch of the signal derived after said low-pass filtering step;

Parameters II, from a sequence of values of a parameter I obtained over a duration of said voice signal;

     1) a series of local minimas, whose members are values in said sequence that correspond to a minima,

     2) a series of local maximas, whose members are values in said sequence that correspond to a maxima,

     3) a series of durations (time intervals), whose members are time values separating local extremas, and

     4) a series, whose members are all the successive sampled values of the sequence; and

Parameters III, calculated from a series of statistical operations on parameter II;

     1) the mean,

     2) the maximum,

     3) the minimum,

     4) the difference between the maximum and the minimum,

     5) the variance,

     6) the median,

     7) the first quartile,

     8) the third quartile,

     9) the interquartile range, defined as the difference between the third and first quartiles,

     10) the mean of the absolute value of the local derivative;

at least one of said features originating from;

i) the intensity of the signal after said low-pass filtering step or from iv) the pitch of the signal after low-pass filtering step of parameters I.

5. Method according to claim 4, wherein said parameters I comprise:

i) the intensity of the signal derived after said low-pass filtering step, ii) the intensity of the signal without low-pass filtering, and iii) the pitch of the signal without low-pass filtering, whereby said combination yields a total of 120 different possible features;

at least one of said features originating from;

i) the intensity of the signal after said low-pass filtering step of parameters I.

6. Method according to claim 4, wherein said parameters III comprise the following five parameters:

2) the maximum,

     3) the minimum,

     6) the median,

     7) the first quartile,

     8) the third quartile, whereby said combination yields a total of 80 different possible features.

7. Method according to claim 6, wherein said extracting step comprises extracting at least one of the following possible features:

the maximum, the minimum, the median, the first quartile, the third quartile, calculated from;

the series of values of the intensity signal derived after said low-pass filtering of the voice signal, the series of minimum values of the intensity signal derived without low-pass filtering the voice signal, the series of pitch values of the signal derived after said low-pass filtering of the voice signal, and the series of pitch values of the signal derived without low-pass filtering of the voice signal.

8. Method according to claim 1, wherein said extracting step comprises extracting features derived from both said voice signal after said low-pass filtering step and from said voice signal without low-pass filtering.

9. Method according to claim 1, wherein said extracting step comprises extracting features only from said voice signal after said low-pass filtering step.

10. Method according to claim 1, wherein said at least one member of said set of features comprises:

the median value of signal intensity values derived after low-pass filtering the voice signal,

the mean value of the signal intensity values derived after low-pass filtering the voice signal,

third quartile value of the signal intensity values derived after low-pass filtering the voice signal,

the first quartile value of the signal intensity values derived after low-pass filtering the voice signal,

the maximum of the signal intensity values derived after low-pass filtering the voice signal, and

the maximum value of the series of signal intensity values derived after low-pass filtering the voice signal.

11. Method according to claim 1, further comprising a step of smoothing a signal extracted from said voice signal prior to said extracting step.

12. Method according to claim 1, further comprising a step of generating an emotion detecting algorithm using a teaching algorithm, said teaching algorithm exploiting said at least one feature extracted from a low-pass filtered voice signal.

13. Method according to claim 12, further comprising a step of programming (PU) an emotion detection device with an off-line emotion detection algorithm and/or an on-line emotion detection algorithm, whereby said algorithms exploit at least one feature extracted from a low-pass filtered voice signal.

14. A device for detecting an emotion conveyed in a voice signal, comprising:

low-pass filter means for filtering said voice signal, wherein said low-pass filtering means has a cut-off frequency (Fco) in the range of 150 to 400 Hz,

extracting means for extracting a set comprising at least one prosodic feature of sound properties derived entirely from said filtered signal,

processing means for processing said set of extracted feature(s) to detect an emotion therefrom.

15. Device according to claim 14, wherein said cut-off frequency (Fco) is in the range of 250 to 300 Hz.

16. Device according to claim 15, wherein said cut-off frequency (Fco) is equal to 250 Hz.

17. Device according to claim 14, wherein the feature extracting means comprises means for selectively extracting one or more among 160 features obtained from the following combination of parameters I, II, and III:

Parameters I;

i) the intensity of the signal derived after said low-pass filtering step, ii) the intensity of the signal without low-pass filtering, iii) the pitch of the signal without low-pass filtering, and iv) the pitch of the signal derived after said low-pass filtering step;

Parameters II, from a sequence of values of a parameter I obtained over a duration of said voice signal;

     1) a series of local minimas, whose members are values in said sequence that correspond to a minima,

     2) a series of local maximas, whose members are values in said sequence that correspond to a maxima,

     3) a series of durations (time intervals), whose members are time values separating local extremas, and

     4) a series, whose members are all the successive sampled values of the sequence; and

Parameters III, calculated from a series of statistical operations on parameter II;

     1) the mean,

     2) the maximum,

     3) the minimum,

     4) the difference between the maximum and the minimum,

     5) the variance,

     6) the median,

     7) the first quartile,

     8) the third quartile,

     9) the interquartile range, defined as the difference between the third and first quartiles,

     10) the mean of the absolute value of the local derivative;

at least one of said features originating from;

i) the intensity of the signal after said low-pass filtering step or from iv) the pitch of the signal after low-pass filtering step of parameters I.

18. Device according to claim 14, wherein said extracting means is set to extract features derived from both said voice signal after said low-pass filtering means and from said voice signal without low-pass filtering.

19. Device according to claim 14, wherein said extracting means are set to extract features only from said voice signal after said low-pass filtering means.

20. Device according to claim 14, wherein said at least one member of said set of features comprises:

the median value of signal intensity values derived after low-pass filtering the voice signal,

the mean value of the signal intensity values derived after low-pass filtering the voice signal,

third quartile value of the signal intensity values derived after low-pass filtering the voice signal,

the first quartile value of the signal intensity values derived after low-pass filtering the voice signal,

the maximum of the signal intensity values derived after low-pass filtering the voice signal, and

the maximum value of the series of signal intensity values derived after low-pass filtering the voice signal.

21. Device according to claim 14, further comprising means for smoothing a signal extracted from said voice signal prior to said extracting means.

22. Device according to claim 14, wherein said processing means comprises an off-line emotion recognition algorithm unit containing a preprogrammed algorithm.

23. Device according to claim 14, wherein said processing means comprises an on-line emotion recognition algorithm unit, said device further comprising a teaching algorithm unit for producing an emotion recognition algorithm in response to data produced said extracting means.

24. A voice signal processing device specifically for the device according to claim 14, low-pass filtering means for supplying a low-passed form of said voice signal to said extracting means.