Method for detecting emotions involving subspace specialists

• US 20030069728A1
• Filed: 10/04/2002
• Published: 04/10/2003
• Est. Priority Date: 10/05/2001
• Status: Active Grant

• Current Assignee: Sony Deutschland GmbH (Sony Group Corp.)
• Original Assignee: Sony Deutschland GmbH (Sony Group Corp.)
• Inventors: Kemp, Thomas, Marasek, Krzysztof, Tato, Raquel
• Granted Patent: US 7,729,914 B2
• Time in Patent Office: Days
• US Class Current: 704/231
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices

Abstract

To detect and determine a current emotional state (CES) of a human being from a spoken speech input (SI), it is suggested in a method for detecting emotions to identify first and second feature classes (A, E) with, in particular distinct, dimensions of an underlying emotional manifold (EM) or emotional space (ES) and/or with subspaces thereof.

Claims

1. Method for detecting emotions from speech input,

wherein at least the first feature class (A) and a second feature class (E) of features are at least in part evaluated, derived and/or extracted from a given speech input (SI),

wherein from said features a current emotional state (CES) of the current speaker and/or parameters (CFS) thereof are derived and

wherein said first and second feature classes (A, E) are identified and/or associated with, in particular distinct, dimensions of an underlying emotional manifold (EM) or emotional space (ES) and/or with subspaces thereof, in particular with activation or arousal and evaluation or pleasure, respectively.

2. Method according to claim 1, wherein for each feature class (A, E), for each dimension/subspace of said emotional manifold (EM) and/or for groups thereof a separate and/or distinct specialist—

in particular a subspace specialist or the like—

or a specialized classifier system (CA, CE) is used and/or constructed, each of which in particular being adapted to classify or obtain features essentially from an assigned feature class (A, E) and/or an assigned dimension or subspace, in particular into or for a current emotional state (CES) of the current speaker and/or parameters thereof.

3. Method according to anyone of the preceding claims, wherein the distinct specialized classifier systems (CA, CE) are applied to the different feature classes (A, E) already extracted from said speech input (SI) and/or to said speech input (SI) directly and wherein thereby evaluated, derived and/or extracted features, emotions, and/or parameters thereof, in particular from different feature subspaces, are collected and/or stored, In particular in order to obtain a final classification by combination of the results.

4. Method according to claim 3, wherein the features, emotions, and/or parameters thereof evaluated, derived and/or extracted, in particular from different feature subspaces, are combined, in particular to describe a current emotional state (CES) of the current speaker and/or parameters thereof.

5. Method according to anyone of the preceding claims, wherein the distinct specialized classifier systems (CA, CE) for each feature class (A, E) and/or of the features or outputs thereofare combined, fused or merged, in particular to form a global classifier system and/or—

in the case of non-orthogonal subspaces or classifier systems (CA, CE)—

in particular by means of a empirical weighting algorithm or the like.

6. Method according to anyone of the preceding claims, wherein a class of prosodic features is used at least as a part of said first feature class (A), which is in particular at least in part identified with an activation and/or arousal dimension of emotions or of the emotional manifold (EM).

7. Method according to anyone of the preceding claims, wherein a class of speech or voice quality features is used at least as a part of said second feature class (E), which is in particular at least in part identified with a pleasure and/or evaluation dimension of emotions or of the emotional manifold (EM).

8. Method according to anyone of the claims 2 to 7, wherein a—

in particular unidimensional—

classifier system (CA) of low complexity is used, in particular as a classifier system (CA) for prosodic features.

9. Method according to anyone of the claims 2 to 7, wherein a—

in particular unidimensional—

classifier system (CE) of high complexity is used, in particular as a classifier system (CE) for speech and/or voice quality features.

12. Method according to anyone of the preceding claims, wherein pitch, pitch range, Intonation attitude, loudness, speaking rate, phone duration, speech element duration features and/or the like are used as prosodic features.

13. Method according to anyone of the preceding claims, wherein phonation type, articulation manner, voice timbre features, spectral tilt, amplitude difference between harmonics and formants, formants bandwidth, jitter, harmonic-to-noise-ratio features, and/or the like are used as speech and/or voice quality features.

14. System for detecting emotions from speech input which is capable of performing and/or realizing a method for detecting emotions according to anyone of the claims 1 to 13 and/or the steps thereof.

15. Computer program product, comprising computer program means adapted to perform and/or to realize a method for detecting emotions according to anyone of the claims 1 to 13 and/or the steps thereof when it is executed on a computer, a digital signal processing means and/or the like.

10. Method according to claim 10, wherein said classifier system (CE) of high complexity includes a multiplicity of classifiers, in particular by implementing speaker dependencies, such as age, gender and/or the like.

11. Method according to anyone of the claims 8 to 10, wherein the different classifiers can give as an output not only proper emotions, but also a degree of emotions, in particular a degree of pleasure and/or acitvation, depending on the feature subspacers they have as an input, which can be combined afterwards to obtain a current emotional state (CES) of the speaker.