CUSTOMIZABLE METHOD AND SYSTEM FOR EMOTIONAL RECOGNITION

• US 20100088088A1
• Filed: 01/31/2007
• Published: 04/08/2010
• Est. Priority Date: 01/31/2007
• Status: Active Grant

• Current Assignee: Telecom Italia S.p.A.
• Original Assignee: Telecom Italia S.p.A.
• Inventors: Esiliato, Antonio, Ettorre, Donato, Bollano, Gianmario
• Granted Patent: US 8,538,755 B2
• Time in Patent Office: Days
• US Class Current: 704/201
• CPC Class Codes: G10L 15/065 Adaptation
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices

Abstract

An automated emotional recognition system is adapted to determine emotional states of a speaker based on the analysis of a speech signal. The emotional recognition system includes at least one server function and at least one client function in communication with the at least one server function for receiving assistance in determining the emotional states of the speaker. The at least one client function includes an emotional features calculator adapted to receive the speech signal and to extract therefrom a set of speech features indicative of the emotional state of the speaker. The emotional state recognition system further includes at least one emotional state decider adapted to determine the emotional state of the speaker exploiting the set of speech features based on a decision model. The server function includes at least a decision model trainer adapted to update the selected decision model according to the speech signal. The decision model to be used by the emotional state decider for determining the emotional state of the speaker is selectable based on a context of use of the recognition system.

Claims

1-37. -37. (canceled)

38. An automated emotional recognition system for determining emotional states of a speaker based on analysis of a speech signal, comprising:

at least one server function;

at least one client function in communication with the at least one server function for receiving assistance in determining the emotional states of the speaker, wherein the at least one client function comprises an emotional features calculator capable of being adapted to receive the speech signal and to extract therefrom a set of speech features indicative of an emotional state of the speaker; and

at least one emotional state decider capable of being adapted to determine the emotional state of the speaker exploiting the set of speech features based on a decision model,

the server function comprising at least a decision model trainer capable of being adapted to update the decision model according to the speech features, and

the decision model to be used by the emotional state decider for determining the emotional state of the speaker being selectable based on a context of use of the recognition system.

39. The emotional recognition system of claim 38, wherein the extracted set of speech features extracted from the speech signal by the emotional features calculator to be used by the emotional state decider for determining the emotional state are configurable based on configuration data related to the context of use and capable of being adapted to define at least one among:

type of features to be extracted; and

precision of the extracted features.

40. The emotional recognition system of claim 38, wherein the emotional features calculator comprises at least one of:

a speech features extractor capable of being adapted to extract from the speech signal a plurality of preliminary sets of emotional features;

a super-frame speech features extractor capable of being adapted to process the plurality of preliminary sets of emotional features for obtaining a super-frame set of emotional features; and

a speech feature filter capable of being adapted to select from the super-frame set of emotional features a filtered subset of emotional features.

41. The emotional recognition system of claim 40, wherein the emotional features calculator further comprises a speech feature coder capable of being adapted to code the filtered subset of emotional features for obtaining the set of speech features and to provide the set of speech features to the at least one emotional state decider.

42. The emotional recognition system of claim 39, wherein said configuration data are provided by a scenario selector.

43. The emotional recognition system of claim 42, wherein the scenario selector is in the client function and wherein said configuration data are provided by a user of the client function through the scenario selector, and the client function comprises a configuration transmitter capable of being adapted to transmit the configuration data to the server function.

44. The emotional recognition system of claim 39, wherein the server function comprises a decision model selector capable of being adapted to select a decision model among a plurality of decision models, stored in a server model repository, according to the configuration data.

45. The emotional recognition system of claim 39, wherein the client function comprises a decision model selector capable of being adapted to select a decision model among a plurality of decision models stored in a client model repository.

46. The emotional recognition system of claim 45, wherein:

the client function comprises a speech features transmitter capable of being adapted to transmit the speech features to the decision model trainer;

the decision model trainer is capable of being adapted to update the server decision model according to the speech features and to generate corresponding model update data; and

the server function comprises a model update transmitter capable of being adapted to transmit the model update data to the client function to update the client decision model.

47. The emotional recognition system of claim 46, wherein the client function comprises a client model transmission block capable of being adapted to transmit the model update data to the server function to update the server decision model.

48. A method for the automatic emotional recognition of a speaker capable of being adapted to determine emotional states of the speaker based on analysis of a speech signal, comprising:

having a client function extract from the speech signal a set of speech features indicative of an emotional state;

using a decision model for performing emotional state decision operations on at least one emotional state decider, thereby determining the emotional state from the set of speech features; and

having a server function, in communication relationship with the client function, at least updating the decision model according to the speech features, the decision model to be used by the emotional state decider for determining the emotional state of the speaker being selectable based on a context of use of the recognition method.

49. The method of claim 48, wherein an extracted set of speech features extracted from the speech signal by an emotional features calculator to be used by the emotional state decider for determining the emotional state are configurable based on configuration data related to context of use and capable of being adapted to define at least one among:

type of features to be extracted; and

precision of the extracted features.

50. The method of claim 49, wherein extracting from the speech signal the set of speech features indicative of the emotional state, comprises the steps of:

extracting from the speech signal a plurality of preliminary sets of emotional features;

processing the plurality of preliminary sets of emotional features for obtaining a super frame set of emotional features;

selecting from the super frame set of emotional features a filtered subset of emotional features;

coding the filtered subset set of emotional features for obtaining the set of speech features; and

sending the set of speech features to the at least one emotional state decider.

51. The method of claim 50, further comprising the step of transmitting the configuration data to the server function.

52. The method of claim 51, wherein a number of emotional features selected from the super frame set is determined by the configuration data.

53. The method of claim 52, wherein the filtered subset set of emotional features is coded with a precision defined by the configuration data.

54. The method of claim 49, further comprising the step of providing the configuration data at the client function.

55. The method of claim 54, further comprising the step of transmitting the configuration data to the server function.

56. The method of claim 49, wherein the decision model is selected from a plurality of decision models according to the configuration data.

57. The method of claim 48, wherein determining the emotional state from the set of speech features is performed at the server function.

58. The method of claim 48, wherein determining the emotional state from the set of speech features is performed at the client function.

59. The method of claim 58, wherein an extracted set of speech features extracted from the speech signal by an emotional features calculator to be used by the emotional state decider for determining the emotional state are configurable based on configuration data related to context of use and capable of being adapted to define at least one among:

type of features to be extracted; and

precision of the extracted features; and

wherein the decision model is selected from a plurality of decision models according to the configuration data,

further comprising the steps of;

storing the selected decision model at the client function; and

storing at least one server decision model at the server function.

60. The method of claim 59, further comprising the steps of:

transmitting the speech features from the client function to the server function; and

updating the server decision model according to the speech features for generating corresponding model update data.

61. The method of claim 60, further comprising the step of transmitting the model update data to the client function.

62. The method of claim 61, further comprising the step of updating the selected decision model according to the model update data.