EMOTION RECOGNITION APPARATUS AND METHOD

• US 20130018837A1
• Filed: 11/15/2011
• Published: 01/17/2013
• Est. Priority Date: 07/14/2011
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Lee, Ho-Sub
• Granted Patent: US 8,781,991 B2
• Time in Patent Office: Days
• US Class Current: 706/52
• CPC Class Codes: A61B 2562/0219 Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/7267 involving training the classification device

Abstract

An emotion recognition apparatus and method are provided. The emotion recognition apparatus acquires a first emotion factor and a second emotion factor of an emotion model. An emotional state of a user is estimated based on the first emotion factor and the second emotion factor. The emotion recognition apparatus may also acquire a third emotion factor of the emotion model.

Claims

1. An emotion recognition apparatus, comprising:

a data collection unit to collect sensing data from a terminal;

a first emotion value acquisition unit to acquire a first emotion value corresponding to a first axis of a multidimensional emotion model based on an amount or an intensity of the sensing data;

a second emotion value acquisition unit to acquire a second emotion value corresponding to a second axis of the multidimensional emotion model based on a meaning of the sensing data; and

an emotion estimation unit to estimate an emotional state of a user based on the first emotion value and the second emotion value.

2. The emotion recognition apparatus of claim 1, wherein the first emotion value acquisition unit further acquires first-level data relating to the user'"'"'s level of arousal by analyzing the amount or the intensity of the sensing data, and acquires the first emotion value based on the first-level data.

3. The emotion recognition apparatus of claim 1, wherein the first emotion value acquisition unit further calculates a touch activeness (TA) level by analyzing the amount or the intensity of sensing data relating to one or more touch inputs detected from the terminal, and calculates the first emotion value based on the calculated TA level.

4. The emotion recognition apparatus of claim 1, wherein the first emotion value acquisition unit further calculates a device movement (DM) level by analyzing the amount or the intensity of sensing data relating to the movement of the terminal, and calculates the first emotion value based on the calculated DM level.

5. The emotion recognition apparatus of claim 1, wherein the first emotion value acquisition unit further calculates a TA level by analyzing the amount or the intensity of sensing data relating to one or more touch inputs detected from the terminal, calculates a DM level by analyzing the amount or the intensity of sensing data relating to the movement of the terminal, and calculates the first emotion value based on a weighted sum of the calculated TA level and the calculated DM level.

6. The emotion recognition apparatus of claim 1, wherein the second emotion value acquisition unit further acquires second-level data relating to valence of the user by analyzing the meaning of the sensing data, and acquires the second emotion value based on the second-level data.

7. The emotion recognition apparatus of claim 1, wherein the second emotion value acquisition unit further calculates an input text (IT) level by analyzing the meaning of sensing data relating to a user input that is entered to the terminal, and calculates the second emotion value based on the calculated IT level.

8. The emotion recognition apparatus of claim 7, wherein the user input includes at least one selected from the group of text data, audio data, video data, an emoticon, and the like.

9. The emotion recognition apparatus of claim 1, wherein the second emotion value acquisition unit further calculates a user context (UC) level by analyzing the meaning of sensing data relating to a context of the user, and calculates the second emotion value based on the calculated UC level.

10. The emotion recognition apparatus of claim 9, wherein the context includes at least one selected from the group of a location of the user, a schedule of the user, and the like.

11. The emotion recognition apparatus of claim 1, wherein the second emotion value acquisition unit further calculates an IT level by analyzing the meaning of sensing data relating to a user input that is entered to the terminal, calculates a UC level by analyzing the meaning of sensing data relating to context of the user, and calculates the second emotion value based on a weighted sum of the calculated IT level and the calculated UC level,

wherein the user input includes at least one selected from the group of text data, audio data, video data, an emoticon, and the like;

and

wherein the context includes at least one selected from the group of a location of the user, a schedule of the user, and the like.

30. The emotion recognition apparatus of claim 1, wherein the first emotion value and the second emotion value are values of Russell'"'"'s emotion model.

31. The emotion recognition apparatus of claim 1, wherein the first emotion value and the second emotion value are values of Watson-Tellegen'"'"'s emotion model.

32. The emotion recognition apparatus of claim 3, wherein the TA level is calculated with respect to a period of time for which a touch input is detected, touch counts detected during the period of time, a touch movement distance during the period of time, a heavy touch movement count during the period of time, and a number of characters entered during the period of time,

wherein the heavy touch movement count is calculated according to a number of substantially continuous touches that include more than a predetermined number of changes in direction.

33. The emotion recognition apparatus of claim 4, wherein the DM level is calculated with respect to a period of time for which a touch input is detected, a value representing x-axis acceleration of the terminal, a value representing y-axis acceleration of the terminal, and a value representing z-axis acceleration of the terminal.

34. The emotion recognition apparatus of claim 7, wherein the IT level is calculated with respect to a period of time for which a touch input is detected, a ratio of positive user inputs to all user inputs that are entered during the period of time, and a ratio of negative user inputs to all the user inputs that are entered during the period of time.

35. The emotion recognition apparatus of claim 9, wherein the UC level is calculated with respect to an elapsed time for a touch input, a duration for which the user remains at a location, a value representing positive or negative context of the location, and a value representing positive or negative context of a schedule of the user.

12. An emotion recognition apparatus, comprising:

a data collection unit to collect sensing data from a terminal;

a first emotion value acquisition unit to acquire a first emotion value corresponding to a first axis of a multidimensional emotion model based on an amount or an intensity of the sensing data;

a second emotion value acquisition unit to acquire a second emotion value corresponding to a second axis of the multidimensional emotion model based on a meaning of the sensing data;

a third emotion value acquisition unit to acquire a third emotion value corresponding to a user'"'"'s personality trait; and

an emotion estimation unit to estimate an emotional state of the user based on at least two selected from the group of the first emotion value, the second emotion value, and the third emotion value.

13. The emotion recognition apparatus of claim 12, wherein the first emotion value acquisition unit further acquires first-level data relating to the user'"'"'s level of arousal by analyzing the amount or the intensity of the sensing data, and acquires the first emotion value based on the first-level data.

14. The emotion recognition apparatus of claim 12, wherein the first emotion value acquisition unit further calculates a TA level by analyzing the amount or the intensity of sensing data relating to one or more touch inputs detected from the terminal, and calculates the first emotion value based on the calculated TA level.

15. The emotion recognition apparatus of claim 12, wherein the first emotion value acquisition unit further calculates a DM level by analyzing the amount or the intensity of sensing data relating to the movement of the terminal, and calculates the first emotion value based on the calculated DM level.

16. The emotion recognition apparatus of claim 12, wherein the first emotion value acquisition unit further calculates a TA level by analyzing the amount or the intensity of sensing data relating to one or more touch inputs detected from the terminal, and calculates a DM level by analyzing the amount or the intensity of sensing data relating to the movement of the terminal, and calculates the first emotion value based on a weighted sum of the calculated TA level and the calculated DM level.

17. The emotion recognition apparatus of claim 12, wherein the second emotion value acquisition unit further acquires second-level data relating to valence of the user by analyzing the meaning of the sensing data, and acquires the second emotion value based on the second-level data.

18. The emotion recognition apparatus of claim 12, wherein the second emotion value acquisition unit further calculates an IT level by analyzing the meaning of sensing data relating to a user input that is entered to the terminal, and calculates the second emotion value based on the calculated IT level.

19. The emotion recognition apparatus of claim 18, wherein the user input includes at least one selected from the group of text data, audio data, video data, an emoticon, and the like.

20. The emotion recognition apparatus of claim 12, wherein the second emotion value acquisition unit further calculates a UC level by analyzing the meaning of sensing data relating to a context of the user, and calculates the second emotion value based on the calculated UC level.

21. The emotion recognition apparatus of claim 20, wherein the context includes at least one selected from the group of a location of the user, a schedule of the user, and the like.

22. The emotion recognition apparatus of claim 12, wherein the second emotion value acquisition unit further calculates an IT level by analyzing the meaning of sensing data relating to a user input that is entered to the terminal, calculates a UC level by analyzing the meaning of sensing data relating to context of the user, and calculates the second emotion value based on a weighted sum of the calculated IT level and the calculated UC level,

wherein the user input includes at least one selected from the group of text data, audio data, video data, an emoticon, and the like;

and

wherein the context includes at least one selected from the group of a location of the user, a schedule of the user, and the like.

23. The emotion recognition apparatus of claim 12, wherein the emotion estimation unit further estimates an internal emotional state of the user based on the first and second emotion values, and estimates an external emotional state of the user based on the first, second, and third emotion values.

24. The emotion recognition apparatus of claim 23, further comprising:

an emotion provision unit to output data regarding the estimated internal emotional state and data regarding the estimated external emotional state of the user.

25. The emotion recognition apparatus of claim 24, wherein the emotion provision unit further outputs the data regarding the estimated internal emotional state of the user to a user interface of the terminal, and outputs the data regarding the estimated external emotional state of the user to an external device.

26. An emotion recognition method, comprising:

collecting sensing data from a terminal;

acquiring a first emotion value corresponding to a first axis of a multidimensional emotion model based on an amount or an intensity of the sensing data;

acquiring a second emotion value corresponding to a second axis of the multidimensional emotion model based on a meaning of the sensing data; and

estimating an emotional state of a user based on the first emotion value and the second emotion value.

27. An emotion recognition method, comprising:

collecting sensing data from a terminal;

acquiring a first emotion value corresponding to a first axis of a multidimensional emotion model based on an amount or an intensity of the sensing data;

acquiring a second emotion value corresponding to a second axis of the multidimensional emotion model based on a meaning of the sensing data;

acquiring a third emotion value corresponding to a user'"'"'s personality trait; and

estimating an emotional state of the user based on at least two selected from the group of the first emotion value, the second emotion value, and the third emotion value.

28. The emotion recognition method of claim 27, wherein the estimating the emotional state of the user comprises:

estimating an internal emotional state of the user based on the first and second emotion values; and

estimating an external emotional state of the user based on the first, second, and third emotion values.

29. The emotion recognition method of claim 28, further comprising:

outputting the estimated internal emotional state and the estimated external emotional state of the user,

wherein the outputting of the estimated internal emotional state and the estimated external emotional state of the user comprises outputting the estimated internal emotional state of the user to a user interface of the terminal, and outputting the estimated external emotional state of the user to an external device.