SENSING DEVICE OF EMOTION SIGNAL AND METHOD THEREOF

• US 20110137137A1
• Filed: 12/02/2010
• Published: 06/09/2011
• Est. Priority Date: 12/08/2009
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Kim, Ji-Hoon, Han, In-Tark, Jo, Jun, SHIN, Hyun-Soon, Lee, Yong-Kwi, Choi, Jun-Sik
• Granted Patent: US 8,764,656 B2
• Time in Patent Office: Days
• US Class Current: 600/301
• CPC Class Codes: A61B 5/0059 using light, e.g. diagnosis by transillumination, diascopy, fluorescence A61B5/0093, A61B5/1455, A61B5/02416 and A61B5/0261 take precedence
• CPC Class Codes: A61B 5/01 Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue clinical contact thermometers G01K13/20
• CPC Class Codes: A61B 5/02405 Determining heart rate variability
• CPC Class Codes: A61B 5/1123 Discriminating type of movement, e.g. walking or running A61B5/1116, A61B5/112 take precedence
• CPC Class Codes: A61B 5/14551 for measuring blood gases
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/441 Skin evaluation, e.g. for skin disorder diagnosis
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G09B 19/00 Teaching not covered by other main groups of this subclass teaching or practice apparatus for gun-aiming or gun-laying F41G3/26

Abstract

The present invention relates to a sensing device of an emotion signal and a method thereof capable of recognizing and analyzing the change in emotion by collecting at least one of the bio signals and peripheral environmental signals, guiding the state of emotion to a user, and sharing emotion information with authorized persons and communicating it between them. The sensing device of the emotion signal according to the embodiment of the present invention includes a sensing signal processor that senses a plurality of bio signals and peripheral environment signals of a user; an emotion signal processor that generates emotion signals representing the emotional state of the user for each of the sensed bio signals and collects the emotion signals to generate an emotion index; and an emotion signal communication unit that optionally transmits the emotion signals and the emotion index to external authorized devices.

Claims

1. A sensing device of an emotion signal, comprising:

a sensing signal processor that senses a plurality of bio signals and peripheral environment signals of a user;

an emotion signal processor that generates emotion signals representing the emotional state of the user for each of the sensed bio signals and collects the emotion signals to generate an emotion index; and

an emotion signal communication unit that optionally transmits the emotion signals and the emotion index to external authorized devices.

2. The sensing device of an emotion signal according to claim 1, wherein the emotion signal processor uses the emotion signals and the sensed peripheral environment signals to generate the emotion index.

3. The sensing device of an emotion signal according to claim 1, wherein the sensing signal processor includes at least one of a PPG sensor, a GSR sensor, a temperature/humidity sensor, an acceleration sensor, a voice sensor, and an infrared sensor for unrestricted and unresponsive sensing.

4. The sensing device of an emotion signal according to claim 1, wherein the emotion signal processor uses emotion changing threshold values that identify the emotion change of the user to extract the emotion signals from the bio signals.

5. The sensing device of an emotion signal according to claim 1, wherein the emotion signal processor recognizes errors for each of the emotion signals using the motion signals together with the peripheral environment signals to generate the error recognition information and uses the error recognition information to perform the error compensation for each of the emotion signals.

6. The sensing device of an emotion signal according to claim 1, wherein the emotion signal processor performs a multimodal convergence emotion inference using the emotion signals to determine the emotion index.

7. The sensing device of an emotion signal according to claim 1, wherein the emotion signal communication unit shares the emotion signals with an external device needing the emotion signals themselves and shares the emotion index with an external device needing the emotion index.

8. The sensing device of an emotion signal according to claim 1, wherein the emotion signal communication unit performs privacy security communication for privacy security for the emotional state of the user.

9. The sensing device of an emotion signal according to claim 1, wherein the sensing signal processor includes:

a sensor unit that has a plurality of sensors sensing the bio signals and the peripheral environment signals;

an amplifier that amplifies signals input from the plurality of sensors;

a noise filtering unit that removes external noises from the amplified signals;

an analog/digital I/F unit that converts the output signals from the noise filtering unit into the digital signal; and

a digital signal output unit that outputs the digital signals.

10. The sensing device of an emotion signal according to claim 1, wherein the emotion signal processor includes:

an emotion signal threshold generation and control unit that generates the emotion changing threshold values that identifies the change in emotion for the user and manages the change in the generated emotion changing threshold values;

an emotion signal extractor that extracts the emotion signals representing the emotional state of the user for each of the sensed bio signals by using the emotion changing threshold values;

an emotion signal error state recognizer that analyzes the peripheral environment signals and the motion signals to perform the error state recognition for each of the emotion signals;

an emotion signal compensator that compensates and optimizes the corresponding emotion signal according to the error state recognition;

an emotion index inference unit that performs the emotion inference on the emotion signals and the peripheral environment signals as an input to infer the emotion index; and

an emotion signal information managing unit that manages the emotion signals and the emotion index.

11. The sensing device of an emotion signal according to claim 1, wherein the emotion signal communication unit includes:

an emotion signal information formatting processor that formats the emotion signals and provides them to the peripheral devices and terminals;

an emotion signal security processor that performs the security processing for privacy security for the emotional state of the user;

an emotion signal communication protocol processor that processes protocols for the emotion signal communication; and

an emotion signal communication matching unit that matches wireless signals for the emotion signal communication.

12. The sensing device of an emotion signal according to claim 9, further comprising a H/W controller that provides a platform for executing the sensing signal processor, the emotion signal processor, and the emotion signal communication unit,

wherein the H/W controller includes:

a sensor analog interface unit that receives the analog signals from the plurality of sensors;

a main signal processor that in charge of the booting of the plurality of sensors and execution of the SW module;

a low power unit that provides a low power service; and

a wireless communication module that performs the emotion signal wireless communication.

13. A sensing method of an emotion signal, comprising:

sensing a plurality of bio signals and peripheral environment signals of a user by a sensing signal processor;

generating emotion signals representing emotional state of the user for each of the bio signals sensed by the emotion signal processor and collects the emotion signals to generate the emotion index; and

optionally providing the emotion signals and the emotion index to the external authorized devices by the emotion signal communication unit.

14. The sensing method of an emotion signal according to claim 13, wherein the sensing performs sensing using at least one of a PPG sensor, a GSR sensor, a temperature/humidity sensor, an acceleration sensor, a voice sensor, and an infrared sensor for unrestricted and unresponsive sensing.

15. The sensing method of an emotion signal according to claim 13, wherein the generating the emotion index generates the emotion index using the emotion signals and the sensed peripheral environment signals.

16. The sensing method of an emotion signal according to claim 13, wherein the generating the emotion index includes:

generating the emotion changing threshold values that identifies the change in emotion of the user; and

extracting the emotion signals from the bio signals using the emotion changing threshold values.

17. The sensing method of an emotion signal according to claim 16, wherein the generating the emotion index generates the error recognition information by recognizing the error state based on the peripheral environment signals and the motion signal and performs the error compensation on each of the emotion signals using the error recognition information.

18. The sensing method of an emotion signal according to claim 17, wherein the generating the emotion index determines the emotion index by performing the convergence emotion inference using the emotion signals in the finally compensated multimodal manner.

19. The sensing method of an emotion signal according to claim 13, wherein the providing to the external authorized devices shares the emotion signals when it is an external device needing the emotion signals and shares the emotion index when it is an external device needing the emotion index.

20. The sensing method of an emotion signal according to claim 13, wherein the providing to the external authorized devices performs privacy security management and privacy security communication for privacy security for the emotional state of the user.