Sensing device of emotion signal and method thereof

• US 8,764,656 B2
• Filed: 12/02/2010
• Issued: 07/01/2014
• Est. Priority Date: 12/08/2009
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Shin, Hyun-Soon, Lee, Yong-Kwi, Jo, Jun, Kim, Ji-Hoon, Choi, Jun-Sik, Han, In-Tark
• Primary Examiner(s): Jackson, Gary
• Assistant Examiner(s): JIAN, SHIRLEY XUEYING
• Application Number:
  US12/959,214
• Publication Number:
  US 20110137137A1
• Time in Patent Office: 1,307 Days
• Field of Search: None
• US Class Current: 600/301
• CPC Class Codes: A61B 5/0059 using light, e.g. diagnosis by transillumination, diascopy, fluorescence A61B5/0093, A61B5/1455, A61B5/02416 and A61B5/0261 take precedence
• CPC Class Codes: A61B 5/01 Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue clinical contact thermometers G01K13/20
• CPC Class Codes: A61B 5/02405 Determining heart rate variability
• CPC Class Codes: A61B 5/1123 Discriminating type of movement, e.g. walking or running A61B5/1116, A61B5/112 take precedence
• CPC Class Codes: A61B 5/14551 for measuring blood gases
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/441 Skin evaluation, e.g. for skin disorder diagnosis
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G09B 19/00 Teaching not covered by other main groups of this subclass teaching or practice apparatus for gun-aiming or gun-laying F41G3/26

Abstract

The present invention relates to a sensing device of an emotion signal and a method thereof capable of recognizing and analyzing the change in emotion by collecting at least one of the bio signals and peripheral environmental signals, guiding the state of emotion to a user, and sharing emotion information with authorized persons and communicating it between them. The sensing device of the emotion signal according to the embodiment of the present invention includes a sensing signal processor that senses a plurality of bio signals and peripheral environment signals of a user; an emotion signal processor that generates emotion signals representing the emotional state of the user for each of the sensed bio signals and collects the emotion signals to generate an emotion index; and an emotion signal communication unit that optionally transmits the emotion signals and the emotion index to external authorized devices.

Claims

1. An emotion sensing device, the device comprising:

a sensing signal processor configured to sense a plurality of bio signals of a user and peripheral environment signals of a user;

an emotion signal processor configured to generate an emotion signal associated with an emotional state of the user for each of the sensed bio signals, and collect the emotion signals to generate an emotion index; and

an emotion signal communication unit configured to transmit any of the emotion signals and the emotion index to an external authorized device,

wherein the emotion signal processor is configured to generate error recognition information for the emotion signals using the peripheral environment signals, and use the error recognition information to recognize an error for the emotion signals and perform error compensation.

2. The sensing device according to claim 1, wherein the emotion signal processor uses the emotion signals and the sensed peripheral environment signals to generate the emotion index.

3. The sensing device according to claim 1, wherein the sensing signal processor includes at least one of a PPG sensor, a GSR sensor, a temperature/humidity sensor, an acceleration sensor, a voice sensor, and an infrared sensor.

4. The sensing device according to claim 1, wherein the emotion signal processor is configured to use an emotion changing threshold value to identify an emotion change of the user, and generate an emotion signal corresponding to a sensed bio signal by using the emotion changing threshold value to extract the emotion signal from the bio signal.

5. The sensing device according to claim 1, wherein the emotion signal processor is configured to perform a multimodal convergence emotion inference using the emotion signals to determine the emotion index.

6. The sensing device according to claim 1, wherein the emotion signal communication unit is configured to perform a privacy security process before transmitting the emotion signals or the emotion index to the external authorized device.

7. The sensing device according to claim 1, wherein the sensing signal processor includes:

a sensor unit including a plurality of sensors configured to sense the bio signals and the peripheral environment signals;

an amplifier configured to amplify signals input from the plurality of sensors;

a noise filtering unit configured to remove external noise from the amplified signals;

an analog/digital I/F unit configured to convert the output signals from the noise filtering unit into digital signals; and

a digital signal output unit configured to output the digital signals.

8. The sensing device according to claim 7, further comprising a hardware (H/W) controller configured to provide a platform for the sensing signal processor, the emotion signal processor, and the emotion signal communication unit, and

wherein the H/W controller includes:

a sensor analog interface unit configured to receive analog signals from the plurality of sensors;

a main signal processor configured to control booting of a software (SW) module associated with the plurality of sensors and execution of the SW module;

a low power unit configured to provide a low power service; and

a wireless communication module configured to perform emotion signal wireless communication.

9. The sensing device according to claim 1, wherein the emotion signal processor includes:

an emotion signal threshold generation and control unit configured to generate a plurality of emotion changing threshold values to identify a change in emotion of the user, and manage a change in the generated emotion changing threshold values;

an emotion signal extractor configured to extract emotion signals corresponding to the sensed bio signals by using the emotion changing threshold values;

an emotion signal error state recognizer configured to analyze the peripheral environment signals and the emotion signals to perform error state recognition for each of the emotion signals;

an emotion signal compensator configured to compensate for the recognized error state and optimize the emotion signals according to the error state recognition;

an emotion index inference unit configured to perform emotion inference using the emotion signals and the peripheral environment signals as inputs to infer the emotion index; and

an emotion signal information managing unit configured to manage the emotion signals and the emotion index.

10. The sensing device according to claim 1, wherein the emotion signal communication unit includes:

an emotion signal information formatting processor configured to format the emotion signals and provide them to any of a peripheral device, an external authorized device, and an emotion service terminal;

an emotion signal security processor configured to perform a privacy security process for privacy security of the emotional state of the user;

an emotion signal communication protocol processor configured to process protocols for emotion signal communication; and

an emotion signal communication matching unit configured to match wireless signals with emotion signals for the emotion signal communication.

11. A method of sensing an emotion signal, the method comprising:

sensing a plurality of bio signals of a user and peripheral environment signals of a user by a sensing signal processor;

generating an emotion signal associated with an emotional state of the user for each of the bio signals sensed by the emotion signal processor and collecting the emotion signals to generate an emotion index; and

providing any of the emotion signals and the emotion index to an external authorized device by the emotion signal communication unit,

wherein generating the emotion index includes generating error recognition information to recognize an error state based on the peripheral environment signals and the emotion signals, and performing error compensation on each of the emotion signals using the error recognition information.

12. The sensing method according to claim 11, wherein the sensing is performed using at least one of a PPG sensor, a GSR sensor, a temperature/humidity sensor, an acceleration sensor, a voice sensor, and an infrared sensor.

13. The sensing method according to claim 11, wherein the emotion index is generated using the emotion signals and the sensed peripheral environment signals.

14. The sensing method according to claim 11, wherein generating the emotion index includes:

generating the emotion changing threshold values to identify a change in emotion of the user; and

extracting emotion signals from corresponding bio signals using the emotion changing threshold values.

15. The sensing method according to claim 11, wherein the emotion index is generated by performing convergence emotion inference using the compensated emotion signals in a multimodal manner.

16. The sensing method according to claim 11, further comprising performing privacy security management and privacy security communication for privacy security for the emotional state of the user before providing the emotion signals or the emotion index to the external authorized device.