System and method for inspecting emotion recognition capability using multisensory information, and system and method for training emotion recognition using multisensory information

• US 9,934,426 B2
• Filed: 09/26/2014
• Issued: 04/03/2018
• Est. Priority Date: 09/27/2013
• Status: Active Grant

• Current Assignee: Korea University Research and Business Foundation (Korea University)
• Original Assignee: Korea University Research and Business Foundation (Korea University)
• Inventors: Kim, Hyun Teak, Jeong, Ji Woon, Kim, Jiyoung, Chun, Yeseul
• Primary Examiner(s): Woldemariam, Aklilu
• Application Number:
  US15/024,453
• Publication Number:
  US 20160217322A1
• Time in Patent Office: 1,285 Days
• Field of Search: 382103, 382118, 382155, 382159, 382190, 600301, 600500
• CPC Class Codes: G06V 40/175 Static expression
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 25/63 for estimating an emotional state
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16Z 99/00 Subject matter not provided for in other main groups of this subclass

Abstract

The present invention relates to a system and a method for inspecting an emotion recognition capability using multisensory information, and a system and a method for training emotion recognition using multisensory information. More particularly, the present invention comprises: an output unit for outputting multisensory information including at least one emotional state of a subject person to the outside; an input unit for receiving, from a tested person, emotional state information, which indicates whether the output multisensory information is identical to at least one emotional state of the subject person, on the basis of the output multisensory information; a comparison identification unit for identifying whether the received emotional state information is identical to reference emotional state information corresponding to the multisensory information pre-stored in a storage unit; and a control unit for determining an emotion recognition capability of the tested person according to a result of the identification of the received emotional state information. Due to such a configuration, a system and a method for inspecting an emotion recognition capability using multisensory information and a system and a method for training emotion recognition using multisensory information according to the present invention can determine an emotional state of a subject person by using multisensory information and thus can easily identify an emotion recognition capability of other people.

Claims

1. A system for training emotion recognition using multisensory information, comprising:

an output unit to output multisensory information made up of at least one emotional state of a subject;

an input unit to receive an input of emotional state information from a test person, the emotional state information indicating whether the at least one emotional state of the subject is the same based on the outputted multisensory information;

a comparison identification unit to determine whether the inputted emotional state information is identical to reference emotional state information corresponding to the multisensory information pre-stored in a storage unit; and

a control unit to judge an emotion recognition capability of the test person based on a result of determining the inputted emotional state information, and generate an output control signal to output multisensory information having a change in difficulty level of emotional state combination based on a result of the judgment and transmit the output control signal to the output unit,

wherein difficulty level of emotional state combination is adjusted according to composite ratio of composed emotions when multisensory information is made up of a combination of a plurality of emotional state information,

the output unit re-outputs multisensory information having a change in difficulty level of emotional state combination based on the result of judging the emotion recognition capability in response to the output control signal, and

the control unit compares the result of judging the emotion recognition capability of the test person to a result of judging an average emotion recognition capability to determine whether the emotion recognition capability of the test person is improved.

2. The system for training emotion recognition using multisensory information according to claim 1, wherein the control unit further comprises:

a difficulty level change module to change a difficulty level of emotional state combination of the multisensory information based on the result of judging the emotion recognition capability of the test person;

an order change module to change an output order of the multisensory information based on the result of judging the emotion recognition capability of the test person; and

a transmission module to generate an output control signal indicating a result of changing the difficulty level of emotional state combination or the output order of the multisensory information, and transmit the output control signal to the output unit.

3. The system for training emotion recognition using multisensory information according to claim 1, wherein the multisensory information is information made up of a combination of visual information showing the subject and auditory information representing a voice of the subject.

6. The system according to claim 1, wherein the output unit is configured to provide visual and/or auditory information.

7. The system according to claim 6, wherein the output unit comprises a monitor or speaker.

8. The system according to claim 1, wherein the input unit is configured to receive visual and/or auditory information.

9. The system according to claim 8, wherein the input unit comprises a camera or microphone.

10. The system according to claim 1, wherein the storage unit comprises a computer-readable recording medium.

11. The system according to claim 10, wherein the computer-readable recording medium is ROM, RAM, CD-ROM, DVD±

ROM, DVD-RAM, a magnetic tape, a floppy disk, a hard disk, or an optical data storage device.

4. A method for training emotion recognition using multisensory information, comprising:

outputting, by an output unit, multisensory information made up of at least one emotional state of a subject;

receiving, by an input unit, an input of emotional state information from a test person, the emotional state information indicating whether the at least one emotional state of the subject is the same based on the outputted multisensory information;

determining, by a comparison identification unit, whether the inputted emotional state information is identical to reference emotional state information corresponding to the multisensory information pre-stored in a storage unit;

judging, by a control unit, an emotion recognition capability of the test person based on a result of determining the emotional state information;

generating, by the control unit, an output control signal to output multisensory information having a change in difficult level of emotional state combination corresponding to a result of judging the emotion recognition capability of the test person, and transmitting the generated output control signal to the output unit;

re-outputting, by the output unit, multisensory information having a change in difficulty level of emotional state combination based on the received output control signal;

receiving, by the input unit, an input of emotional state information of the subject from the test person based on the re-outputted multisensory information;

re-determining, by the comparison identification unit, whether the inputted emotional state information is identical to pre-stored reference emotional state information corresponding to the multisensory information; and

comparing, by the control unit, the result of judging the emotion recognition capability of the test person to a result of judging an average emotion recognition capability pre-stored in the storage unit based on a result of re-determining the emotional state information, to determine whether the emotion recognition capability of the test person is improved.

5. The method for training emotion recognition using multisensory information according to claim 4, wherein the generating and transmitting of an output control signal to the output unit by the control unit comprises generating, by the control unit, an output control signal indicating a result of changing the difficulty level of emotional state combination of the multisensory information or an output order of the multisensory information based on a result of judging the emotion recognition capability of the test person, and transmitting the output control signal to the output unit.