Smart home control method based on emotion recognition and the system thereof

• US 10,013,977 B2
• Filed: 01/06/2016
• Issued: 07/03/2018
• Est. Priority Date: 11/18/2015
• Status: Active Grant

• Current Assignee: Skyworth Electronics Incorporated (Skyworth Group Ltd.)
• Original Assignee: Skyworth Electronics Incorporated (Skyworth Group Ltd.)
• Inventors: Fu, Chunyuan
• Primary Examiner(s): Lerner, Martin
• Application Number:
  US15/328,299
• Publication Number:
  US 20170270922A1
• Time in Patent Office: 909 Days
• Field of Search: 704231, 704236, 704239, 704246, 704251, 704270, 704275
• CPC Class Codes: G05B 2219/2642 Domotique, domestic, home control, automation, smart house
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 17/06 Decision making techniques; Pattern matching strategies
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 25/63 for estimating an emotional state

Abstract

The present invention discloses a smart home control method based on emotion recognition and the system thereof, wherein, the method comprises: acquiring a user'"'"'s voice information before performing an emotion recognition for a speech tone on the voice information and generating a first emotion recognition result; after converting the voice information into a text information, performing an emotion recognition for a semantics of the text information before generating a second emotion recognition result; based on the first emotion recognition result and the second emotion recognition result, a user'"'"'s emotion recognition result is generated according to a preset determination method for emotion recognition result; also, based on the user'"'"'s emotion recognition result, each smart home device is controlled to perform a corresponding operation.

Claims

1. A smart home control method comprising:

acquiring a voice information from a user;

performing a first emotion recognition for a speech tone on the voice information and generating a first emotion recognition result;

converting the voice information into a text information;

performing a second emotion recognition for a semantics of the text information and generating a second emotion recognition result;

based on the first emotion recognition result and the second emotion recognition result, generating a user'"'"'s emotion recognition result according to a preset determination method for emotion recognition result; and

based on the user'"'"'s emotion recognition result, controlling a smart home device to perform a corresponding operation,

wherein the first emotion recognition result is a commendatory emotion, while the second emotion recognition result is a derogatory emotion or the first emotion recognition result is a derogatory emotion, while the second emotion recognition result is a commendatory emotion, recollecting the voice information of the user; and

redoing the first emotion recognition and the second emotion recognition for the voice information and generating a new first emotion recognition result and a new second emotion recognition result.

2. The smart home control method according to claim 1, wherein, the step of:

acquiring a voice information from a user;

performing a first emotion recognition for a speech tone on the voice information and generating a first emotion recognition result, comprises;

after obtaining the voice information, matching the speech tone of the voice information according to a Chinese emotional speech database to detect emotion variations, and generating the first emotion recognition result.

3. The smart home control method according to claim 1, after the step of:

based on the first emotion recognition result and the second emotion recognition result, generating a user'"'"'s emotion recognition result according to a preset determination method for emotion recognition result; and

based on the user'"'"'s emotion recognition result, controlling a smart home device to perform the corresponding operation, further comprising;

based on a preset database for speech features, matching the semantic feature of the user'"'"'s voice information to determine a user'"'"'s identity.

4. The smart home control method according to claim 1, wherein:

the first emotion recognition result comprises five levels of emotion types including a high-level commendatory emotion, a low-level commendatory emotion, a neutral emotion, a high-level derogatory emotion, and a low-level derogatory emotion; and

the emotion types included in the second emotion recognition result are the same as the emotion types included in the first emotion recognition result.

5. The smart home control method according to claim 4, wherein, the preset determination method for emotion recognition result comprises:

when the first emotion recognition result and the second emotion recognition result have different levels of commendatory emotion, determining a current user emotion recognition result as a low level commendatory emotion;

when the first emotion recognition result and the second emotion recognition result are different levels of derogatory emotion, determining the current user emotion recognition result as a low level derogatory emotion;

when one of the first emotion recognition result and the second emotion recognition result is a neutral emotion, while the other is the derogatory emotion or the commendatory emotion, determining the current user emotion recognition result as the commendatory emotion or the derogatory emotion.

6. A smart home control method comprising:

acquiring a voice information from a user;

performing a first emotion recognition for a speech tone of the voice information and generating a first emotion recognition result;

converting the voice information into a text information;

performing a second emotion recognition for a semantics of the text information and generating a second motion recognition result, comprising;

selecting a plurality of commendatory words as commendatory-seed-words and a plurality of derogatory words as derogatory-seed-words and generating an emotion dictionary;

respectively calculating a similarity between a plurality of words in the text information, the commendatory-seed-words and the derogatory-seed-words in the emotion dictionary; and

generating the second emotion recognition result through a preset emotion recognition method for semantics, according to a word similarity;

based on the first emotion recognition result and the second emotion recognition result, generating a user'"'"'s emotion recognition result according to a preset determination method for emotion recognition result;

based on the user'"'"'s emotion recognition result, controlling a smart home device to perform a corresponding operation;

wherein the first emotion recognition result is a commendatory emotion, while the second emotion recognition result is a derogatory emotion or the first emotion recognition result is a derogatory emotion, while the second emotion recognition result is a commendatory emotion, recollecting the voice information of the user; and

redoing the first emotion recognition and the second emotion recognition for the voice information and generating a new first emotion recognition result and a new second emotion recognition result.

7. The smart home control method according to claim 6, wherein, the step of respectively calculating a similarity between the plurality of words in the text information, the commendatory-seed-words and the derogatory-seed-words in the emotion dictionary, comprises:

based on a calculation method for semantic similarity, calculating respectively the word similarity between the plurality of words in the text information and the commendatory-seed-words, and the word similarity between the plurality of words in the text information and the derogatory-seed-words.

8. The smart home control method according to claim 7, wherein, the step of:

generating the second emotion recognition result through a preset emotion recognition method for semantics, according to the word similarity, comprises;

calculating a word emotion tendency value through a word emotion tendency calculation formula;

9. The smart home control method according to claim 6, wherein, the step of:

acquiring a voice information from a user;

performing a first emotion recognition for a speech tone on the voice information and generating a first emotion recognition result, comprises;

after acquiring the voice information of the user, matching the speech tone based on a Chinese emotional speech database to detect emotion variations and generating the first emotion recognition result.

10. The smart home control method according to claim 6, after the step of:

based on the first emotion recognition result and the second emotion recognition result, generating a user'"'"'s emotion recognition result according to a preset determination method for emotion recognition result; and

based on the user'"'"'s emotion recognition result, controlling a smart home device to perform the corresponding operation, further comprising;

based on a preset database for speech features, matching the semantic feature of the user'"'"'s voice information to determine an identity of the user.

11. The smart home control method according to claim 6, wherein:

the first emotion recognition result comprises five levels of emotion types including a high-level commendatory emotion, a low-level commendatory emotion, a neutral emotion, a high-level derogatory emotion and a low-level derogatory emotion; and

the emotion types included in the second emotion recognition result are the same as the emotion types included in the first emotion recognition result.

12. The smart home control method according to claim 11, wherein, the preset determination method for emotion recognition result comprises:

when the first emotion recognition result and the second emotion recognition result are different levels of commendatory emotion, determining a current user emotion recognition result as a low level commendatory emotion;

when the first emotion recognition result and the second emotion recognition result are different levels of derogatory emotion, determining the current user emotion recognition result as a low level derogatory emotion;

when one of the first emotion recognition result and the second emotion recognition result is a neutral emotion, and the other is the derogatory emotion or the commendatory emotion, determining the current user emotion recognition result as the commendatory emotion or the derogatory emotion.

13. A smart home control system comprising:

a first recognition and acquisition module, for acquiring a voice information from a user, performing a first emotion recognition for a speech tone on the voice information and generating a first emotion recognition result;

a second recognition and acquisition module, for converting the voice information into a text information, performing a second emotion recognition for a semantics of the text information and generating a second emotion recognition result;

a comprehensive emotion determination and control module, for generating a user'"'"'s user emotion recognition result according to a preset determination method for emotion recognition result, based on the first emotion recognition result and the second emotion recognition result, and controlling a smart home device to perform a corresponding operation according to the user'"'"'s emotion recognition result,

wherein the first emotion recognition result is a commendatory emotion, while the second emotion recognition result is a derogatory emotion or the first emotion recognition result is a derogatory emotion, while the second emotion recognition result is a commendatory emotion, recollecting the voice information of the user; and

redoing the first emotion recognition and the second emotion recognition for the voice information and generating a new first emotion recognition result and a new second emotion recognition result.