Smart home appliances, operating method of thereof, and voice recognition system using the smart home appliances

• US 10,269,344 B2
• Filed: 11/04/2014
• Issued: 04/23/2019
• Est. Priority Date: 12/11/2013
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Lee, Juwan, Kim, Lagyoung, Shin, Taedong, Seo, Daegeun
• Primary Examiner(s): Yen, Eric
• Application Number:
  US15/103,528
• Publication Number:
  US 20170004828A1
• Time in Patent Office: 1,631 Days
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G10L 13/027 Concept to speech synthesisers; Generation of natural phrases from machine-based concepts generation of parameters for speech synthesis out of text G10L13/08
• CPC Class Codes: G10L 13/033 Voice editing, e.g. manipulating the voice of the synthesiser
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 15/25 using position of the lips, movement of the lips or face analysis
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems
• CPC Class Codes: G10L 17/24 the user being prompted to utter a password or a predefined phrase
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/226 using non-speech characteristics
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: G10L 25/63 for estimating an emotional state
• CPC Class Codes: G10L 25/84 : for discriminating voice from noise
• CPC Class Codes: H04L 12/2803 : Home automation networks
• CPC Class Codes: H04L 12/282 : based on user interaction within the home receiver circuitry for displaying additional information being controlled by a remote control apparatus H04N21/42204

Abstract

Provided is a smart home appliance. The smart home appliance includes: a voice input unit collecting a voice; a voice recognition unit recognizing a text corresponding to the voice collected through the voice input unit; a capturing unit collecting an image for detecting a user'"'"'s visage or face; a memory unit mapping the text recognized by the voice recognition unit and a setting function and storing the mapped information; and a control unit determining whether to perform a voice recognition service on the basis of at least one information of image information collected by the capturing unit and voice information collected by the voice input unit.

Claims

1. A smart home appliance comprising:

a voice input unit collecting a voice of a user, the voice input unit comprising a microphone;

a voice recognition unit recognizing a text corresponding to the voice collected through the voice input unit;

a capturing unit collecting an image for detecting a visage of the user, the capturing unit comprising a camera;

a memory unit mapping the text recognized by the voice recognition unit and a setting function and storing the mapped information, and storing a keyword information that the user may input to start a voice recognition service;

a control unit determining whether to perform a voice recognition service on the basis of at least one information of image information collected by the capturing unit and voice information collected by the voice input unit;

a filter unit removing a noise sound from the voice inputted through the voice input unit; and

a memory unit mapping voice information related to an operation of the smart home appliance and voice information unrelated to an operation of the smart home appliance in the voice inputted through the voice input unit and storing the mapped information;

wherein the control unit comprises a face detection unit recognizing that the user is in a staring state for voice input when image information on the user'"'"'s visage is collected for more than a setting time through the capturing unit, and

wherein the control unit determines that a voice recognition service standby state is entered when it is recognized that there is keyword information in the user'"'"'s voice collected through the voice input unit and the user is in the staring state through the face detection unit.

2. The smart home appliance according to claim 1, further comprising:

a region recognition unit determining a region of the user from among a plurality of regions of the user on the basis of information on the user'"'"'s voice collected through the voice input unit; and

an output unit outputting region customized information on the basis of information on the region determined by the region recognition unit and information on the setting function.

3. The smart home appliance according to claim 2, wherein the setting function comprises a plurality of functions divided according to the plurality of regions;

and

the region customized information including one function matching the region information determined by the region recognition unit from among the plurality of functions is outputted through the output unit.

4. The smart home appliance according to claim 2, wherein the output unit outputs the region customized information by using a dialect in the region determined by the region recognition unit.

5. The smart home appliance according to claim 2, wherein the output unit outputs a key word for security setting and the voice input unit sets a reply word corresponding to the key word.

6. The smart home appliance according to claim 1, further comprising an emotion recognition unit and an output unit, wherein

the voice recognition unit recognizes a text corresponding to first voice information in the user'"'"'s voice collected through the voice input unit;

the emotion recognition unit extracts an emotion of the user on the basis of second voice information in the user'"'"'s voice collected through the voice input unit; and

the output unit outputs user customized information on information on the user'"'"'s emotion determined by the emotion recognition unit and information on the setting function.

7. The smart home appliance according to claim 6, wherein the first voice information comprises a language element in the user'"'"'s voice collected through the voice input unit;

and

the second voice information comprises a non-language element related to the user'"'"'s emotion.

8. The smart home appliance according to claim 6, wherein

the emotion recognition unit comprises a database where information on voice characteristics of the user and information on an emotion state are mapped;

and

the information on the user'"'"'s voice characteristics comprises information on a speech spectrum having characteristics for the user'"'"'s emotion.

9. The smart home appliance according to claim 6, wherein the setting function comprises a plurality of functions to be recommended or selected;

and

the user customized information including one function matching the emotion information determined by the emotion recognition unit from among the plurality of functions is outputted through the output unit.

10. The smart home appliance according to claim 1, further comprising:

a position information recognition unit recognizing position information; and

an output unit outputting information on the setting function on the basis of position information recognized by the position information recognition unit.

11. The smart home appliance according to claim 10, wherein the position information recognition unit comprises:

a GPS reception unit receiving a position coordinate from a position information transmission unit; and

a first communication module communicably connected to a second communication module equipped in a server.

12. The smart home appliance according to claim 10, wherein the output unit comprises a voice output unit outputting the information on the setting function as a voice, by using the position information or a dialect.

13. The smart home appliance according to claim 10, wherein the output unit outputs information optimized for a region recognized by the position information recognition unit among a plurality of information on the setting function.

14. The smart home appliance according to claim 10, wherein the position information recognized by the position information recognition unit comprises weather information.

15. An operating method of the smart home appliance according to claim 1, the method comprising:

collecting the user'"'"'s voice through the voice input unit;

recognizing whether keyword information is included in the collected voice;

collecting image information on the user'"'"'s visage through the capturing unit; and

entering the standby state of the voice recognition service on the basis of the image information on the user'"'"'s visage.

16. The operating method according to claim 15, further comprising performing a security setting,

wherein the performing of the security setting comprises:

outputting a predetermined key word; and

inputting a reply word in response to the outputted key word.

17. The method according to claim 15, further comprising:

extracting an emotion state of the user on the basis of information on the user'"'"'s voice collected through the voice input unit; and

recommending an operation mode on the basis of information on the user'"'"'s emotion state.

18. The method according to claim 15, further comprising:

recognizing an installation position of the smart home appliance through a position information recognition unit; and

driving the smart home appliance on the basis of information on the installation position.

19. The method according to claim 18, wherein the recognizing of the installation position of the smart home appliance comprises receiving GPS coordinate information from a GPS satellite or a communication base station.

20. The method according to claim 18, wherein the recognizing of the installation position of the smart home appliance comprises checking a communication address as a first communication module equipped in the smart home appliance is connected to a second communication module equipped in a server.

21. A voice recognition system using

a mobile device comprising a voice input unit having a microphone for collecting a voice of a user and a mobile device communication module;

and

a smart home appliance comprising;

a voice recognition unit recognizing a text corresponding to the voice collected through the voice input unit;

a capturing unit collecting an image for detecting a visage of the user, the capturing unit comprising a camera;

a memory unit mapping the text recognized by the voice recognition unit and a setting function and storing the mapped information, and storing a keyword information that the user may input to start a voice recognition service;

a filter unit removing a noise sound from the voice inputted through the voice input unit; and

a memory unit mapping voice information related to an operation of the smart home appliance and voice information unrelated to an operation of the smart home appliance in the voice inputted through the voice input unit and storing the mapped information; and

a control unit comprising a face detection unit recognizing that the user is in a staring state for voice input when image information on the user'"'"'s visage is collected for more than a setting time through the capturing unit; and

a smart home appliance communication module,

wherein the control unit determines that a voice recognition service standby state is entered when it is recognized that there is keyword information in the user'"'"'s voice collected through the voice input unit and the user is in the staring state through the face detection unit,

wherein the smart home appliance operates based on the user'"'"'s voice collected through the voice input unit and the respective communication module equipped in each of the mobile device and the smart home appliance, and

wherein the mobile device further comprises a movement detection unit determining whether to enter the standby state of the voice recognition service in the smart home appliance by detecting a movement of the mobile device.

22. The voice recognition system according to claim 21, wherein the movement detection unit comprises an acceleration sensor or a gyro sensor detecting a change in an inclined angle of the mobile device, wherein

the voice input unit is disposed at a lower part of the mobile device.

23. The voice recognition system according to claim 21, wherein the movement detection unit comprises an illumination sensor detecting an intensity of an external light collected by the mobile device.