Electronic device and method for controlling the same

• US 10,811,002 B2
• Filed: 11/02/2016
• Issued: 10/20/2020
• Est. Priority Date: 11/10/2015
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Cho, Chi-Hyun, Son, Dong-Il, Kim, Kyung-Tae, Lee, Ji-Min, Heo, Chang-Ryong, Chun, Jae-Bong, Choi, Gyu-Cheol, Kang, Chang-Taek, Lee, Chang-Soo
• Primary Examiner(s): Jackson, Jakieda R
• Application Number:
  US15/341,454
• Publication Number:
  US 20170133009A1
• Time in Patent Office: 1,448 Days
• Field of Search: 704275
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 25/63 for estimating an emotional state

Abstract

An electronic device and a method for controlling the electronic device are disclosed. The electronic device includes a memory and a processor electrically connected with the memory. The memory stores instructions to be executed by the processor. The processor is configured to obtain a user voice, determine a response level of the user voice, when the response level of the user voice is a first response level, perform an operation corresponding to the user voice based on a time point of obtaining the user voice, and when the response level of the user voice is a second response level, perform the operation corresponding to the user voice based on a time point of detecting an additional event.

Claims

1. An electronic device comprising:

a display;

a microphone;

a memory; and

at least one processor electrically connected with the memory, the display and the microphone,

wherein the memory is configured to store instructions for the at least one processor,

wherein the instructions, which when executed by the at least one processor, cause the electronic device to;

obtain a user voice of a first user via the microphone,

obtain at least one keyword from the user voice,

identify a response level of the user voice based on the at least one keyword obtained from the user voice,

when the response level of the user voice identified based on the at least one keyword obtained from the user voice is a first response level, perform a first operation corresponding to the user voice as a response to the user voice,

when the response level of the user voice identified based on the at least one keyword obtained from the user voice is a second response level different from the first response level, control the display to display a user interface including one or more operation candidates based on the at least one keyword, and

when the response level of the user voice is the second response level and an event associated with the first user is detected within a predetermined time after obtaining the user voice of the first user, begin to perform a second operation corresponding to at least one operation candidate among the one or more operation candidates as the response to the user voice,

wherein the first response level is a statement to directly perform a command, and

wherein the second response level is a statement to in-directly perform a command.

2. The electronic device of claim 1, wherein the instructions being further configured to, which when executed by the at least one processor, cause the electronic device to:

obtain text by recognizing the user voice, and

identify the response level of the user voice based on whether the at least one keyword included in the text meets a predetermined rule.

3. The electronic device of claim 2, wherein the instructions being further configured to, which when executed by the at least one processor, cause the electronic device to, when the at least one keyword included in the text meets a rule corresponding to the first response level, identify that the response level of the user voice is the first response level.

4. The electronic device of claim 3, wherein the rule corresponding to the first response level comprises detecting at least one of a control keyword indicating control of the electronic device, a keyword indicating the electronic device, a keyword indicating a place, or a keyword indicating a time.

5. The electronic device of claim 2, wherein the instructions being further configured to, which when executed by the at least one processor, cause the electronic device to, when the at least one keyword included in the text meets a rule corresponding to the second response level, identify that the response level of the user voice is the second response level.

6. The electronic device of claim 5, wherein the rule corresponding to the second response level comprises detecting at least one of a keyword indicating a user'"'"'s emotion or status, a keyword indicating a status of an environment, a keyword indicating a history of the electronic device, a keyword indicating a content of an operation of the electronic device, or a keyword indicating an operation status of the electronic device.

7. The electronic device of claim 2, wherein the predetermined rule comprises an ontology having layers classified depending on abstraction levels related to accuracy for grasping an intention of a command and including association information between the layers or association information between included keywords.

8. The electronic device of claim 7, wherein the instructions being further configured to, which when executed by the at least one processor, cause the electronic device to:

compare the at least one keyword included in the text with the ontology,

assign a score to each keyword included in the text according to a result of the comparison, and

compare a sum of assigned scores with a predetermined threshold to determine the response level of the user voice.

9. The electronic device of claim 8, wherein the instructions being further configured to, which when executed by the at least one processor, cause the electronic device to:

when a sum of the assigned scores exceeds a first threshold, identify that the response level of the user voice is the first response level, and

when the sum of the assigned scores exceeds a second threshold and not more than the first threshold, identify that the response level of the user voice is the second response level.

10. The electronic device of claim 2, wherein the instructions being further configured to, which when executed by the at least one processor, cause the electronic device to expand or update the predetermined rule based on obtained information.

11. The electronic device of claim 2, wherein the instructions being further configured to, which when executed by the at least one processor, cause the electronic device to:

interpret the text in natural language, and

determine whether a keyword classified as a verb as a result of the interpretation of the natural language meets the predetermined rule.

12. The electronic device of claim 2, wherein the instructions being further configured to, which when executed by the at least one processor, cause the electronic device to:

obtain a user'"'"'s emotion using a result of analysis of the user voice or using additional information from another electronic device,

identify the response level of the user voice using the user'"'"'s emotion, and

determine whether the predetermined rule is met.

13. The electronic device of claim 1,

wherein the instructions being further configured to, which when executed by the at least one processor, cause the electronic device to:

output a voice output including the at least one operation candidate via a speaker of the electronic device, and

upon detecting the event within the predetermined time after obtaining the user voice, select the at least one operation candidate, and

wherein the second operation corresponding to the selected at least one operation candidate is performed based on a time point of detecting the event.

14. The electronic device of claim 13, wherein the instructions being further configured to, which when executed by the at least one processor cause the electronic device to, upon detecting another event associated with the first user, output the at least one operation candidate.

15. The electronic device of claim 1,

wherein the memory is configured to store an ontology having layers classified depending on abstraction levels related to accuracy for grasping an intention of a command and association information between the layers or association information between included keywords, and

wherein the instructions being further configured to, which when executed by the at least one processor, cause the electronic device to:

obtain text by recognizing the user voice, and

identify the at least one operation candidate by combining the at least one keyword included in the text with an associated keyword on the ontology.

16. The electronic device of claim 1, wherein the instructions being further configured to, which when executed by the at least one processor, cause the electronic device to:

obtain user identification information corresponding to the user voice, and

when the response level of the user voice is the second response level, perform the second operation corresponding to the user voice depending on whether a user corresponding to an event is the same as a user identified by the user identification information corresponding to the user voice.

17. The electronic device of claim 1, wherein the instructions being further configured to, which when executed by the at least one processor, cause the electronic device to:

determine a first time when the user voice is obtained, and

when the response level of the user voice is the second response level, perform the first operation corresponding to the user voice depending on whether a difference between the first time and a second time when an event is detected is less than a predetermined threshold.

18. The electronic device of claim 1, wherein the instructions being further configured to, which when executed by the at least one processor, cause the electronic device to:

obtain another user voice and compare the response level of the user voice with a response level of the other user voice, and

perform one of an operation corresponding to the user voice or an operation corresponding to the other user voice based on the comparison result.

19. The electronic device of claim 18, wherein the instructions being further configured to, which when executed by the at least one processor, cause the electronic device to perform one of the operation corresponding to the user voice or the operation corresponding to the other user voice by using relation information between the first user corresponding to the user voice and a second user corresponding to the other user voice.

20. The electronic device of claim 1, wherein the instructions being further configured to, which when executed by the at least one processor, cause the electronic device to output a control signal for performing the operation to hardware that is different for each of the first response level and the second response level.

21. The electronic device of claim 1, wherein the instructions being further configured to, which when executed by the at least one processor, cause the electronic device to, when the response level of the user voice identified based on the at least one keyword obtained from the user voice is a third response level, disregard the user voice.

22. The electronic device of claim 1, wherein the user interface including the one or more operation candidates is displayed after the identifying of the second response level.