Voice recognition apparatus and method for providing response information

• US 9,865,252 B2
• Filed: 01/31/2014
• Issued: 01/09/2018
• Est. Priority Date: 01/31/2013
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Bak, Eun-sang, Kim, Kyung-duk, Kim, Myung-jae, Liu, Yu, Ryu, Seong-han, Lee, Geun-bae
• Primary Examiner(s): Adesanya, Olujimi
• Application Number:
  US14/169,177
• Publication Number:
  US 20140214425A1
• Time in Patent Office: 1,439 Days
• Field of Search: 704 9, 704231, 704235, 704246, 704270, 704275
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

A voice recognition apparatus and a method for providing response information are provided. The voice recognition apparatus according to the present disclosure includes an extractor configured to extract a first utterance element representing a user action and a second utterance element representing an object from a user'"'"'s utterance voice signal; a domain determiner configured to detect an expansion domain related to the extracted first and second utterance elements based on a hierarchical domain model, and determine at least one candidate domain related to the detected expansion domain as a final domain; a communicator which performs communication with an external apparatus; and a controller configured to control the communicator to transmit information regarding the first and second utterance elements and information regarding the determined final domain.

Claims

1. A dialogue type voice recognition apparatus comprising:

a communicator which is configured to perform communication with an external apparatus; and

at least one hardware processor configured to execute voice recognition of the dialogue type voice recognition apparatus to;

convert a user'"'"'s utterance voice signal into text;

extract a first utterance element representing a user action and a second utterance element representing an object from the text of the user'"'"'s utterance voice signal,

detect an expansion domain, being related to the extracted first utterance element and second utterance element, in a hierarchical domain model, when a user action script, comprising a main action corresponding to the first utterance element and a component slot corresponding to the second utterance element, is included in the expansion domain, wherein the hierarchical domain model comprises a plurality of candidate domains of a lowermost concept and the expansion domain, including at least two candidate domains, of an uppermost concept,

detect at least one candidate domain related to the detected expansion domain when the user action script is included in the candidate domain,

determine the at least one candidate domain related to the detected expansion domain as a final domain,

detect a candidate domain, when the user action script is included in the candidate domain and the expansion domain is not detected,

control the communicator to transmit information regarding the first and second utterance elements and information regarding the determined final domain as a result of the voice recognition of the dialogue type voice recognition apparatus, to an external apparatus, and

control the communicator to receive response information related to the first and second utterance elements within the determined final domain from the external apparatus.

2. The dialogue type voice recognition apparatus according to claim 1, where in the at least one hardware processor is configured to:

detect at least one candidate domain including the user action script, and

determine a domain as the expansion domain when two or more of the detected candidate domains comprise a hierarchy with the domain as a common superior concept.

3. The dialogue type voice recognition apparatus according to claim 1, wherein the at least one hardware processor is configured to:

use a multi classifier to detect a subordinate candidate domain related to the detected expansion domain, where the main action corresponding to the first utterance element and the component slot corresponding to the extracted second utterance element are matched to each other is detected, and

detect a candidate domain where the main action corresponding to the first utterance element and the component slot corresponding to the second utterance element matched to each other, when the expansion domain is not detected.

4. The dialogue type voice recognition apparatus according to claim 1, wherein the at least one hardware processor is configured to use at least one binary classifier to detect at least one domain from among the candidate domains, where the main action corresponding to the first utterance element and the component slot corresponding to the second utterance element are matched to each other, and the expansion domain.

5. The dialogue type voice recognition apparatus according to claim 3, wherein the at least one hardware processor is configured to:

determine a user'"'"'s intention from at least one of the extracted first and second utterance elements and the detected candidate domain to determine whether or not a context initialization occurred, and

when it is determined that the context initialization has not occurred, determine at least one candidate domain from among the detected candidate domains as the final domain based on a previous domain, the previous domain being a domain determined from a user'"'"'s previous utterance voice before receiving the text of the user'"'"'s utterance voice signal.

6. The dialogue type voice recognition apparatus according to claim 5, wherein the at least one hardware processor is configured to determine the previous domain as the final domain in response to one candidate domain from among the detected candidate domains and the previous domain corresponding to each other.

7. The dialogue type voice recognition apparatus according to claim 6, wherein the at least one hardware processor is configured to determine the candidate domain, related to the first utterance element representing the user action, from among the at least one candidate domain related to the detected expansion domain, as the final domain, in response to determining that the detected candidate domain and the previous domain are different from each other or the context initialization has occurred.

8. The dialogue type voice recognition apparatus according to claim 7, wherein the at least one hardware processor is configured to determine all the detected candidate domains as the final domain in response to the detected candidate domain including main action information corresponding to the first utterance element representing the user action.

9. The dialogue type voice recognition apparatus according to claim 8, wherein the at least one hardware processor is configured to determine the candidate domain selected by a user as the final domain or arbitrarily select one candidate domain from among the detected candidate domains as the final domain, in response to the detected candidate domain including the main action information corresponding to the first utterance element representing the user action.

10. The dialogue type voice recognition apparatus according to claim 7, wherein the at least one hardware processor is configured to determine the candidate domain corresponding to the previous domain from among the detected candidate domains as the final domain, in response to there being a plurality of previous domains, one of the plurality of previous domains and one of the candidate domains corresponding to each other, and the candidate domain including main action information corresponding to the first utterance element representing the user action.

11. A method of performing voice recognition for providing response information corresponding to a user'"'"'s utterance voice in a dialogue type voice recognition apparatus, implemented by at least one hardware processor, the method comprising:

converting a user'"'"'s utterance voice signal into text;

extracting a first utterance element representing a user action and a second utterance element representing an object from the text of the user'"'"'s utterance voice signal;

detecting an expansion domain, being related to the extracted first utterance element and second utterance element, in a hierarchical domain model, when a user action script, comprising a main action corresponding to the first utterance element and a component slot corresponding to the second utterance element, is included in the expansion domain, wherein the hierarchical domain model comprises a plurality of candidate domains of a lowermost concept and the expansion domain, including at least two candidate domains, of an uppermost concept;

detecting at least one candidate domain related to the detected expansion domain when the user action script is included in the candidate domain;

determining the at least one candidate domain related to the detected expansion domain as a final domain;

detecting a candidate domain, when the user action script is included in the candidate domain and the expansion domain is not detected;

transmitting information regarding the first and second utterance elements and information regarding the determined final domain as a result of the voice recognition to an external apparatus; and

receiving response information related to the first and second utterance elements within the determined final domain from the external apparatus.

12. The method according to claim 11 further comprising:

detecting at least one candidate domain including the user action script; and

determining a domain as the expansion domain when two or more of the detected candidate domains comprise a hierarchy with the domain as a common superior concept.

13. The method according to claim 11, wherein the detecting the expansion domain comprises:

using a multi classifier to detect a subordinate candidate domain related to the detected expansion domain, where the main action corresponding to the first utterance element and the component slot corresponding to the extracted second utterance element are matched to each other is detected, and

when the expansion domain is not detected, the detecting the expansion domain incorporates using a multi-classifier to detect a candidate domain, where the main action corresponding to the first utterance element and the component slot corresponding to the second utterance element are matched to each other.

14. The method according to claim 11, wherein the detecting the expansion domain comprises using at least one binary classifier to detect at least one domain from among the candidate domains, where the main action corresponding to the first utterance element and the component slot corresponding to the second utterance element are matched to each other, and the expansion domain.

15. The method according to claim 13, wherein the determining the final domain comprise determining a user'"'"'s intention from at least one of the extracted first and second utterance elements and the detected candidate domain to determine whether or not a context initialization occurred, and in response to determining that the context initialization has not occurred, determining at least one candidate domain from among the detected candidate domains as the final domain based on a previous domain, the previous domain being a domain determined from a user'"'"'s previous utterance voice before receiving the text of the user'"'"'s utterance voice signal.

16. The method according to claim 15, wherein the determining the final domain comprises determining the previous domain as the final domain in response to one candidate domain from among the detected candidate domains and the previous domain corresponding to each other.

17. The method according to claim 16, wherein the determining the final domain comprises determining the candidate domain related to the first utterance element representing the user action from among the at least one candidate domain related to the detected expansion domain as the final domain, in response to determining that the detected candidate domain and the previous domain are different from each other or the context initialization has occurred.

18. The method according to claim 17, wherein the determining the final domain comprises determining all the detected candidate domains as the final domain, in response to the detected candidate domain including main action information corresponding to the first utterance element representing the user action.

19. The method according to claim 18, wherein the determining the final domain comprises determining the candidate domain selected by a user as the final domain or arbitrarily selecting one candidate domain from among the detected candidate domains as the final domain, in response to the detected candidate domain including the main action information corresponding to the first utterance element representing the user action.

20. The method according to claim 17, wherein the determining the final domain incorporates determining the candidate domain corresponding to the previous domain from among the detected candidate domains as the final domain, in response to there being a plurality of previous domains, one of the plurality of previous domains and one of the candidate domains corresponding to each other, and the candidate domain including main action information corresponding to the first utterance element representing the user action.