DIALOG MANAGEMENT APPARATUS AND METHOD

• US 20170084274A1
• Filed: 09/07/2016
• Published: 03/23/2017
• Est. Priority Date: 09/17/2015
• Status: Abandoned Application

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Jung Hoe, Woo, Kyoung Gu, KANG, Byung Kon, KAM, Hye Jin
• Application Number:
  US15/258,281
• Publication Number:
  US 20170084274A1
• Time in Patent Office: Days
• CPC Class Codes: G10L 13/02 Methods for producing synthetic speech; Speech synthesisers
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/221 Announcement of recognition results
• CPC Class Codes: G10L 2015/225 Feedback of the input speech

Abstract

An intelligent dialog processing apparatus and method. The intelligent dialog processing apparatus includes a speech understanding processor, of one or more processors, configured to perform an understanding of an uttered primary speech of a user using an idiolect of the user based on a personalized database (DB) for the user, and an additional-query processor, of the one or more processors, configured to extract, from the primary speech, a select unit of expression that is not understood by the speech understanding processor, and to provide a clarifying query for the user that is associated with the extracted unit of expression to clarify the extracted unit of expression.

Claims

1. An intelligent dialog processing apparatus, the apparatus comprising:

a speech understanding processor, of one or more processors, configured to perform an understanding of an uttered primary speech of a user using an idiolect of the user based on a personalized database (DB) for the user; and

an additional-query processor, of the one or more processors, configured to extract, from the primary speech, a select unit of expression that is not understood by the speech understanding processor, and to provide a clarifying query for the user that is associated with the extracted unit of expression to clarify the extracted unit of expression.

2. The apparatus of claim 1, wherein the speech understanding processor comprises a reliability calculator configured to calculate a reliability of each unit of expression that makes up the primary speech, using the personalized DB, and

the speech understanding processor performs the understanding of the primary speech using the idiolect of the user based on the calculated reliability.

3. The apparatus of claim 2, wherein the providing of the clarifying query includes analyzing a context of the extracted unit of expression in the primary speech and/or the personalized DB for a potentially related term for the extracted unit of expression and generating a contextualized clarifying query based on a result of the analyzing.

4. The apparatus of claim 2, wherein the personalized DB comprises at least one of the following:

a common DB storing common speech expressions among multiple users;

a personal DB storing various expressions in the idiolect of the user; and

an ontology DB storing either or both the common speech expressions and the expressions in the idiolect of the user in an ontology form.

5. The apparatus of claim 4, wherein the reliability calculator differently weights understanding results from at least two DBs out of the common DB, the personal DB, and the ontology DB, and then calculates the reliability using the differently weighted understanding results.

6. The apparatus of claim 1, wherein the additional-query processor generates the clarifying query based on either or both the extracted unit of expression and a query template.

7. The apparatus of claim 6, wherein the additional-query processor comprises a category determiner configured to determine a category of the extracted unit of expression, and a template extractor configured to extract the query template that corresponds to the determined category from a query template DB.

8. The apparatus of claim 6, wherein the additional-query processor further comprises a voice extractor configured to extract, from audio of the primary speech, audio of the user'"'"'s voice that corresponds to the extracted unit of expression, and

the additional-query creator generates the clarifying query by mixing the extracted audio of the user'"'"'s voice with a generated voicing of the query template.

9. The apparatus of claim 1, wherein the additional-query processor is further configured to interpret a clarifying speech which is received from the user in response to an outputting of the provided clarifying query to the user, and the additional-query processor further comprises an answer detector configured to detect an answer related to the extracted unit of expression in the clarifying speech based on a result of the interpretation of the clarifying speech.

10. The apparatus of claim 9, wherein the additional-query processor comprises an answer confirmation processor configured to make a confirmation query to the user regarding the detected answer, and an answer personalization processor configured to update the personalized DB according to a confirmation reply received from the user in response to the confirmation query.

11. The apparatus of claim 9, further comprising:

a speech determiner configured to determine which of primary and clarifying speeches is intended by an input utterance of the user.

12. The apparatus of claim 1, wherein one of the one or more processors is configured to receive an utterance of the user captured by a voice inputter, to perform recognition of the received utterance, and to provide results of the recognition to the speech understanding processor to perform the understanding based on the provided results.

13. The apparatus of claim 12, further comprising a reply processor, of the one or more processors, configured to provide the clarifying query to the user in a natural language voice.

14. An intelligent dialog processing method, the method comprising:

performing an automated understanding of an uttered primary speech of a user using an idiolect of the user based on a personalized DB for the user;

extracting, from the primary speech, a select unit of expression that is not understood based on the understanding; and

providing a clarifying query associated, through an automated process, with the extracted unit of expression to clarify the extracted unit of expression.

15. The method of claim 14, wherein the understanding of the uttered primary speech comprises calculating a reliability of each unit of expression that makes up the primary speech, based on the personalized DB, and performing the understanding of the primary speech using the idiolect of the user based on the calculated reliability.

16. The method of claim 15, wherein the personalized DB comprises at least one of the following:

a common DB storing common speech expressions among multiple users;

a personal DB storing various expressions in the idiolect of the user; and

an ontology DB storing either or both the common speech expressions and the expressions in the idiolect of the user in an ontology form.

17. The method of claim 14, wherein the providing of the clarifying query comprises generating the clarifying query, for output to the user, based on either or both the extracted unit of expression and a query template.

18. The method of claim 17, wherein the providing of the clarifying query comprises determining a category of the extracted unit of expression, and extracting the query template that corresponds to the determined category from a query template DB.

19. The method of claim 17, wherein the providing of the clarifying query comprises extracting, from audio of the primary speech, audio of the user'"'"'s voice that corresponds to the extracted unit of expression, generating the clarifying query by mixing the extracted audio of the user'"'"'s voice with a generated voicing of the query template, and outputting the generated clarifying query.

20. The method of claim 14, wherein the providing of the clarifying query comprises interpreting a clarifying speech which is received from the user in response to an outputting of the provided clarifying query to the user, and detecting an answer related to the extracted unit of expression in the clarifying speech based on a result of the interpretation of the clarifying speech.

21. The method of claim 20, wherein the providing of the clarifying query comprises generating a confirmation query regarding the detected answer, presenting the generated confirmation query to the user, and updating the personalized DB according to a confirmation reply received from the user in response to the confirmation query.

22. The method of claim 20, further comprising:

determining which of primary and clarifying speeches is intended by an input utterance of the user.

23. The method of claim 14,

wherein the performing of the understanding of the uttered primary speech further comprises receiving the uttered primary speech from a remote terminal that captured the uttered primary speech, and

wherein the providing of the clarifying query comprises providing the clarifying query to the remote terminal to output the clarifying query to the user.

24. The method of claim 23, wherein the received uttered primary speech is in a text form as having been recognized by a recognizer processor of the remote terminal using at least one of an acoustic model and a language model to recognize the captured uttered primary speech.

25. The method of claim 14, further comprising:

receiving an utterance of the user captured by a voice inputter;

performing recognition on the received utterance, where the performing of the understanding includes performing the understanding using results of the recognition; and

outputting the clarifying query to the user, as a reply to the utterance, in a natural language voice.

26. An intelligent dialog processing system comprising:

a speech recognizer processor, of one or more processors, configured to receive an initial utterance of a statement by the user, and to perform a recognition of the received initial utterance;

an utterance processor, of the one or more processors, configured to perform an understanding of the recognized initial utterance using an idiolect of the user based on results of the recognition and a personalized DB of the user, process a clarifying query associated with a unit of expression that is not understood in the understanding of the recognized initial utterance, and to output the clarifying query; and

a reply processor, of the one or more processors, configured to generate a natural language reply to the received initial utterance of the user using the clarifying query for clarify a portion of the initial utterance to the utterance processor.

27. The system of claim 26, wherein the speech recognizer processor recognizes the received initial utterance using either or both an acoustic model and a language model, and provides the results of the recognition to the utterance processor in a text form.

28. The system of claim 26, wherein the utterance processor determines a category of the unit of expression, and generates the clarifying query by combining the unit of expression and a query template that corresponds to the determined category.

29. The system of claim 28, wherein the utterance processor extracts, from audio of the initial utterance, audio of the user'"'"'s voice that corresponds to the unit of expression, and generates the clarifying query by mixing the extracted audio of the user'"'"'s voice with a generated voicing of the query template.

30. The system of claim 28, wherein, when a clarifying speech is received in response to the clarifying query, the utterance processor detects an answer related to the unit of expression from the clarifying speech and provides a final result of an understanding of the initial utterance based on both the detected answer and the performed understanding of the initial utterance.

31. The system of claim 26, wherein the reply processor extracts a reply candidate from the personalized DB based on results of the understanding of the initial utterance, generates a natural language question using the extracted reply candidate, converts the generated question into a natural language voice, and provides the natural language voice for output to the user.