System and method for a cooperative conversational voice user interface

• US 9,015,049 B2
• Filed: 08/19/2013
• Issued: 04/21/2015
• Est. Priority Date: 10/16/2006
• Status: Active Grant

• Current Assignee: VB Assets, LLC
• Original Assignee: VoiceBox Technologies Corporation (Microsoft Corporation)
• Inventors: Baldwin, Larry, Freeman, Tom, Tjalve, Michael, Ebersold, Blane, Weider, Chris
• Primary Examiner(s): YEN, ERIC L
• Application Number:
  US13/987,645
• Publication Number:
  US 20130339022A1
• Time in Patent Office: 610 Days
• Field of Search: 704/9, 704270-275
• US Class Current: 704/275
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/0631 Creating reference templates; Clustering
• CPC Class Codes: G10L 2015/225 Feedback of the input speech
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 25/51 for comparison or discrimination
• CPC Class Codes: G10L 25/63 for estimating an emotional state

Abstract

A cooperative conversational voice user interface is provided. The cooperative conversational voice user interface may build upon short-term and long-term shared knowledge to generate one or more explicit and/or implicit hypotheses about an intent of a user utterance. The hypotheses may be ranked based on varying degrees of certainty, and an adaptive response may be generated for the user. Responses may be worded based on the degrees of certainty and to frame an appropriate domain for a subsequent utterance. In one implementation, misrecognitions may be tolerated, and conversational course may be corrected based on subsequent utterances and/or responses.

Claims

1. A computer-implemented method of facilitating conversation-based responses, the method being implemented by a computer system that includes one or more physical processors executing one or more computer program instructions which, when executed, perform the method, the method comprising:

receiving, at the computer system, a natural language utterance during a conversation between a user and the computer system;

identifying, by the computer system, a first model that includes short-term knowledge about the conversation, wherein the short-term knowledge is based on one or more prior natural language utterances received during the conversation;

identifying, by the computer system, based on the short-term knowledge, context information for the natural language utterance;

determining, by the computer system, based on the context information, an interpretation of the natural language utterance; and

generating, by the computer system, based on the interpretation of the natural language utterance, a response to the natural language utterance.

2. The method of claim 1, wherein determining the interpretation of the natural language utterance comprises determining, based on the context information, an interpretation of one or more recognized words of the natural language utterance.

3. The method of claim 1, further comprising:

updating, by the computer system, based on information about the natural language utterance, during the conversation, the short-term knowledge in the first model,

wherein the updated short-term knowledge is used to determine subsequent context information for one or more subsequent natural language utterances received during the conversation.

4. The method of claim 1, further comprising:

identifying, by the computer system, a second model that includes long-term knowledge about one or more prior conversations between the user and the computer system,

wherein determining the context information comprises determining, based on the short-term knowledge and the long-term knowledge, the context information.

5. The method of claim 4, further comprising:

updating, by the computer system, based on information about the natural language utterance, the long-term knowledge in the second model,

wherein the updated long-term knowledge is used to determine subsequent context information for one or more subsequent natural language utterances received during the conversation.

6. The method of claim 5, wherein the updated long-term knowledge is used to determine subsequent context information for one or more subsequent conversations.

7. The method of claim 1, further comprising:

identifying, by the computer system, based on the short-term knowledge, a manner in which the natural language utterance is spoken,

wherein generating the response comprises generating, based on the identified manner and the interpretation of the natural language utterance, the response.

8. The method of claim 7, further comprising:

identifying, by the computer system, a second model that includes long-term knowledge about one or more prior conversations between the user and the computer system,

wherein identifying the manner comprises identifying, based on the short-term knowledge and the long-term knowledge, the manner.

9. The method of claim 7, wherein generating the response comprises generating, based on a response format associated with the identified manner, the response.

10. The method of claim 1, wherein the natural language utterance and the one or more prior natural language utterances are associated with the user.

11. A system for facilitating conversation-based responses, the system comprising:

one or more physical processors programmed with one or more computer program instructions such that, when executed, the one or more computer program instructions cause the one or more physical processors to;

receive a natural language utterance during a conversation between a user and the system;

identify a first model that includes short-term knowledge about the conversation, wherein the short-term knowledge is based on one or more prior natural language utterances received during the conversation;

identify, based on the short-term knowledge, context information for the natural language utterance;

determine, based on the context information, an interpretation of the natural language utterance; and

generate, based on the interpretation of the natural language utterance, a response to the natural language utterance.

12. The system of claim 11, wherein determining the interpretation of the natural language utterance comprises determining, based on the context information, an interpretation of one or more recognized words of the natural language utterance.

13. The system of claim 11, wherein the one or more physical processors are caused to:

update, based on information about the natural language utterance, during the conversation, the short-term knowledge in the first model,

wherein the updated short-term knowledge is used to determine subsequent context information for one or more subsequent natural language utterances received during the conversation.

14. The system of claim 11, wherein the one or more physical processors are caused to:

identify a second model that includes long-term knowledge about one or more prior conversations between the user and the system,

wherein determining the context information comprises determining, based on the short-term knowledge and the long-term knowledge, the context information.

15. The system of claim 14, wherein the one or more physical processors are caused to:

update, based on information about the natural language utterance, the long-term knowledge in the second model,

wherein the updated long-term knowledge is used to determine subsequent context information for one or more subsequent natural language utterances received during the conversation.

16. The system of claim 15, wherein the updated long-term knowledge is used to determine subsequent context information for one or more subsequent conversations.

17. The system of claim 11, wherein the one or more physical processors are caused to:

identify, based on the short-term knowledge, a manner in which the natural language utterance is spoken,

wherein generating the response comprises generating, based on the identified manner and the interpretation of the natural language utterance, the response.

18. The system of claim 17, wherein the one or more physical processors are caused to:

identify a second model that includes long-term knowledge about one or more prior conversations between the user and the computer system,

wherein identifying the manner comprises identifying, based on the short-term knowledge and the long-term knowledge, the manner.

19. The system of claim 17, wherein generating the response comprises generating, based on a response format associated with the identified manner, the response.

20. The system of claim 11, wherein the natural language utterance and the one or more prior natural language utterances are associated with the user.