System and method for a cooperative conversational voice user interface

• US 20130339022A1
• Filed: 08/19/2013
• Published: 12/19/2013
• Est. Priority Date: 10/16/2006
• Status: Active Grant

• Current Assignee: VB Assets, LLC
• Original Assignee: VoiceBox Technologies Corporation (Microsoft Corporation)
• Inventors: Baldwin, Larry, Freeman, Tom, Tjalve, Michael, Ebersold, Blane, Weider, Chris
• Granted Patent: US 9,015,049 B2
• Time in Patent Office: Days
• US Class Current: 704/257
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/0631 Creating reference templates; Clustering
• CPC Class Codes: G10L 2015/225 Feedback of the input speech
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 25/51 for comparison or discrimination
• CPC Class Codes: G10L 25/63 for estimating an emotional state

Abstract

A cooperative conversational voice user interface is provided. The cooperative conversational voice user interface may build upon short-term and long-term shared knowledge to generate one or more explicit and/or implicit hypotheses about an intent of a user utterance. The hypotheses may be ranked based on varying degrees of certainty, and an adaptive response may be generated for the user. Responses may be worded based on the degrees of certainty and to frame an appropriate domain for a subsequent utterance. In one implementation, misrecognitions may be tolerated, and conversational course may be corrected based on subsequent utterances and/or responses.

Claims

1. A computer-implemented method of facilitating conversation-based responses, the method being implemented by a computer system that includes one or processors executing one or more computer program instructions which, when executed, perform the method, the method comprising:

receiving, by the computer system, a natural language utterance during a conversation between a user and the computer system;

identifying, by the computer system, a first model that includes short-term knowledge about the conversation or related conversations, wherein the short-term knowledge is based on one or more prior natural language utterances received during the conversation or the related conversations;

identifying, by the computer system, context information for the natural language utterance based on the short-term knowledge;

determining, by the computer system, an interpretation of the natural language utterance based on the context information; and

generating, by the computer system, a response based on the interpretation of the natural language utterance.

2. The method of claim 1, wherein determining the interpretation of the natural language utterance comprises determining an interpretation of one or more recognized words of the natural language utterance based on the context.

3. The method of claim 1, further comprising:

updating, by the computer system during the conversation, the short-term knowledge in the first model based on information about the natural language utterance, wherein the updated short-term knowledge is used to determine subsequent context information for one or more subsequent natural language utterances received during the conversation.

4. The method of claim 1, further comprising:

identifying, by the computer system, a second model that includes long-term knowledge about one or more prior conversations between the user and the computer system,

wherein determining the context information comprises determining the context information based on the short-term knowledge and the long-term knowledge.

5. The method of claim 4, further comprising:

updating, by the computer system, the long-term knowledge in the second model based on the information about the natural language utterance, wherein the updated long-term knowledge is used to determine subsequent context information for one or more subsequent natural language utterances received during the conversation.

6. The method of claim 5, wherein the updated long-term knowledge is used to determine subsequent context information for one or more subsequent conversations.

7. The method of claim 1, further comprising:

identifying, by the computer system, a manner in which the natural language utterance is spoken based on the short-term knowledge,

wherein generating the response comprises generating the response based on the identified manner and the interpretation of the natural language utterance.

8. The method of claim 7, further comprising:

identifying, by the computer system, a second model that includes long-term knowledge about one or more prior conversations between the user and the computer system,

wherein identifying the manner comprises identifying the manner based on the short-term knowledge and the long-term knowledge.

9. The method of claim 7, wherein generating the response comprises generating the response based on a response format associated with the identified manner.

10. The method of claim 1, wherein the natural language utterance and the one or more prior natural language utterances are associated with the user.

11. A system for facilitating conversation-based responses, the system comprising one or more processors programmed with one or more computer program instructions such that, when executed, the one or more computer program instructions cause the one or more processors to:

receive a natural language utterance during a conversation between a user and the system;

identify a first model that includes short-term knowledge about the conversation or related conversations, wherein the short-term knowledge is based on one or more prior natural language utterances received during the conversation or the related conversations;

identify context information for the natural language utterance based on the short-term knowledge;

determine an interpretation of the natural language utterance based on the context information; and

generate a response based on the interpretation of the natural language utterance.

12. The system of claim 11, wherein determining the interpretation of the natural language utterance comprises determining an interpretation of one or more recognized words of the natural language utterance based on the context.

13. The system of claim 11, wherein the one or more processors are caused to:

update, during the conversation, the short-term knowledge in the first model based on information about the natural language utterance, wherein the updated short-term knowledge is used to determine subsequent context information for one or more subsequent natural language utterances received during the conversation.

14. The system of claim 11, wherein the one or more processors are caused to:

identify a second model that includes long-term knowledge about one or more prior conversations between the user and the system,

wherein determining the context information comprises determining the context information based on the short-term knowledge and the long-term knowledge.

15. The system of claim 14, wherein the one or more processors are caused to:

update the long-term knowledge in the second model based on the information about the natural language utterance, wherein the updated long-term knowledge is used to determine subsequent context information for one or more subsequent natural language utterances received during the conversation.

16. The system of claim 15, wherein the updated long-term knowledge is used to determine subsequent context information for one or more subsequent conversations.

17. The system of claim 11, wherein the one or more processors are caused to:

identify a manner in which the natural language utterance is spoken based on the short-term knowledge,

wherein generating the response comprises generating the response based on the identified manner and the interpretation of the natural language utterance.

18. The system of claim 17, wherein the one or more processors are caused to:

identify a second model that includes long-term knowledge about one or more prior conversations between the user and the computer system,

wherein identifying the manner comprises identifying the manner based on the short-term knowledge and the long-term knowledge.

19. The system of claim 17, wherein generating the response comprises generating the response based on a response format associated with the identified manner.

20. The system of claim 11, wherein the natural language utterance and the one or more prior natural language utterances are associated with the user.