Mobile systems and methods of supporting natural language human-machine interactions

• US 9,495,957 B2
• Filed: 08/25/2014
• Issued: 11/15/2016
• Est. Priority Date: 08/29/2005
• Status: Active Grant

• Current Assignee: Dialect, LLC
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Weider, Chris, Kennewick, Richard, Kennewick, Mike, Di Cristo, Philippe, Kennewick, Robert A., Menaker, Samuel, Armstrong, Lynn Elise
• Primary Examiner(s): YEN, ERIC L
• Application Number:
  US14/467,641
• Publication Number:
  US 20140365222A1
• Time in Patent Office: 813 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: G10L 21/06 Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids G10L15/26 takes precedence
• CPC Class Codes: H04M 2250/74 with voice recognition means

Abstract

A mobile system is provided that includes speech-based and non-speech-based interfaces for telematics applications. The mobile system identifies and uses context, prior information, domain knowledge, and user specific profile data to achieve a natural environment for users that submit requests and/or commands in multiple domains. The invention creates, stores and uses extensive personal profile information for each user, thereby improving the reliability of determining the context and presenting the expected results for a particular question or command. The invention may organize domain specific behavior and information into agents, that are distributable or updateable over a wide area network.

Claims

1. A system for processing a natural language utterance, the system including one or more processors executing one or more computer program modules which, when executed, cause the one or more processors to:

generate a context stack comprising context information that corresponds to a plurality of prior utterances, wherein the context stack includes a plurality of context entries;

receive the natural language utterance, wherein the natural language utterance is associated with a command or is associated with a request;

determine one or more words of the natural language utterance by performing speech recognition on the natural language utterance;

identify, from among the plurality of context entries, one or more context entries that correspond to the one or more words, wherein the context information includes the one or more context entries, wherein identifying the one or more context entries comprises;

comparing the plurality of context entries to the one or more words;

generating, based on the comparison, one or more rank scores for individual context entries of the plurality of context entries; and

identifying, based on the one or more rank scores, the one or more context entries from among the plurality of context entries; and

determine, based on the determined one or more words and the context information, the command or the request associated with the natural language utterance.

2. The system of claim 1, wherein the one or more processors are further caused to:

provide a prompt regarding the command or request, wherein the prompt is a prompt for additional information or a prompt for confirmation; and

receive, in response to providing the prompt, a non-speech input, wherein the command or request is determined further based on the non-speech input.

3. The system of claim 1, wherein the plurality of context entries are ordered in the context stack, and wherein the one or more processors are further caused to update, based on the one or more rank scores, the ordering of the plurality of context entries in the context stack.

4. The system of claim 3, wherein the natural language utterance is associated with a user, and wherein the one or more processors are further caused to:

receive a subsequent natural language utterance associated with the user;

determine one or more words of the subsequent natural language utterance by performing speech recognition on the subsequent natural language utterance;

identify, based on the updated ordering of the plurality of context entries in the context stack, one or more other context entries that correspond to the determined one or more words of the subsequent natural language utterance; and

determine, based on the determined one or more words of the subsequent natural language utterance and the identified one or more other context entries, a subsequent command or a subsequent request.

5. The system of claim 1, wherein identifying the one or more context entries comprises identifying, from among the plurality of context entries, the one or more context entries that most closely correspond to the one or more words.

6. The system of claim 1, wherein the context stack is a first context stack of a first device, and wherein the one or more processors are further caused to:

synchronize the context stack with a second context stack of a second device.

7. A computer-implemented method of processing a natural language utterance, the method comprising:

generating a context stack comprising context information that corresponds to a plurality of prior utterances, wherein the context stack includes a plurality of context entries;

receiving, by one or more computer processors, the natural language utterance, wherein the natural language utterance is associated with a command or is associated with a request;

determining one or more words of the natural language utterance by performing speech recognition on the natural language utterance;

identifying, from among the plurality of context entries, one or more context entries that correspond to the one or more words, wherein the context information includes the one or more context entries, wherein identifying the one or more context entries comprises;

comparing the plurality of context entries to the one or more words;

generating, based on the comparison, one or more rank scores for individual context entries of the plurality of context entries; and

identifying, based on the one or more rank scores, the one or more context entries from among the plurality of context entries; and

determining based on the determined one or more words and the context information, the command or the request associated with the natural language utterance.

8. The method of claim 7, further comprising:

providing a prompt regarding the command or request, wherein the prompt is a prompt for additional information or a prompt for confirmation; and

receiving, in response to providing the prompt, a non-speech input, wherein the command or request is determined further based on the non-speech input.