MOBILE SYSTEMS AND METHODS OF SUPPORTING NATURAL LANGUAGE HUMAN-MACHINE INTERACTIONS

• US 20130253929A1
• Filed: 05/20/2013
• Published: 09/26/2013
• Est. Priority Date: 08/29/2005
• Status: Active Grant

• Current Assignee: Dialect, LLC
• Original Assignee: VoiceBox Technologies, Inc. (Microsoft Corporation)
• Inventors: Weider, Chris, Kennewick, Richard, Kennewick, Mike, Kennewick, Robert A., Di Cristo, Philippe, Menaker, Samuel, Armstrong, Lynn Elise
• Granted Patent: US 8,849,652 B2
• Time in Patent Office: Days
• US Class Current: 704/235
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: G10L 21/06 Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids G10L15/26 takes precedence
• CPC Class Codes: H04M 2250/74 with voice recognition means

Abstract

A mobile system is provided that includes speech-based and non-speech-based interfaces for telematics applications. The mobile system identifies and uses context, prior information, domain knowledge, and user specific profile data to achieve a natural environment for users that submit requests and/or commands in multiple domains. The invention creates, stores and uses extensive personal profile information for each user, thereby improving the reliability of determining the context and presenting the expected results for a particular question or command. The invention may organize domain specific behavior and information into agents, that are distributable or updateable over a wide area network.

Claims

1-49. -49. (canceled)

50. A system for processing natural language utterances where recognized words of the natural language utterances alone are insufficient to completely determine one or more commands or requests, the system including one or more processors executing one or more computer program modules which, when executed, cause the one or more processors to:

generate a context stack comprising context information that corresponds to a plurality of prior utterances;

receive a natural language utterance associated with a command or request;

determine one or more words of the natural language utterance by performing speech recognition on the natural language utterance; and

determine the command or request based on the one or more words and the context information.

51. The system of claim 50, wherein the one or more processors are further caused to:

prompt a user associated with the natural language utterance for additional information or confirmation regarding the command or request; and

receive a non-speech input regarding the additional information or confirmation in response to the prompt, wherein the command or request is determined further based on the non-speech input.

52. The system of claim 50, wherein the context stack includes a plurality of context entries, and wherein the one or more processors are further caused to:

identify, from among the plurality of context entries, one or more context entries that correspond to the one or more words, wherein the context information includes the one or more context entries.

53. The system of claim 52, wherein identifying the one or more context entries comprises:

comparing the plurality of context entries with the one or more words;

generating one or more rank scores for individual context entries of the plurality of context entries based on the comparison; and

identifying, from among the plurality of context entries, the one or more context entries based on the one or more rank scores.

54. The system of claim 53, wherein the plurality of context entries are ordered in the context stack, and wherein the one or more processors are further caused to update the ordering of the plurality of context entries in the context stack based on the one or more rank scores.

55. The system of claim 54, wherein the natural language utterance is associated with a user, and wherein the one or more processors are further caused to:

receive a subsequent natural language utterance associated with the user; and

determining one or more subsequent words of the subsequent natural language utterance by performing speech recognition on the subsequent natural language utterance;

identify, from among the plurality of context entries, one or more other context entries that correspond to the one or more subsequent words based on the updated ordering; and

determine a subsequent command or request based on the one or more subsequent words and the one or more other context entries.

56. The system of claim 52, wherein identifying the one or more context entries comprises identifying, from among the plurality of context entries, the one or more context entries that most closely correspond to the one or more words.

57. The system of claim 50, wherein the one or more processors are further caused to:

identify one or more domain agents associated with the one or more context entries, wherein the one or more domain agents are configured to process the command or request; and

generate a response to the command or request using the one or more domain agents.

58. The system of claim 57, wherein each of the one or more domain agents comprises domain knowledge associated with a particular domain, the domain knowledge comprising:

(i) a keyword;

(ii) a link to an information source;

(iii) a list of responses associated with a plurality commands or requests;

(iv) a substitution list used to format the plurality of commands or requests;

or (v) content including dictionaries, encyclopedias, or almanacs.

59. The system of claim 57, wherein the response comprises an aggregation of one or more responses generated by the one or more domain agents based on the command or request.

60. The system of claim 57, wherein the one or more processors are further caused to:

obtain information relating to a license agreement that is associated with at least one of the one or more domain agents;

determine that use of the at least one of the one or more domain agents to process the command or request is permitted based on the information related to the license agreement; and

generate, using the at least one of the one or more domain agents, the response based on the determination that the use is permitted.

61. The system of claim 50, wherein the natural language utterance is associated with a user, and wherein the one or more processors are further caused to:

obtain a cognitive model or an environmental model associated with the user, wherein the cognitive model comprises information relating to one or more interactions between the user and the system, and the environmental model comprises information indicative of how noisy an environment surrounding the user is,

wherein the one or more words are determined further based on the cognitive model or the environmental model.

62. The system of claim 50, wherein the natural language utterance is associated with a user, and wherein the one or more processors are further caused to:

obtain a first cognitive model that comprises information relating to one or more interactions between the user and the system; and

obtain a second cognitive model that comprises information relating to one or more interactions between the system and a plurality of users of the system,

wherein the one or more words are determined further based on the first cognitive model and the second cognitive model.

63. The system of claim 50, wherein the context stack is a first context stack of a first device, and wherein the one or more processors are further caused to:

synchronize the context stack with a second context stack of a second device.

64. A computer-implemented method of processing natural language utterances where recognized words of the natural language utterances alone are insufficient to completely determine one or more commands or requests, the method being implemented by a computer system that includes one or more processors executing one or more computer program modules which, when executed, perform the method, the method comprising:

generating a context stack comprising context information that corresponds to a plurality of prior utterances;

receiving a natural language utterance associated with a command or request;

determining one or more words of the natural language utterance by performing speech recognition on the natural language utterance; and

determining the command or request based on the one or more words and the context information.

65. The method of claim 64, further comprising:

prompting a user associated with the natural language utterance for additional information or confirmation regarding the command or request; and

receiving a non-speech input regarding the additional information or confirmation in response to the prompt, wherein the command or request is determined further based on the non-speech input.

66. The method of claim 64, wherein the context stack includes a plurality of context entries, the method further comprising:

identifying, from among the plurality of context entries, one or more context entries that correspond to the one or more words, wherein the context information includes the one or more context entries.

67. The method of claim 66, wherein identifying the one or more context entries comprises:

comparing the plurality of context entries with the one or more words;

generating one or more rank scores for individual context entries of the plurality of context entries based on the comparison; and

identifying, from among the plurality of context entries, the one or more context entries based on the one or more rank scores.

68. The method of claim 64, further comprising:

identifying one or more domain agents associated with the one or more context entries, wherein the one or more domain agents are configured to process the command or request; and

generating a response to the command or request using the one or more domain agents.

69. The method of claim 64, wherein the natural language utterance is associated with a user, the method further comprising:

obtaining a first cognitive model that comprises information relating to one or more interactions between the user and the system; and

obtaining a second cognitive model that comprises information relating to one or more interactions between the system and a plurality of users of the system,

wherein the one or more words are determined further based on the first cognitive model and the second cognitive model.