SYSTEM AND METHOD FOR PROVIDING A NATURAL LANGUAGE VOICE USER INTERFACE IN AN INTEGRATED VOICE NAVIGATION SERVICES ENVIRONMENT

• US 20120101810A1
• Filed: 12/30/2011
• Published: 04/26/2012
• Est. Priority Date: 12/11/2007
• Status: Active Grant

• Current Assignee: Cerence Inc.
• Original Assignee: VoiceBox Technologies, Inc. (Microsoft Corporation)
• Inventors: Cheung, Catherine, Baldwin, Larry, Salomon, Ari, Tjalve, Michael, Guttigoli, Sheetal, Armstrong, Lynn, Zimmerman, Bernie, Menaker, Sam, KENNEWICK, Michael R., Dichristo, Philippe
• Granted Patent: US 8,370,147 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G01C 21/3608 using speech input, e.g. using speech recognition
• CPC Class Codes: G06Q 30/0261 based on user location
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 15/193 Formal grammars, e.g. finite state automata, context free grammars or word networks
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

A conversational, natural language voice user interface may provide an integrated voice navigation services environment. The voice user interface may enable a user to make natural language requests relating to various navigation services, and further, may interact with the user in a cooperative, conversational dialogue to resolve the requests. Through dynamic awareness of context, available sources of information, domain knowledge, user behavior and preferences, and external systems and devices, among other things, the voice user interface may provide an integrated environment in which the user can speak conversationally, using natural language, to issue queries, commands, or other requests relating to the navigation services provided in the environment.

Claims

1. A method for providing a natural language voice user interface in an integrated voice navigation services environment, comprising:

receiving a natural language utterance at an input device coupled to a navigation device, wherein the natural language utterance relates to a navigation context;

generating one or more preliminary interpretations of the natural language utterance using a speech recognition engine associated with the navigation device, wherein the speech recognition engine uses a dynamic recognition grammar to generate the one or more preliminary interpretations of the natural language utterance;

analyzing, with a conversational language processor on the navigation device, the one or more preliminary interpretations using shared knowledge and information associated with the navigation context to determine a probable interpretation of the natural language utterance in the navigation context; and

executing, on the navigation device, a navigation agent associated with the navigation context to process the probable interpretation of the natural language utterance, wherein executing the navigation agent to process the probable interpretation of the natural language utterance includes;

identifying, by the navigation agent executing on the navigation device, one or more requests in the natural language utterance that relate to the navigation context from the probable interpretation of the natural language utterance; and

resolving, by the navigation agent executing on the navigation device, the one or more requests using information associated with a plurality of information sources, which include at least a navigation-specific information source.

2. The method of claim 1, wherein resolving the one or more requests using the information associated with the plurality of information sources includes:

determining, by the navigation agent executing on the navigation device, that the one or more requests in the natural language utterance approximate requested information; and

causing the conversational language processor to manage a dialogue that includes one or more subsequent interactions to successively refine and resolve the requested information approximated in the one or more requests, wherein the one or more subsequent interactions in the dialogue include one or more output prompts and one or more subsequent multi-modal inputs to successively refine and resolve the requested information approximated in the one or more requests.

3. The method of claim 1, wherein resolving the one or more requests using the information associated with the plurality of information sources includes:

determining, by the navigation agent executing on the navigation device, that the one or more requests identified in the natural language utterance include a navigation request to calculate a route to a full or partial address;

calculating the route from a current location associated with the navigation device to a destination having an address that best corresponds to the full or partial address; and

generating dynamic directions to provide a guide on the route from the current location associated with the navigation device to the destination associated with the route, wherein the navigation agent drives the dynamic directions using the information associated with the plurality of information sources and the current location associated with the navigation device.

4. The method of claim 3, wherein the dynamic directions include information about one or more destinations, one or more points of interest, traffic, parking, weather, or one or more events relevant to the route and the current location associated with the navigation device.

5. The method of claim 3, further comprising:

receiving a multi-modal input at the input device subsequent to calculating the route, wherein the multi-modal includes a subsequent request; and

invoking one or more domain agents to resolve the subsequent request in the multi-modal input, wherein the one or more domain agents filter results associated with the subsequent request according to the calculated route.

6. The method of claim 3, wherein executing the navigation agent to process the probable interpretation of the natural language utterance further includes:

determining that the one or more requests identified in the natural language utterance further include a multi-modal request to control a map display associated with the route, wherein the navigation agent executing on the navigation device determines that the one or more requests further include the multi-modal request to control the map display using the information associated with the plurality of information sources;

associating a non-voice component in the multi-modal input with the probable interpretation of the natural language utterance, wherein the non-voice component in the multi-modal input identifies a portion of the map display; and

issuing a command to control the identified portion of the map display in accordance with the probable interpretation of the natural language utterance to resolve the one or more requests identified in the natural language utterance.

7. The method of claim 1, wherein the dynamic recognition grammar includes information associated with one or more topological domains derived from one or more physical, temporal, directional, or civil organizational proximities relative to a current location associated with the navigation device.

8. The method of claim 7, wherein generating the one or more preliminary interpretations of the natural language utterance includes:

recognizing one or more words in the natural language utterance that define a command in the navigation context, wherein the speech recognition engine includes a multi-pass speech recognition module that recognizes the one or more words that define the command in the navigation context;

recognizing, at the multi-pass speech recognition module, one or more additional words in the natural language utterance that define a location associated with the command in the navigation context; and

generating, at the multi-pass speech recognition module, the dynamic recognition grammar based on the location defined in the one or more additional words recognized in the natural language utterance, wherein the speech recognition engine uses the dynamic recognition grammar to generate the one or more preliminary interpretations of the natural language utterance.

9. The method of claim 1, wherein the shared knowledge that the conversational language processor uses to analyze the one or more preliminary interpretations and determine the probable interpretation of the natural language utterance includes one or more inferences generated by an inferencing engine associated with the navigation device.

10. The method of claim 9, further comprising generating a response to the natural language utterance to suggest one or more services available in the navigation context, wherein the conversational language processor uses the one or more inferences generated by the inferencing engine to determine the one or more suggested services.

11. A system for providing a natural language voice user interface in an integrated voice navigation services environment, comprising:

an input device configured to receive a natural language utterance, wherein the natural language utterance relates to a navigation context;

a speech recognition engine configured to generate one or more preliminary interpretations of the natural language utterance using a dynamic recognition grammar;

a conversational language processor configured to analyze the one or more preliminary interpretations using shared knowledge and information associated with the navigation context to determine a probable interpretation of the natural language utterance in the navigation context; and

a navigation agent associated with the navigation context and configured to;

identify one or more requests in the natural language utterance that relate to the navigation context from the probable interpretation of the natural language utterance; and

resolve the one or more requests using information associated with a plurality of information sources, which include at least a navigation-specific information source.

12. The system of claim 11, wherein to resolve the one or more requests using the information associated with the plurality of information sources, the navigation agent is further configured to:

determine that the one or more requests in the natural language utterance approximate requested information; and

cause the conversational language processor to manage a dialogue that includes one or more subsequent interactions to successively refine and resolve the requested information approximated in the one or more requests, wherein the one or more subsequent interactions in the dialogue include one or more output prompts and one or more subsequent multi-modal inputs to successively refine and resolve the requested information approximated in the one or more requests.

13. The system of claim 11, wherein to resolve the one or more requests using the information associated with the plurality of information sources, the navigation agent is further configured to:

determine that the one or more requests identified in the natural language utterance include a navigation request to calculate a route to a full or partial address;

calculate the route from a current location associated with the navigation device to a destination having an address that best corresponds to the full or partial address; and

generate dynamic directions to provide a guide on the route from the current location associated with the navigation device to the destination associated with the route, wherein the navigation agent drives the dynamic directions using the information associated with the plurality of information sources and the current location associated with the navigation device.

14. The system of claim 13, wherein the dynamic directions include information about one or more destinations, one or more points of interest, traffic, parking, weather, or one or more events relevant to the route and the current location associated with the navigation device.

15. The system of claim 13, further comprising one or more domain agents configured to:

resolve a subsequent request in a multi-modal input received at the input device subsequent to the navigation agent having calculated the route; and

filter results associated with the subsequent request according to the calculated route to resolve the subsequent request.

16. The system of claim 13, wherein to resolve the one or more requests using the information associated with the plurality of information sources, the navigation agent is further configured to:

determine that the one or more requests identified in the natural language utterance further include a multi-modal request to control a map display associated with the route using the information associated with the plurality of information sources;

associate a non-voice component in the multi-modal input with the probable interpretation of the natural language utterance, wherein the non-voice component in the multi-modal input identifies a portion of the map display; and

issue a command to control the identified portion of the map display in accordance with the probable interpretation of the natural language utterance to resolve the one or more requests identified in the natural language utterance.

17. The method of claim 11, wherein the dynamic recognition grammar includes information associated with one or more topological domains derived from one or more physical, temporal, directional, or civil organizational proximities relative to a current location associated with the navigation device.

18. The system of claim 17, further comprising a multi-pass speech recognition module configured to:

recognize one or more words in the natural language utterance that define a command in the navigation context;

recognize one or more additional words in the natural language utterance that define a location associated with the command in the navigation context; and

generate the dynamic recognition grammar based on the location defined in the one or more additional words recognized in the natural language utterance.

19. The system of claim 11, further comprising an inferencing engine configured to generate one or more inferences, wherein the shared knowledge used in the conversational language processor to analyze the one or more preliminary interpretations and determine the probable interpretation of the natural language utterance includes the one or more inferences.

20. The system of claim 19, wherein the conversational language processor is further configured to generate a response to the natural language utterance to suggest one or more services available in the navigation context using the one or more inferences.

21. A method for providing a natural language voice user interface in an integrated voice navigation services environment, comprising:

receiving a natural language utterance at an input device coupled to a navigation device, wherein the natural language utterance relates to a navigation context;

generating one or more preliminary interpretations of the natural language utterance using a speech recognition engine associated with the navigation device, wherein the speech recognition engine uses a dynamic recognition grammar to generate the one or more preliminary interpretations of the natural language utterance;

analyzing, with a conversational language processor on the navigation device, the one or more preliminary interpretations using shared knowledge and information associated with the navigation context to determine a preliminary destination having an address that best corresponds to a full or partial address associated with the approximated destination;

executing, on the navigation device, a navigation agent associated with the navigation context to calculate a route from the current location associated with the navigation device to the preliminary destination; and

managing, via the conversational language processor, a dialogue that includes one or more subsequent interactions to successively refine the approximated destination until a final destination associated with the navigation command has been resolved.

22. The method of claim 21, wherein the one or more subsequent interactions in the managed dialogue include one or more output prompts and one or more subsequent multi-modal inputs to successively refine the approximated destination into the final destination.

23. The method of claim 22, wherein generating the one or more preliminary interpretations of the natural language utterance includes:

recognizing one or more words in the natural language utterance that define a navigation command in the navigation context, wherein the speech recognition engine includes a multi-pass speech recognition module that recognizes the one or more words that define the navigation command;

recognizing, at the multi-pass speech recognition module, one or more additional words in the natural language utterance that define the approximated destination associated with the navigation command or the final destination successively refined in the managed dialogue; and

generating, at the multi-pass speech recognition module, the dynamic recognition grammar based on a current location associated with the navigation device, the approximated destination associated with the navigation command, or the final destination successively refined in the managed dialogue.

24. The method of claim 21, further comprising dynamically recalculating the route in response to the one or more subsequent interactions refining the approximated destination.

25. The method of claim 24, wherein dynamically recalculating the route includes generating directions from the current location associated with the navigation device to the refined approximated destination.

26. The method of claim 21, wherein the one or more subsequent interactions in the managed dialogue resolve the final destination substantially later in time relative to the navigation agent initially calculating the route to the preliminary destination.

27. The method of claim 21, wherein analyzing the one or more preliminary interpretations to determine the preliminary destination includes:

identifying multiple destinations having addresses that possibly correspond to the full or partial address associated with the approximated destination;

ranking the multiple identified destinations according to proximities to the current location associated with the navigation device or the full or partial address associated with the approximated destination; and

selecting one of the multiple identified destinations having a highest ranking to be the preliminary destination.

28. The method of claim 27, wherein the proximities depend on one or more topological domains that include physical, temporal, directional, and civil organizational proximities relative to the current location associated with the navigation device or the full or partial address associated with the approximated destination.

29. The method of claim 21, wherein executing the navigation agent to calculate the route includes generating dynamic directions to provide a guide on the route from the current location associated with the navigation device to the preliminary destination, wherein the navigation agent drives the dynamic directions using the information associated with the navigation context and the current location associated with the navigation device.

30. The method of claim 21, further comprising dynamically recalculating the route in response to the managed dialogue resolving the final destination, wherein dynamically recalculating the route includes generating directions from the current location associated with the navigation device to the resolved final destination.

31. A system for providing a natural language voice user interface in an integrated voice navigation services environment, comprising:

an input device configured to receive a natural language utterance, wherein the natural language utterance relates to a navigation context;

a speech recognition engine configured to generate one or more preliminary interpretations of the natural language utterance using a dynamic recognition grammar;

a conversational language processor configured to analyze the one or more preliminary interpretations using shared knowledge and information associated with the navigation context to determine a preliminary destination having an address that best corresponds to a full or partial address associated with the approximated destination; and

a navigation agent configured to;

calculate a route from the current location associated with the navigation device to the preliminary destination; and

manage, via the conversational language processor, a dialogue that includes one or more subsequent interactions to successively refine the approximated destination until a final destination associated with the navigation command has been resolved.

32. The system of claim 31, wherein the one or more subsequent interactions in the managed dialogue include one or more output prompts and one or more subsequent multi-modal inputs to successively refine the approximated destination into the final destination.

33. The system of claim 32, further comprising a multi-pass speech recognition module configured to:

recognize one or more words in the natural language utterance that define a navigation command in the navigation context;

recognize one or more additional words in the natural language utterance that define the approximated destination associated with the navigation command or the final destination successively refined in the managed dialogue; and

generate the dynamic recognition grammar based on a current location associated with the navigation device, the approximated destination associated with the navigation command, or the final destination successively refined in the managed dialogue.

34. The system of claim 31, wherein the navigation agent is further configured to dynamically recalculate the route in response to the one or more subsequent interactions refining the approximated destination.

35. The system of claim 34, wherein the navigation agent is configured to generate directions from the current location associated with the navigation device to the refined approximated destination to dynamically recalculate the route.

36. The system of claim 31, wherein the one or more subsequent interactions in the managed dialogue resolve the final destination substantially later in time relative to when the navigation agent initially calculated the route to the preliminary destination.

37. The system of claim 31, wherein to analyze the one or more preliminary interpretations to determine the preliminary destination, the conversational language processor is further configured to:

identify multiple destinations having addresses that possibly correspond to the full or partial address associated with the approximated destination;

rank the multiple identified destinations according to proximities to the current location associated with the navigation device or the full or partial address associated with the approximated destination; and

select one of the multiple identified destinations having a highest ranking to be the preliminary destination.

38. The system of claim 37, wherein the proximities depend on one or more topological domains that include physical, temporal, directional, and civil organizational proximities relative to the current location associated with the navigation device or the full or partial address associated with the approximated destination.

39. The system of claim 31, wherein the navigation agent is further configured to:

generate dynamic directions to provide a guide on the route from the current location associated with the navigation device to the preliminary destination; and

drive the dynamic directions using the information associated with the navigation context and the current location associated with the navigation device.

40. The system of claim 31, wherein the navigation agent is further configured to:

dynamically recalculate the route in response to the managed dialogue resolving the final destination; and

generate directions from the current location associated with the navigation device to the resolved final destination to dynamically recalculate the route.