System and method for providing in-vehicle services via a natural language voice user interface

• US 10,347,248 B2
• Filed: 04/03/2017
• Issued: 07/09/2019
• Est. Priority Date: 12/11/2007
• Status: Active Grant

• Current Assignee: Cerence Inc.
• Original Assignee: VoiceBox Technologies Corporation (Microsoft Corporation)
• Inventors: Kennewick, Michael R., Cheung, Catherine, Baldwin, Larry, Salomon, Ari, Tjalve, Michael, Guttigoli, Sheetal, Armstrong, Lynn, Di Cristo, Philippe, Zimmerman, Bernie, Menaker, Sam
• Primary Examiner(s): Vo, Huyen X
• Application Number:
  US15/477,179
• Publication Number:
  US 20170270925A1
• Time in Patent Office: 827 Days
• Field of Search: 704 1- 10, 704230-277
• CPC Class Codes: G01C 21/3608 using speech input, e.g. using speech recognition
• CPC Class Codes: G06Q 30/0261 based on user location
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 15/193 Formal grammars, e.g. finite state automata, context free grammars or word networks
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

A conversational, natural language voice user interface may provide an integrated voice navigation services environment. The voice user interface may enable a user to make natural language requests relating to various navigation services, and further, may interact with the user in a cooperative, conversational dialogue to resolve the requests. Through dynamic awareness of context, available sources of information, domain knowledge, user behavior and preferences, and external systems and devices, among other things, the voice user interface may provide an integrated environment in which the user can speak conversationally, using natural language, to issue queries, commands, or other requests relating to the navigation services provided in the environment.

Claims

1. A method for providing in-vehicle services via a natural language voice user interface, the method being implemented by an in-vehicle telematics system comprising a plurality of in-vehicle devices and one or more physical processors programmed with computer program instructions that, when executed, program the in-vehicle telematics system to perform the method, the method comprising:

connecting, by the in-vehicle telematics system, to a mobile device separate from the plurality of in-vehicle devices;

receiving, by the in-vehicle telematics system, a first natural language utterance;

determining, by the in-vehicle telematics system, a first intent of the first natural language utterance;

determining, by the in-vehicle telematics system, that the first intent relates to a first request to be handled by the mobile device;

transmitting, by the in-vehicle telematics system, the first request to the mobile device;

receiving, by the in-vehicle telematics system, a second natural language utterance;

determining, by the in-vehicle telematics system, a second intent of the second natural language utterance;

determining, by the in-vehicle telematics system, that the second intent relates to a second request to be handled by an in-vehicle device selected from among the plurality of in-vehicle devices;

transmitting, by the in-vehicle telematics system, the second request to the in-vehicle device; and

in the case one of (i) insufficient information is available to the mobile device to perform the first request, or (ii) insufficient information is available to the in-vehicle device to perform the second request, generating and transmitting by the vehicle telematics system a third request for additional information to one of a networked search engine or an external service provider.

2. The method of claim 1, the method further comprising:

obtaining, by the in-vehicle telematics system, second information indicating one or more services provided by the plurality of in-vehicle devices; and

selecting, by the in-vehicle telematics system, the in-vehicle device based on the second information.

3. The method of claim 1, wherein obtaining the first intent of the first natural language utterance comprises:

providing, by the in-vehicle telematics system, the first natural language utterance as an input to a speech recognition engine;

in response to receiving words or phrases, recognized from the first natural language utterance, as an output of the speech recognition engine, providing, by the in-vehicle telematics system, the words or phrases as an input to a conversational language processor; and

interpreting, by the in-vehicle telematics system, the first natural language utterance based on the provision of the words or phrases as input to the conversational language processor to determine the first intent of the first natural language utterance.

4. The method of claim 1, wherein the first request relates to initiating a phone call, wherein transmitting the first request to the mobile device comprises:

causing, by the in-vehicle telematics system, the first request to be sent to the mobile phone to initiate the phone call based on the first request.

5. The method of claim 1, wherein the second request relates to providing guidance from a first location to a destination, and wherein the in-vehicle device is a navigation device.

6. The method of claim 5, the method further comprising:

obtaining, by the in-vehicle telematics system, an indication that the gas in the vehicle is below a threshold amount; and

calculating, by the in-vehicle telematics system, a route to the destination based on the indication that the gas in the vehicle is a below the threshold amount.

7. The method of claim 5, the method further comprising:

recalculating, by the in-vehicle telematics system, a route to the destination in response to one or more subsequent interactions of the user that indicate refinement of the route and/or destination.

8. The method of claim 1, wherein the second request relates to playing music, and wherein the in-vehicle device is a radio.

9. The method of claim 1, wherein the additional information relates to movie show times, wherein transmitting the third request to one of the networked search engine or the external service provider comprises:

causing, by the in-vehicle telematics system, the networked search engine to search for movie show times.

10. The method of claim 1, the method further comprising:

causing, by the in-vehicle telematics system, display information related to the first request to be displayed on a display device, wherein the display information indicates that a service related to the first request has been performed.

11. An in-vehicle telematics system for providing in-vehicle services via a natural language voice user interface, the in-vehicle telematics system comprising:

a plurality of in-vehicle devices and one or more physical processors programmed with computer program instructions that, when executed, program the in-vehicle telematics system to;

connect to a mobile device separate from the plurality of in-vehicle devices;

receive a first natural language utterance;

determine a first intent of the first natural language utterance;

determine that the first intent relates to a first request to be handled by the mobile device;

transmit the first request to the mobile device;

receive a second natural language utterance;

determine a second intent of the second natural language utterance;

determine that the second intent relates to a second request to be handled by an in-vehicle device selected from among the plurality of in-vehicle devices;

transmit the second request to the in-vehicle device; and

in the case one of (i) insufficient information is available to the mobile device to perform the first request, or (ii) insufficient information is available to the in-vehicle device to perform the second request, generate and transmit by the vehicle telematics system a third request for additional information to one of a networked search engine or an external service provider.

12. The in-vehicle telematics system of claim 11, wherein the in-vehicle telematics system is further programmed to:

obtain second information indicating one or more services provided by the plurality of in-vehicle devices; and

select the in-vehicle device based on the second information.

13. The in-vehicle telematics system of claim 11, wherein to obtain the first intent of the first natural language utterance, the in-vehicle telematics system is further programmed to:

provide the first natural language utterance as an input to a speech recognition engine;

in response to receiving words or phrases, recognized from the first natural language utterance, as an output of the speech recognition engine, provide the words or phrases as an input to a conversational language processor; and

interpret the first natural language utterance based on the provision of the words or phrases as input to the conversational language processor to determine the first intent of the first natural language utterance.

14. The in-vehicle telematics system of claim 11, wherein the first request relates to initiating a phone call, wherein to transmit the first request to the mobile device, the in-vehicle telematics system is further programmed to:

cause the first request to be sent to the mobile phone to initiate the phone call based on the first request.

15. The in-vehicle telematics system of claim 11, wherein the second request relates to providing guidance from a first location to a destination, and wherein the in-vehicle device is a navigation device.

16. The in-vehicle telematics system of claim 15, wherein the in-vehicle telematics system is further programmed to:

obtain an indication that the gas in the vehicle is below a threshold amount; and

calculate a route to the destination based on the indication that the gas in the vehicle is a below the threshold amount.

17. The in-vehicle telematics system of claim 15, wherein the in-vehicle telematics system is further programmed to:

recalculate a route to the destination in response to one or more subsequent interactions of the user that indicate refinement of the route and/or destination.

18. The in-vehicle telematics system of claim 11, wherein the second request relates to playing music, and wherein the in-vehicle device is a radio.

19. The in-vehicle telematics system of claim 11, wherein the additional information relates to movie show times, wherein to transmit the third request to one of the networked search engine or the external service provider, the in-vehicle telematics system is further programmed to:

cause the networked search engine to search for movie show times.

20. The in-vehicle telematics system of claim 11, wherein the in-vehicle telematics system is further programmed to:

cause display information related to the first request to be displayed on a display device, wherein the display information indicates that a service related to the first request has been performed.