VEHICLE PERSONAL ASSISTANT

• US 20140136187A1
• Filed: 11/15/2012
• Published: 05/15/2014
• Est. Priority Date: 11/15/2012
• Status: Active Grant

• Current Assignee: SRI International, Inc.
• Original Assignee: SRI International, Inc.
• Inventors: Wolverton, Michael J., Mark, William S., Bratt, Harry, Bercow, Douglas A.
• Granted Patent: US 9,798,799 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/226 using non-speech characteristics
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A vehicle personal assistant to engage a user in a conversational dialog about vehicle-related topics, such as those commonly found in a vehicle owner'"'"'s manual, includes modules to interpret spoken natural language input, search a vehicle knowledge base and/or other data sources for pertinent information, and respond to the user'"'"'s input in a conversational fashion. The dialog may be initiated by the user or more proactively by the vehicle personal assistant based on events that may be currently happening in relation to the vehicle. The vehicle personal assistant may use real-time inputs obtained from the vehicle and/or non-verbal inputs from the user to enhance its understanding of the dialog and assist the user in a variety of ways.

Claims

1. A conversational spoken natural language-enabled interactive vehicle user'"'"'s guide embodied in one or more machine readable storage media and executable by a computing system to:

receive human-generated spoken natural language input relating to a component of a vehicle, the human-generated spoken natural language input relating to one or more of;

factual information about the vehicle component, the operation of the vehicle component, and a vehicle-related event currently or recently occurring in the vehicle;

interpret the human-generated spoken natural language input based on other received human-generated input;

conduct a search based on the interpreted human-generated spoken natural language input using a vehicle-specific knowledge base, the vehicle-specific knowledge base comprising factual information about the vehicle; and

based on a result of the search, present a response to the human-generated spoken natural language input in a conversational fashion.

2. The conversational spoken natural language-enabled interactive vehicle user'"'"'s guide of claim 1, executable by the computing system to determine whether the human-generated spoken natural language input relates to a dialog comprising one or more previously-received human-generated spoken natural language inputs and, in response to determining that the human-generated spoken natural language input relates to the dialog, interpret the human-generated spoken natural language input based on one or more of the previously-received human-generated spoken natural language inputs.

3. The conversational spoken natural language-enabled interactive vehicle user'"'"'s guide of claim 1, executable by the computing system to convert the received human-generated conversational spoken natural language input into a machine-readable format and interpret the converted human-generated conversational spoken natural language input using a vehicle-specific conversation model.

4. The conversational spoken natural language-enabled interactive vehicle user'"'"'s guide of claim 1, executable by the computing system to receive real-time vehicle-related sensor input from the vehicle and interpret the human-generated conversational spoken natural language input based on the real-time vehicle-related sensor input.

5. The conversational spoken natural language-enabled interactive vehicle user'"'"'s guide of claim 1, executable by the computing system to determine whether the human-generated conversational spoken natural language input comprises a request for information;

in response to determining that the human-generated conversational spoken natural language input comprises a request for information, classify the request for information as one of a plurality of question types; and

respond to the question based on the question type.

6. The conversational spoken natural language-enabled interactive vehicle user'"'"'s guide of claim 1, executable by the computing system to determine whether the received human-generated conversational spoken natural language input comprises an instruction and, in response to determining that the human-generated conversational spoken natural language input comprises an instruction, perform an action.

7. The spoken natural language-enabled interactive vehicle user'"'"'s guide of claim 1, wherein the vehicle-specific knowledge base comprises geospatial tags associated with data relating to components of the vehicle, and the spoken natural language-enabled interactive vehicle user'"'"'s guide is executable by the computing system to interpret the human-generated conversational spoken natural language input based on the geospatial tags.

8. The spoken natural language-enabled interactive vehicle user'"'"'s guide of claim 1, executable by the computing system to respond to the human-generated conversational spoken natural language input by presenting factual information corresponding to the particular make, model, and options of the vehicle in a conversational fashion without presenting information that is not applicable to the particular make, model and options of the vehicle.

9. The spoken natural language-enabled interactive vehicle user'"'"'s guide of claim 1, executable by the computing system to receive non-verbal user-generated input comprising one or more of a gesture, a gaze, a touch, a facial expression and user-generated digital media, interpret the non-verbal user-generated input, and interpret the human-generated conversational spoken natural language input based on the interpreted non-verbal user-generated input.

10. The spoken natural language-enabled interactive vehicle user'"'"'s guide of claim 1, executable by the computing system to receive further human-generated input relating to the human-generated conversational spoken natural language input and provide a response to the further human-generated input.

11. The spoken natural language-enabled interactive vehicle user'"'"'s guide of claim 1, executable by the computing system to determine whether further human-generated input is needed to respond to the human-generated conversational spoken natural language input and solicit further human-generated input in response to determining that further human-generated input is needed.

12. The spoken natural language-enabled interactive vehicle user'"'"'s guide of claim 1, executable by the computing system to receive real-time vehicle-related sensor inputs and vary the response to the human-generated conversational spoken natural language input based on the real-time vehicle-related sensor inputs.

13. The spoken natural language-enabled interactive vehicle user'"'"'s guide of claim 12, executable by the computing system to determine, based on the real-time vehicle-related sensor inputs, whether the vehicle is turned on, and vary the response to the human-generated conversational spoken natural language input based on whether the vehicle is turned on.

14. The spoken natural language-enabled interactive vehicle user'"'"'s guide of claim 12, executable by the computing system to determine, based on the real-time vehicle-related sensor inputs, whether the vehicle is moving, and vary the response to the human-generated conversational spoken natural language input based on whether the vehicle is moving.

15. The spoken natural language-enabled interactive vehicle user'"'"'s guide of claim 1, executable by the computing system to receive real-time vehicle-related sensor inputs and vary the manner in which the response to the human-generated conversational spoken natural language input is presented based on the real-time vehicle-related sensor inputs.

16. The spoken natural language-enabled interactive vehicle user'"'"'s guide of claim 15, executable by the computing system to determine, based on the real-time vehicle-related sensor inputs, whether the vehicle is turned on, and vary the manner in which the response to the human-generated conversational spoken natural language input is presented based on whether the vehicle is turned on.

17. The spoken natural language-enabled interactive vehicle user'"'"'s guide of claim 14, executable by the computing system to determine, based on the real-time vehicle-related sensor inputs, whether the vehicle is moving, and vary the manner in which the response to the human-generated conversational spoken natural language input is presented based on whether the vehicle is moving.

18. The spoken natural language-enabled interactive vehicle user'"'"'s guide of claim 1, executable by the computing system to determine, based on the human-generated conversational spoken natural language input, whether to access an approved third-party data source or a non-approved third-party data source to conduct the search.

19. The spoken natural language-enabled interactive vehicle user'"'"'s guide of claim 1, executable by the computing system to respond to the human-generated conversational spoken natural language input by presenting a combination of machine-generated spoken natural language output and visual output.

20. A vehicle personal assistant embodied in one or more machine readable storage media, the vehicle personal assistant executable by a computing system to:

receive human-generated conversational spoken natural language input relating to a component of a vehicle;

receive real-time sensor input relating to a current state of the vehicle component;

use the real-time sensor input to interpret the human-generated input;

apply a vehicle-specific knowledge base to generate a reply to the interpreted human-generated input, the reply comprising information relating to the vehicle component; and

present the reply in response to the human-generated input.

21. The vehicle personal assistant of claim 20, executable by a computing system to determine a current vehicle-related context of the human-generated input based on the real-time sensor input.

22. The vehicle personal assistant of claim 21, executable by a computing system to determine the current vehicle-related context by comparing the real-time sensor input to a stored vehicle context template.

23. The vehicle personal assistant of claim 21, executable by a computing system to interpret the human-generated input based on the current vehicle-related context.

24. The vehicle personal assistant of claim 21, executable by a computing system to determine whether to search another data source based on the current vehicle-related context.

25. The vehicle personal assistant of claim 21, executable by a computing system to select a reply from a plurality of possible replies based on the current vehicle-related context.

26. The vehicle personal assistant of claim 21, executable by the computing system to select a presentation mode from a plurality of possible presentation modes based on the current vehicle-related context and present the reply using the selected presentation mode.

27. The vehicle personal assistant of claim 26, wherein the plurality of possible presentation modes comprises two or more of:

machine-generated conversational spoken natural language, text, recorded audio, recorded video, and digital images.

28. The vehicle personal assistant of claim 21, executable by the computing system to receive further human-generated input in response to the reply and determine whether to generate another reply based on the further human-generated input and the current vehicle-related context.

29. The vehicle personal assistant of claim 28, wherein the further human-generated input comprises one or more of conversational spoken language input and non-verbal input comprising one or more of a gesture, a gaze, a touch, and a facial expression, and the vehicle personal assistant is executable by the computing system to interpret the further human-generated input based on the current vehicle-related context.

30. The vehicle personal assistant of claim 28, executable by the computing system to determine a current vehicle-related context of the further human-generated input based on the real-time sensor input, and interpret the further human-generated input based on the current vehicle-related context of the further human-generated input.

31. The vehicle personal assistant of claim 20, wherein the real-time sensor input indicates a current status of a feature of the vehicle.

32. The vehicle personal assistant of claim 20, wherein the real-time sensor input provides information about a current driving situation of the vehicle, including one or more of:

vehicle location, vehicle speed, vehicle acceleration, fuel status, and weather information.

33. The vehicle personal assistant of claim 20, wherein the vehicle-specific knowledge base provides information about the make, model, features or functions of the vehicle, and the vehicle personal assistant is executable by the computing system to omit from the reply information that is not applicable to the make, model, features, or functions of the vehicle.

34. The vehicle personal assistant of claim 33, executable by the computing system to use the vehicle-specific knowledge base to interpret the human-generated input.

35. The vehicle personal assistant of claim 20, executable by a computing system to obtain the real-time sensor input from a vehicle network of the vehicle.

36. The vehicle personal assistant of claim 20, wherein the real-time sensor input relates to a current status of one or more of an engine, a transmission, a safety feature, a security feature, a navigation feature, a climate control feature, an entertainment feature, an instrument panel-mounted feature, a steering-wheel mounted feature, and an environment feature of the vehicle.

37. The vehicle personal assistant of claim 20, executable by the computing system to calculate a derived value based on the real-time sensor input.