VEHICLE PERSONAL ASSISTANT

• US 20140136013A1
• Filed: 11/15/2012
• Published: 05/15/2014
• Est. Priority Date: 11/15/2012
• Status: Active Grant

• Current Assignee: SRI International, Inc.
• Original Assignee: SRI International, Inc.
• Inventors: Wolverton, Michael J., Mark, William S., Bratt, Harry, Bercow, Douglas A.
• Granted Patent: US 9,085,303 B2
• Time in Patent Office: Days
• US Class Current: 701/1
• CPC Class Codes: B60K 2360/146 Instrument input by gesture
• CPC Class Codes: B60K 2360/148 Instrument input by voice
• CPC Class Codes: B60K 2360/161 Explanation of functions, e.g. instructions
• CPC Class Codes: B60K 2360/165 Videos and animations
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60W 2040/089 Driver voice
• CPC Class Codes: B60W 50/085 Changing the parameters of the control units, e.g. changing limit values, working points by control input
• CPC Class Codes: B60W 50/10 Interpretation of driver requests or demands
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

A vehicle personal assistant to engage a user in a conversational dialog about vehicle-related topics, such as those commonly found in a vehicle owner'"'"'s manual, includes modules to interpret spoken natural language input, search a vehicle knowledge base and/or other data sources for pertinent information, and respond to the user'"'"'s input in a conversational fashion. The dialog may be initiated by the user or more proactively by the vehicle personal assistant based on events that may be currently happening in relation to the vehicle. The vehicle personal assistant may use real-time inputs obtained from the vehicle and/or non-verbal inputs from the user to enhance its understanding of the dialog and assist the user in a variety of ways.

Claims

1. A vehicle personal assistant embodied in one or more machine readable storage media, the vehicle personal assistant executable by a computing system to:

receive real-time sensor inputs relating to a current state of a vehicle;

use one or more of the real-time sensor inputs to determine a current context of human activity in relation to the vehicle; and

determine a style of presentation of output based on the current context.

2. The vehicle personal assistant of claim 1, executable by a computing system to determine the current context by comparing the real-time sensor inputs to a vehicle-specific knowledge base.

3. The vehicle personal assistant of claim 1, wherein the real-time sensor inputs comprise one or more of accelerating, braking and steering inputs.

4. The vehicle personal assistant of claim 3, executable by a computing system to determine the current context by comparing the real-time sensor inputs to a stored template.

5. The vehicle personal assistant of claim 4, executable by a computing system to compare the current context to a stored template and, based on the comparison, determine an aspect of driving behavior.

6. The vehicle personal assistant of claim 5, executable by a computing system to determine the output based on the aspect of driving behavior.

7. The vehicle personal assistant of claim 5, executable by a computing system to determine the presentation style of the output based on the aspect of driving behavior.

8. The vehicle personal assistant of claim 7, executable by a computing system to determine the presentation style by comparing the real-time sensor inputs to a stored rule corresponding to the aspect of driving behavior.

9. The vehicle personal assistant of claim 1, wherein determining the style of the presentation of output comprises selecting one or more of conversational spoken natural language, text, and recorded media.

10. The vehicle personal assistant of claim 1, executable by the computing system to present the output using the style of presentation.

11. A vehicle personal assistant embodied in one or more machine readable storage media, the vehicle personal assistant executable by a computing system to:

monitor one or more real-time sensor inputs relating to the vehicle;

compare the real-time sensor inputs to stored data corresponding to a vehicle-related condition to determine whether the real-time sensor inputs indicate that the vehicle-related condition has occurred; and

in response to determining that the vehicle-related condition has occurred, initiate a real-time natural language conversation between the vehicle personal assistant and a person by presenting system-generated output comprising information about the vehicle-related condition.

12. The vehicle personal assistant of claim 11, executable by the computing system to present machine-generated spoken natural language output at a speaker of the vehicle in response to determining that the vehicle-related condition has occurred.

13. The vehicle personal assistant of claim 11, executable by the computing system to display one or more of text, images, graphics, and video on a display of the computing system in response to determining that the vehicle-related condition has occurred.

14. The vehicle personal assistant of claim 11, executable by the computing system to present a suggested action to be performed by the person in relation to the vehicle in response to determining that the vehicle-related condition has occurred.

15. The vehicle personal assistant of claim 11, executable by the computing system to communicate data relating to the vehicle-related condition to another computing system in response to determining that the vehicle-related condition has occurred.

16. The vehicle personal assistant of claim 11, executable by the computing system to receive the real-time sensor inputs from a vehicle network of the vehicle.

17. The vehicle personal assistant of claim 11, wherein the stored data comprises a value associated with the occurrence of the vehicle-related condition, and the vehicle personal assistant is executable by the computing system to compare the real-time sensor inputs to the value.

18. The vehicle personal assistant of claim 11, wherein the stored data relates to a user-specified driving situation associated with the occurrence of the vehicle-related condition, and the vehicle personal assistant is executable by the computing system to compare the real-time sensor inputs to the user-specified driving situation.

19. The vehicle personal assistant of claim 11, executable by the computing system to receive human-generated input in response to the system-generated output and present further system-generated output relating to the conversation in response to the human-generated input.

20. The vehicle personal assistant of claim 11, executable by the computing system to receive human-generated input in response to the system-generated output and continue the conversation based on the human-generated input.

21. The vehicle personal assistant of claim 11, executable by the computing system to monitor the real-time sensor inputs and compare the real-time sensor inputs to the stored data while the vehicle is turned on.

22. The vehicle personal assistant of claim 11, executable by the computing system to access one or more third-party data sources to determine whether the real-time sensor inputs indicate that the vehicle-related condition has occurred.

23. A vehicle personal assistant embodied in one or more machine readable storage media, the vehicle personal assistant executable by a computing system to:

receive real-time human-generated conversational spoken language input from a vehicle user, the real-time human-generated conversational spoken language input comprising a vehicle-related triggering condition that may occur during the operation of the vehicle and an action to be performed by the vehicle personal assistant if the vehicle-related triggering condition occurs during the operation of the vehicle;

monitor one or more vehicle-related real-time sensor inputs;

apply a vehicle-specific knowledge base to the real-time sensor inputs to determine whether the vehicle-related condition has occurred; and

in response to determining that the vehicle-related triggering condition has occurred, execute the action.

24. The vehicle personal assistant of claim 23, executable by the computing system to apply the vehicle-specific knowledge base to the human-generated input to interpret the vehicle-related triggering condition.

25. The vehicle personal assistant of claim 23, executable by the computing system to determine a current context based on the vehicle-related real-time sensor inputs and, in response to determining that the vehicle-related condition has occurred, present the system-generated output using a presentation style based on the current context.

26. The vehicle personal assistant of claim 25, executable by the computing system to determine the current context by comparing the vehicle-related real-time sensor inputs to a stored template.

27. The vehicle personal assistant of claim 23, executable by the computing system to compare the vehicle-related real-time sensor inputs to the vehicle-related condition while the vehicle is in operation.

28. The vehicle personal assistant of claim 23, executable by the computing system to receive the vehicle-related real-time sensor inputs from a vehicle network of the vehicle.

29. The vehicle personal assistant of claim 23, executable by the computing system to, in response to determining that the vehicle-related triggering condition has occurred, present machine-generated conversational spoken language output relating to the vehicle-related triggering condition.

30. The vehicle personal assistant of claim 23, executable by the computing system to receive further human-generated input in response to the system-generated output and determine whether to modify the vehicle-related triggering condition based on the further human-generated input.

31. The vehicle personal assistant of claim 23, wherein the further human-generated input comprises one or more of conversational spoken language input and non-verbal input.

32. The vehicle personal assistant of claim 23, executable by the computing system to present further system-generated output in response to the further human-generated input.