Driver familiarity adapted explanations for proactive automated vehicle operations

• US 9,376,117 B1
• Filed: 03/23/2015
• Issued: 06/28/2016
• Est. Priority Date: 03/23/2015
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Inventors: Parundekar, Rahul, Agosta, John Mark
• Primary Examiner(s): Beaulieu, Yonel
• Application Number:
  US14/665,598
• Time in Patent Office: 463 Days
• Field of Search: 434 64- 66, 701/1
• US Class Current: 1/1
• CPC Class Codes: B60W 2556/10 Historical data
• CPC Class Codes: B60W 50/087 where the control system corrects or modifies a request from the driver

Abstract

The disclosure includes a system, method and tangible memory for providing a driver familiarity adapted explanation to a driver of a vehicle. The system includes a sensor set configured to measure sensor data describing a task and the driver. The system also includes an onboard vehicle computer including a tangible memory, a processor, a task tracker and a familiarity application. The task tracker causes the processor to determine a vehicle operation to be proactively performed by the task tracker to autocomplete the task. The familiarity application causes the processor to execute steps including estimating how familiar the driver is with the vehicle operation based on the sensor data describing the driver of the vehicle, retrieving a template explanation associated with the vehicle operation, and adapting the template explanation to generate a driver familiarity adapted explanation describing the autocomplete of the vehicle operation.

Claims

1. A system of a vehicle comprising:

a sensor set including one or more sensors configured to measure sensor data describing a task and a driver of the vehicle;

an onboard vehicle computer including a tangible memory, a processor, a task tracker and a familiarity application, wherein the task tracker and the familiarity application are stored in the tangible memory which is communicatively coupled to the processor;

wherein the task tracker, responsive to being executed by the processor, causes the processor to determine, based on the sensor data describing the task, a vehicle operation to be proactively performed by the task tracker to autocomplete the task for the driver of the vehicle;

wherein the familiarity application, responsive to being executed by the processor, causes the processor to;

estimate how familiar the driver is with the vehicle operation based on the sensor data describing the driver of the vehicle;

retrieve a template explanation associated with the vehicle operation;

adapt the template explanation, based on the estimate of how familiar the driver is with the vehicle operation, to generate a driver familiarity adapted explanation describing the autocomplete of the vehicle operation; and

provide the driver familiarity adapted explanation to the driver.

2. The system of claim 1, wherein the sensor set includes one or more in-vehicle cameras and the sensor data describes an image of the driver.

3. The system of claim 2, wherein the steps further comprise:

analyzing the image to determine a facial expression of the driver; and

wherein the estimate of how familiar the driver is with the vehicle operation is based at least in part on the facial expression of the driver.

4. The system of claim 2, wherein the steps further comprise:

analyzing the image to determine an eye gaze of the driver; and

wherein the estimate of how familiar the driver is with the vehicle operation is based at least in part on the eye gaze of the driver.

5. The system of claim 2, wherein the steps further comprise:

analyzing the image to determine a pupil dilation of the driver; and

wherein the estimate of how familiar the driver is with the vehicle operation is based at least in part on the pupil dilation of the driver.

6. The system of claim 1, wherein the sensor set includes one or more microphones and the sensor data describes one or more vocalizations of the driver.

7. The system of claim 6, wherein the steps further comprise:

analyzing the one or more vocalizations to determine whether the one or more vocalizations are related to how familiar the driver is with the vehicle operation; and

wherein the estimate of how familiar the driver is with the vehicle operation is based at least in part on the analysis of the one or more vocalizations.

8. The system of claim 6, wherein the steps further comprise:

analyzing the one or more vocalizations to determine whether the one or more vocalizations include an express instruction of the driver related to adapting the template explanation; and

wherein the adaption of the template explanation is based at least in part on the express instruction of the driver.

9. The system of claim 1, wherein the estimate of how familiar the driver is with the vehicle operation is based at least in part on a count of how many times the task tracker has proactively performed the task.

10. The system of claim 1, wherein the estimate of how familiar the driver is with the vehicle operation is based at least in part on a count of how many times the task tracker has proactively performed the vehicle operation.

11. The system of claim 10, wherein the count indicates a first instance of the task tracker proactively performing the vehicle operation and the estimate of how familiar the driver is with the vehicle operation is based on an expected familiarity with the vehicle operation for a typical driver.

12. The system of claim 2, wherein the steps further comprise analyzing the image of the driver to determine an identity of the driver and wherein the estimate of how familiar the driver is with the vehicle operation is based at least in part on a count of how many times the task tracker has performed the autocomplete of the task for the identified driver.

13. The system of claim 2, wherein the steps further comprise analyzing the image of the driver to determine an identity of the driver and wherein the estimate of how familiar the driver is with the vehicle operation is based at least in part on a count of how many times the task tracker has proactively performed the vehicle operation for the identified driver.

14. The system of claim 1, wherein the familiarity application includes two or more template explanations for the vehicle operation and wherein the steps further comprise:

determining which of the two or more template explanations were not used in a most recent performance of the vehicle operation by the task tracker;

forming a set of one or more template explanations that were not used in the most recent performance of the vehicle operation by the task tracker; and

wherein the retrieving step includes retrieving the template explanation from the set of one or more template explanations that were not used in the most recent performance of the vehicle operation by the task tracker.

15. The system of claim 1, wherein the estimating step includes estimating how familiar the driver is with the vehicle operation based on familiarity data describing how familiar the driver is with the vehicle operation based on a past experience of the driver with the vehicle operation.

16. The system of claim 15, wherein the familiarity data indicates that the driver is familiar with the vehicle operation, the sensor data indicates that the driver is not familiar with the vehicle operation, and the estimating step includes determining that the driver is not familiar with the vehicle operation based on the sensor data.

17. The system of claim 16, wherein the sensor data is measured in real time.

18. The system of claim 1, wherein the sensor data is measured substantially contemporaneous to the task tracker performing the vehicle operation.

19. A method comprising:

measuring, using a sensor set including one or more in-vehicle sensors, sensor data describing a task and a driver of a vehicle;

estimating how familiar the driver is with a vehicle operation based on the sensor data describing the driver of the vehicle;

retrieving, from a tangible memory communicatively coupled to a processor, a template explanation associated with the vehicle operation;

adapting the template explanation using the processor, wherein the adaptation of the template explanation is based on the estimate of how familiar the driver is with the vehicle operation, to generate a driver familiarity adapted explanation describing an autocomplete of the vehicle operation in a detail proportionate to the estimate of how familiar the driver is with the vehicle operation; and

providing the driver familiarity adapted explanation to the driver.

20. A tangible memory of a mobile device comprising computer code which, when executed by an electronic device, causes the electronic device to perform steps comprising:

measuring, using a sensor set including one or more in-vehicle sensors, sensor data describing a task and a driver of a vehicle;

estimating how familiar the driver is with a vehicle operation based on the sensor data describing the driver of the vehicle;

retrieving, from the tangible memory, a template explanation associated with the vehicle operation;

adapting the template explanation based on the estimate of how familiar the driver is with the vehicle operation, to generate a driver familiarity adapted explanation describing an autocomplete of the vehicle operation in a detail proportionate to the estimate of how familiar the driver is with the vehicle operation; and

providing the driver familiarity adapted explanation to the driver.