Systems and methods for promoting driver engagement using active feedback

• US 10,586,454 B2
• Filed: 06/23/2017
• Issued: 03/10/2020
• Est. Priority Date: 05/05/2017
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Toyoda, Heishiro, Hamada, Hiroto
• Primary Examiner(s): Worden, Thomas E
• Application Number:
  US15/631,235
• Publication Number:
  US 20180322715A1
• Time in Patent Office: 991 Days
• CPC Class Codes: B60K 2360/186 Displaying information according to relevancy
• CPC Class Codes: B60K 2360/334 Projection means
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/29 Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions
• CPC Class Codes: B60W 2554/4046 Behavior, e.g. aggressive or erratic
• CPC Class Codes: B60W 30/08 Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: B60W 40/04 Traffic conditions
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B60W 60/00276 for two or more other traffic participants
• CPC Class Codes: B62D 15/029 Steering assistants using warnings or proposing actions to the driver without influencing the steering system
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/0101 characterised by optical features G02B27/0172 takes precedence
• CPC Class Codes: G06F 18/2415 based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G07C 5/0825 using optical means
• CPC Class Codes: G07C 5/0841 Registering performance data recording measured values G01D; information storage G11B
• CPC Class Codes: G08G 1/0112 : from the vehicle, e.g. floating car data [FCD]
• CPC Class Codes: G08G 1/0962 : having an indicator mounted inside the vehicle, e.g. giving voice messages
• CPC Class Codes: G08G 1/16 : Anti-collision systems road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08
• CPC Class Codes: G08G 1/166 : for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

System, methods, and other embodiments described herein relate to engaging a driver of a vehicle about driving behaviors. In one embodiment, a method includes computing predicted controls according to at least a defined skill level of the driver. The predicted controls indicate how to control the vehicle to maintain the vehicle along a driving path on a roadway. The method includes, in response to receiving manual control inputs from the driver, generating control feedback to the driver about the manual control inputs based, at least in part, on a difference between the manual control inputs and the predicted controls.

Claims

1. A feedback system for engaging a driver of a vehicle about driving behaviors, comprising:

one or more processors;

a memory communicably coupled to the one or more processors and storing;

a controls module including instructions that when executed by the one or more processors cause the one or more processors to compute predicted controls to maintain the vehicle along a driving path on a roadway, the driving path is defined according to a defined skill level of the driver in relation to a range of skill levels for controlling the vehicle differently along the roadway,

wherein the predicted controls indicate how to control the vehicle to perform maneuvers including steering, braking, and accelerator inputs corresponding with the defined skill level, and wherein the controls module includes instructions to compute the predicted controls for the driving path including instructions to generate the driving path to include the maneuvers within the defined skill level of the driver; and

a feedback module including instructions that when executed by the one or more processors cause the one or more processors to, in response to receiving manual control inputs from the driver, generate control feedback to the driver about the manual control inputs based, at least in part, on a difference between the manual control inputs and the predicted controls, wherein the feedback module includes instructions to generate the control feedback including instructions to selectively provide reinforcing feedback and negative feedback according to whether the difference satisfies a threshold variance,

wherein the controls module further includes instructions to determine a progression of the defined skill level of the driver over time according to at least a previous skill level of the driver, and

wherein the feedback module includes the instructions to generate the control feedback according to the difference and the progression of the defined skill level of the driver to induce the driver to improve the defined skill level.

2. The feedback system of claim 1, wherein the feedback module includes the instructions to generate the control feedback by providing the control feedback to a defined degree according to an extent of the difference between the manual control inputs and the predicted controls,

wherein the feedback module includes instructions to generate the control feedback intermittently according to a random timing, and wherein the feedback module includes instructions to vary the control feedback by generating the feedback with a varying intensity, and a varying timing.

3. The feedback system of claim 1, wherein the feedback module includes the instructions to generate the control feedback including instructions to blend the predicted controls with the manual control inputs according to an extent of the difference between the manual control inputs and the predicted controls to control the vehicle to avoid straying from the driving path and to inform the driver of the difference.

4. The feedback system of claim 1, wherein the feedback module includes the instructions to generate the control feedback including instructions to modify the manual control inputs by generating resistive feedback in the manual control inputs when the manual control inputs vary from the predicted controls.

5. The feedback system of claim 1, wherein the feedback module includes the instructions to generate the control feedback including instructions to control an input device of the vehicle to apply a resistive force in opposition to the manual control inputs using a force feedback mechanism in the vehicle to control the vehicle according to the predicted controls instead of the manual control inputs.

6. The feedback system of claim 1, wherein the controls module further includes instructions to:

collect sensor data from one or more sensors of the vehicle by electronically controlling the one or more sensors to at least semi-continuously scan a surrounding environment of the vehicle to detect objects and obstacles in the surrounding environment, and

wherein the sensor data includes environmental information about the surrounding environment of the vehicle including objects and obstacles in the surrounding environment.

7. The feedback system of claim 1, wherein the controls module further includes instructions to:

determine the defined skill level of the driver according to a log of previous driving activity about the driver using a machine learning algorithm to model the defined skill level for the driver.

8. The feedback system of claim 1, wherein the controls module includes the instructions to compute the predicted controls including instructions to generate the predicted controls as a function of the defined skill level and sensor data collected from one or more sensors of the vehicle, and,

wherein the threshold variance defines an extent of deviation from the predicted controls that is acceptable before generating the control feedback as the negative feedback, the predicted controls identify expected controls for the defined skill level of the driver for current driving conditions associated with the driving path along the roadway.

9. A non-transitory computer-readable medium for engaging a driver of a vehicle about driving behaviors and storing instructions that when executed by one or more processors cause the one or more processors to:

compute predicted controls to maintain the vehicle along a driving path on a roadway, the driving path is defined according to a defined skill level of the driver in relation to a range of skill levels for controlling the vehicle differently along the roadway,

wherein the predicted controls indicate how to control the vehicle to perform maneuvers including steering, braking, and accelerator inputs corresponding with the defined skill level, and wherein the instructions to compute the predicted controls for the driving path include instructions to generate the driving path to include the maneuvers within the defined skill level of the driver; and

in response to receiving manual control inputs from the driver, generate control feedback to the driver about the manual control inputs based, at least in part, on a difference between the manual control inputs and the predicted controls, wherein the instructions to generate the control feedback include instructions to selectively provide reinforcing feedback and negative feedback according to whether the difference satisfies a threshold variance,

wherein the instructions further include instructions to determine a progression of the defined skill level of the driver over time according to at least a previous skill level of the driver, and

wherein the instructions to generate the control feedback include instructions to generated the control feedback according to the difference and the progression of the defined skill level of the driver to induce the driver to improve the defined skill level.

10. The non-transitory computer-readable medium of claim 9, wherein the instructions to generate the control feedback include instructions to provide the control feedback to a defined degree according to an extent of the difference between the manual control inputs and the predicted controls,

wherein the instructions to generate the control feedback include instructions to modify the manual control inputs by generating resistive feedback in the manual control inputs when the manual control inputs vary from the predicted controls,

wherein the instructions to generate the control feedback include instructions to generate the control feedback intermittently according to a random timing, and

wherein the instructions to generate the control feedback include instructions to vary the control feedback by generating the control feedback with a varying intensity, and a varying timing.

11. The non-transitory computer-readable medium of claim 9, wherein the instructions to generate the control feedback include instructions to blend the predicted controls with the manual control inputs according to an extent of the difference between the manual control inputs and the predicted controls to control the vehicle to avoid straying from the driving path and to inform the driver of the difference.

12. The non-transitory computer-readable medium of claim 9, wherein the instructions to compute the predicted controls include instructions to generate the predicted controls as a function of the defined skill level and sensor data collected from one or more sensors of the vehicle,

wherein the threshold variance defines an extent of deviation from the predicted controls that is acceptable before generating the control feedback as the negative feedback, the predicted controls identify expected controls for the defined skill level of the driver for current driving conditions associated with the driving path along the roadway.

13. The non-transitory computer-readable medium of claim 9, wherein the instructions to generate the control feedback include instructions to control an input device of the vehicle to apply a resistive force in opposition to the manual control inputs using a force feedback mechanism in the vehicle to control the vehicle according to the predicted controls instead of the manual control inputs,

wherein the instructions further include instructions to determine a progression of the defined skill level of the driver according to at least a previous skill level of the driver, and wherein the instructions to generate the control feedback include instructions to generate the control feedback according to the difference and the progression of the defined skill level of the driver.

14. A method for engaging a driver of a vehicle about driving behaviors, comprising:

computing predicted controls to maintain the vehicle along a driving path on a roadway, the driving path is defined according to a defined skill level of the driver in relation to a range of skill levels for controlling the vehicle differently along the roadway,

wherein the predicted controls indicate how to control the vehicle to perform driving maneuvers including steering, braking, and accelerator inputs corresponding with the defined skill level, and wherein computing the predicted controls for the driving path includes generating the driving path to include the driving maneuvers within the defined skill level of the driver; and

in response to receiving manual control inputs from the driver, generating control feedback to the driver about the manual control inputs based, at least in part, on a difference between the manual control inputs and the predicted controls, wherein generating the control feedback includes selectively providing reinforcing feedback and negative feedback according to whether the difference satisfies a threshold variance,

wherein the defined skill level is determined as a progression of the defined skill level over time according to at least a previous skill level of the driver, and

wherein generating the control feedback includes generating the control feedback according to the difference and the progression of the defined skill level of the driver to induce the driver to improve the defined skill level.

15. The method of claim 14, wherein generating the control feedback includes modifying the manual control inputs according to an extent of the difference between the manual control inputs and the predicted controls and by generating resistive feedback in the manual control inputs when the manual control inputs satisfy the threshold variance from the predicted controls, wherein generating the control feedback includes generating the control feedback intermittently according to a random timing,

wherein generating the control feedback includes varying the control feedback by generating the control feedback with a varying intensity, and a varying timing, and

wherein computing the predicted controls includes computing the driving path according to the defined skill level and a manner of operating the vehicle that optimizes safety.

16. The method of claim 14, wherein computing the predicted controls according to the skill level of the driver includes generating the predicted controls for the driving path as a function of the defined skill level and sensor data collected from one or more sensors of the vehicle,

wherein the threshold variance defines an extent of deviation from the predicted controls that is acceptable before generating the control feedback as the negative feedback, the predicted controls identifying expected controls for the defined skill level of the driver for current driving conditions associated with the driving path along the roadway.

17. The method of claim 14, wherein generating the control feedback includes blending the predicted controls with the manual control inputs according to an extent of the difference between the manual control inputs and the predicted controls to avoid the vehicle from straying from the driving path.

18. The method of claim 14, wherein generating the control feedback includes controlling an input device of the vehicle to push back against the manual control inputs using a force feedback mechanism in the vehicle to control the vehicle according to the predicted controls instead of the manual control inputs.

19. The method of claim 14, further comprising:

collecting sensor data from one or more sensors of the vehicle by electronically controlling the one or more sensors to at least semi-continuously scan a surrounding environment of the vehicle to detect objects and obstacles in the surrounding environment, wherein the sensor data includes environmental information about the surrounding environment of the vehicle including objects and obstacles in the surrounding environment.

20. The method of claim 14, further comprising:

determining the defined skill level of the driver according to a log of previous driving activity about the driver using a machine learning algorithm to model the defined skill level for the driver.