Integrated framework for vehicle operator assistance based on a trajectory prediction and threat assessment

• US 8,437,890 B2
• Filed: 07/15/2010
• Issued: 05/07/2013
• Est. Priority Date: 03/05/2009
• Status: Active Grant

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Anderson, Sterling J., Peters, Steven C., Iagnemma, Karl D.
• Primary Examiner(s): Tarcza, Thomas
• Assistant Examiner(s): Dunn, Alex C
• Application Number:
  US13/254,761
• Publication Number:
  US 20120083947A1
• Time in Patent Office: 1,027 Days
• Field of Search: 701/3, 701/45
• US Class Current: 701/3
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 50/0098 Details of control systems ensuring comfort, safety or stability not otherwise provided for
• CPC Class Codes: B60W 50/10 Interpretation of driver requests or demands
• CPC Class Codes: G05D 1/00 Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots drive control systems specially adapted for autonomous road vehicles B60W60/00
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

Various types and levels of operator assistance are performed within a unified, configurable framework. A model of the device with a model of the environment and the current state of the device and the environment are used to iteratively generate a sequence of optimal device control inputs that, when applied to a model of the device, generate an optimal device trajectory through a constraint-bounded corridor or region within the state space. This optimal trajectory and the sequence of device control inputs that generates it is used to generate a threat assessment metric. An appropriate type and level of operator assistance is generated based on this threat assessment. Operator assistance modes include warnings, decision support, operator feedback, vehicle stability control, and autonomous or semi-autonomous hazard avoidance. The responses generated by each assistance mode are mutually consistent because they are generated using the same optimal trajectory.

Claims

1. A method for generating a threat response, related to threat posed to a mechanical device, comprising:

using a computer control system, determining an optimal sequence of predicted device controller inputs based on at least a model of the device and an objective function, the optimal sequence of predicted device controller inputs being determined so as to optimize predicted performance of the model over a time horizon over which the device model is simulated and satisfy device dynamic constraints, device input constraints, and environmental constraints;

using the computer control system, determining an optimal sequence of predicted device states resulting from applying the sequence of optimal device inputs to the model of the device;

using the computer control system, combining at least one of;

the predicted device states and the predicted device controller inputs, into a scalar threat assessment metric based at least on predicted stability of the device through the time horizon as a function of the at least one of the predicted device states and the predicted device controller inputs; and

,

using the computer control system, determining a type and degree of operator assistance to provide as the threat response based on at least the scalar threat assessment metric, and providing an actuator input based on at least the threat response to an actuator that actuates a system of the device to execute the operator assistance.

2. The method of claim 1, wherein determining the type and degree of operator assistance to provide as the threat response comprises choosing an operator assistance from a set of at least two mutually consistent threat responses, each response of the set being based on the same threat assessment metric.

3. The method of claim 1, the threat response being at least one selected from the group of mutually consistent responses consisting of:

triggering a warning for a human device operator;

triggering a collision prevention mechanism;

triggering a collision mitigation mechanism;

regulating an input of a human device operator;

modifying an input of a human device operator;

regulating an input of the optimal controller;

modifying an input of the optimal controller; and

triggering autonomous control of the device by the optimal controller.

4. The method of claim 3, the step of wherein the combining at least one of the predicted states and the predicted inputs into a scalar threat assessment metric comprises:

determining a threat cost at each prediction step in the optimal sequence using a weighting function of the same form as the objective function that was used to generate a sequence of at least one optimal device state and optimal device inputs and combining the sequence of determined threat costs to generate an aggregate threat assessment metric.

5. The method of claim 4, wherein generating the aggregate threat assessment metric comprises basing the generation on at least one norm, selected from the group consisting of:

a maximum predicted value of at least one threat cost over a prediction horizon;

an average predicted value of at least one threat cost over a prediction horizon; and

a Root Mean Square (RMS) of predicted value of at least one threat cost over a prediction horizon.

6. The method of claim 4, wherein generating the aggregate threat assessment metric comprises basing the generation on at least one norm, selected for the group consisting of:

a. a maximum predicted value of at least one state over a prediction horizon;

b. an average predicted value of at least one state over a prediction horizon;

c. a Root Mean Square (RMS) of predicted value of at least one state over a prediction horizon;

d. a maximum predicted value of at least one input over a prediction horizon;

e. an average predicted value of at least one input over a prediction horizon;

f. a Root Mean Square (RMS) of predicted value of at least one input over a prediction horizon; and

g. a maximum predicted value of at least one input and at least one state over a prediction horizon;

h. an average predicted value of at least one input and at least one state over a prediction horizon;

i. a Root Mean Square (RMS) of predicted value of at least one input and at least one state input over a prediction horizon; and

j. a weighted prediction of any of the factors of states and inputs listed in sub-paragraphs a-i of this claim, formed by weighting predictions that are at a chosen time more heavily in the aggregate threat metric than a predicted factor at a different times.

7. The method of claim 3, the warning for a human device operator comprising a signal selected from the group consisting of:

an audible signal, a visual signal, a haptic signal, and an olfactory signal.

8. The method of claim 3, the humanly perceptible signal varying in at least one of frequency and amplitude based on the threat assessment metric.

9. The method of claim 3, the control assistance comprising a passive device control assistance.

10. The method of claim 3, the control assistance comprising a reactive device control assistance.

11. The method of claim 3, the control assistance comprising an active device control assistance.

12. The method of claim 10, the device comprising a ground vehicle, the active device control assistance comprising at least one selected from the group consisting of:

traction control, adaptive cruise control yaw stability control, roll stability control, electronic stability control;

lane keeping and obstacle avoidance.

13. The method of claim 1, the device comprising a ground vehicle, the operator assistance comprising at least one selected from the group consisting of:

resistance torques on a steering input member;

resistance torques on a braking input member;

resistance torques on an acceleration input member;

acceleration torques;

braking torques;

priming wheel brakes, preparing safety devices and modifying device properties.

14. The method of claim 1, the operator assistance comprising an active device modulation.

15. The method of claim 14, the device comprising a ground vehicle, the device modulation signal comprising suspension modification.

16. The method of claim 1, the device comprising a ground vehicle.

17. The method of claim 1, the device comprising an aerial vehicle.

18. The method of claim 1, the device comprising a naval vessel.

19. The method of claim 1, the device comprising heavy construction equipment.

20. The method of claim 1, the device comprising a fork lift truck.

21. The method of claim 1, the device comprising a surgical device.

22. The method of claim 1, the device comprising a teleoperated device residing in the environment, which device environment is distant from an environment in which a human device operator resides.

23. A method according to claim 1, wherein the predicted stability of the device is determined based on at least (i) spatial constraints on the plurality of predicted device states, and (ii) predicted non-spatial states of the device given the spatial constraints.

24. A method according to claim 23, wherein the device comprises a vehicle, and wherein the predicted non-spatial states of the device comprise at least one of predicted vehicle states of:

longitudinal acceleration, lateral acceleration, vehicle sideslip angle, tire sideslip angle, road friction utilization, vehicle yaw rate, vehicle roll angle, vehicle roll rate, and vehicle pitch angle.

25. A method according to claim 1, wherein the device comprises a vehicle, and wherein a steering angle of the vehicle diverges from a steering angle of a human device operator based on at least the predicted stability of the device.