INTEGRATED FRAMEWORK FOR VEHICLE OPERATOR ASSISTANCE BASED ON A TRAJECTORY PREDICTION AND THREAT ASSESSMENT

• US 20140032017A1
• Filed: 04/09/2013
• Published: 01/30/2014
• Est. Priority Date: 03/05/2009
• Status: Active Grant

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Anderson, Sterling J., Peters, Steven C., Iagnemma, Karl D.
• Granted Patent: US 8,744,648 B2
• Time in Patent Office: Days
• US Class Current: 701/3
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 50/0098 Details of control systems ensuring comfort, safety or stability not otherwise provided for
• CPC Class Codes: B60W 50/10 Interpretation of driver requests or demands
• CPC Class Codes: G05D 1/00 Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots drive control systems specially adapted for autonomous road vehicles B60W60/00
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

Various types and levels of operator assistance are performed within a unified, configurable framework. A model of the device with a model of the environment and the current state of the device and the environment are used to iteratively generate a sequence of optimal device control inputs that, when applied to a model of the device, generate an optimal device trajectory through a constraint-bounded corridor or region within the state space. This optimal trajectory and the sequence of device control inputs that generates it is used to generate a threat assessment metric. An appropriate type and level of operator assistance is generated based on this threat assessment. Operator assistance modes include warnings, decision support, operator feedback, vehicle stability control, and autonomous or semi-autonomous hazard avoidance. The responses generated by each assistance mode are mutually consistent because they are generated using the same optimal trajectory.

Claims

1. A system for providing semi-autonomous control of a mechanical device, the system comprising:

a computer control system, configured to combine at least predicted device controller inputs into a scalar threat assessment metric, the computer control system determining the scalar threat assessment metric based at least on predicted stability of the device through the time horizon, the computer control system being further configured to determine a type and degree of operator assistance to provide as a threat response based on at least the scalar threat assessment metric; and

an actuator configured to actuate a system of the device to execute operator assistance based on an actuator input, the actuator input being based on at least the threat response determined by the computer control system.

2. The system of claim 1, wherein the predicted device controller inputs comprise an optimal sequence of predicted device controller inputs, having been determined based on at least a model of the device and an objective function, the optimal sequence of predicted device controller inputs having being determined so as to optimize predicted performance of the model over a time horizon over which the device model is simulated and satisfy device dynamic constraints, device input constraints, and environmental constraints.

3. The system of claim 1, wherein the computer control system is further configured to determine an optimal sequence of predicted device states resulting from applying the sequence of optimal device inputs to the model of the device;

the computer control system being further configured to combine at least the optimal sequence of predicted device states into the scalar threat assessment metric based at least on the predicted stability of the device through the time horizon as a function of at least one of the predicted device states and the predicted device controller inputs.

4. The system of claim 3, wherein determining the type and degree of operator assistance to provide as the threat response comprises choosing an operator assistance from a set of at least two mutually consistent threat responses, each response of the set being based on the same threat assessment metric.

5. The system of claim 3, the threat response being at least one selected from the group of mutually consistent responses consisting of:

triggering a warning for a human device operator;

triggering a collision prevention mechanism;

triggering a collision mitigation mechanism;

regulating an input of a human device operator;

modifying an input of a human device operator;

regulating an input of an optimal controller;

modifying an input of an optimal controller; and

triggering autonomous control of the device by an optimal controller.

6. The system of claim 5, wherein the computer control system is further configured to combine at least one of the predicted states and the predicted inputs into a scalar threat assessment metric by at least:

determining a threat cost at each prediction step in the optimal sequence using a weighting function of the same form as the objective function that was used to generate a sequence of at least one optimal device state and optimal device inputs and combining the sequence of determined threat costs to generate an aggregate threat assessment metric.

7. The system of claim 6, wherein the computer control system is further configured to generate the aggregate threat assessment metric by at least basing the generation on at least one norm, selected from the group consisting of:

a maximum predicted value of at least one threat cost over a prediction horizon;

an average predicted value of at least one threat cost over a prediction horizon; and

a Root Mean Square (RMS) of predicted value of at least one threat cost over a prediction horizon.

8. The system of claim 6, wherein the computer control system is further configured to generate the aggregate threat assessment metric based on at least one norm, selected for the group consisting of:

a. a maximum predicted value of at least one state over a prediction horizon;

b. an average predicted value of at least one state over a prediction horizon;

c. a Root Mean Square (RMS) of predicted value of at least one state over a prediction horizon;

d. a maximum predicted value of at least one input over a prediction horizon;

e. an average predicted value of at least one input over a prediction horizon;

f. a Root Mean Square (RMS) of predicted value of at least one input over a prediction horizon; and

g. a maximum predicted value of at least one input and at least one state over a prediction horizon;

h. an average predicted value of at least one input and at least one state over a prediction horizon;

i. a Root Mean Square (RMS) of predicted value of at least one input and at least one state input over a prediction horizon; and

j. a weighted prediction of any of the factors of states and inputs listed in sub-paragraphs a-i of this claim, formed by weighting predictions that are at a chosen time more heavily in the aggregate threat metric than a predicted factor at a different times.

9. The system of claim 5, the warning for a human device operator comprising a signal selected from the group consisting of:

an audible signal, a visual signal, a haptic signal, and an olfactory signal.

10. The system of claim 5, the humanly perceptible signal varying in at least one of frequency and amplitude based on the threat assessment metric.

11. The system of claim 5, the control assistance comprising a passive device control assistance.

12. The system of claim 5, the control assistance comprising a reactive device control assistance.

13. The system of claim 5, the control assistance comprising an active device control assistance.

14. The system of claim 12, the device comprising a ground vehicle, the active device control assistance comprising at least one selected from the group consisting of:

traction control, adaptive cruise control yaw stability control, roll stability control, electronic stability control;

lane keeping and obstacle avoidance.

15. The system of claim 1, the device comprising a ground vehicle, the operator assistance comprising at least one selected from the group consisting of:

resistance torques on a steering input member;

resistance torques on a braking input member;

resistance torques on an acceleration input member;

acceleration torques;

braking torques;

priming wheel brakes, preparing safety devices and modifying device properties.

16. The system of claim 1, the operator assistance comprising an active device modulation.

17. The system of claim 16, the device comprising a ground vehicle, the device modulation signal comprising suspension modification.

18. The system of claim 1, the device comprising a ground vehicle.

19. The system of claim 1, the device comprising an aerial vehicle.

20. The system of claim 1, the device comprising a naval vessel.

21. The system of claim 1, the device comprising heavy construction equipment.

22. The system of claim 1, the device comprising a fork lift truck.

23. The system of claim 1, the device comprising a surgical device.

24. The system of claim 1, the device comprising a teleoperated device residing in the environment, which device environment is distant from an environment in which a human device operator resides.

25. A system according to claim 3, wherein the computer control system is further configured to determine the predicted stability of the device based on at least (i) spatial constraints on the plurality of predicted device states, and (ii) predicted non-spatial states of the device given the spatial constraints.

26. A system according to claim 25, wherein the device comprises a vehicle, and wherein the predicted non-spatial states of the device comprise at least one of predicted vehicle states of:

longitudinal acceleration, lateral acceleration, vehicle sideslip angle, tire sideslip angle, road friction utilization, vehicle yaw rate, vehicle roll angle, vehicle roll rate, and vehicle pitch angle.

27. A system according to claim 3, wherein the device comprises a vehicle, and wherein a steering angle of the vehicle diverges from a steering angle of a human device operator based on at least the predicted stability of the device.