Extra-vehicular threat predictor

• US 7,579,942 B2
• Filed: 10/09/2006
• Issued: 08/25/2009
• Est. Priority Date: 10/09/2006
• Status: Active Grant

• Current Assignee: Toyota Motor Corporation
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Kalik, Steven F.
• Primary Examiner(s): Nguyen; Hung T.
• Application Number:
  US11/539,686
• Publication Number:
  US 20080084283A1
• Time in Patent Office: 1,051 Days
• Field of Search: 340/435, 340/436, 340/438, 340/439, 340/425.5, 340/903, 340/988, 340/995.17, 340/933, 180/166, 180/167, 701/300, 701/301
• US Class Current: 340/435
• CPC Class Codes: B60Q 9/00 Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00 , e.g. haptic signalling
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

An apparatus for facilitating the safe operation of a vehicle comprises a sensor system, a sensor signal to object identifier, and a world model implemented using, for example, a multiple-input multiple-output (MIMO) algorithm such as a spiking neuron algorithm. Alerts may be provided to the vehicle operator under several conditions, such as the identification of a collision risk between the vehicle and an object, identification of a vehicle risk due to excessive uncertainty in the predicted vehicle path, or identification of an object risk due to excessive uncertainty in a predicted object path.

Claims

1. A method of facilitating safe operation of a vehicle, the vehicle having a vehicle operator and having a vehicle environment including an object, the method comprising:

obtaining sensor data relating to the vehicle environment;

predicting a vehicle path within the vehicle environment, the vehicle path having a vehicle path uncertainty;

predicting an object path for the object within the vehicle environment;

identifying a collision risk if the vehicle path and the object path intersect;

identifying a vehicle risk if the vehicle path uncertainty is greater than a threshold uncertainty;

providing an alert on identifying the collision risk or the vehicle risk, the alert facilitating tile safe operation of the vehicle;

determining an object oath uncertainty for the object path;

identifying an object risk if the object oath uncertainty is greater than an object threshold uncertainty;

providing the alert on identifying the object risk; and

wherein predicting the vehicle path includes;

creating a world model of the vehicle environment using the sensor data;

detecting changes in the world model at intervals, the vehicle path and the object oath being predicted from the changes in the world model.

2. The method of claim 1, wherein the world model is created using a multiple-input multiple-output (MIMO) algorithm.

3. The method of claim 2, wherein the multiple-input multiple-output (MIMO) algorithm is a spiking neuron algorithm.

4. The method of claim 1, further comprising:

identifying the object within the vehicle environment using the sensor data; and

selecting an object model for the object using identification of the object,

the object path being predicted using the object model.

5. The method of claim 4, further comprising:

identifying of a plurality of objects within the vehicle environment using the sensor data;

selecting an object model for each object using identification of that object;

predicting the object path for each object; and

providing the alert if any object path intersects with the vehicle path.

6. The method of claim 5, further comprising:

determining an object path uncertainty for each object path;

identifying an object risk if the object path uncertainty of any object is greater than a threshold value for that object; and

providing the alert on identifying the object risk.

7. The method of claim 1, wherein the sensor data is provided by a sensor system,

the sensor system including at least one sensor chosen from the group consisting of an image sensor, IR sensor, microwave sensor, millimeter wave sensor, ultrasound sensor, and laser sensor.

8. The method of claim 1, wherein obtaining sensor data at intervals includes receiving sensor data from one or more image sensors.

9. The method of claim 8, further comprising identifying the object within the vehicle environment using an image processor.

10. The method of claim 1, wherein the threshold uncertainty for the vehicle path uncertainty is determined from previously calculated vehicle path uncertainties.

11. The method of claim 1. further comprising determination of an object path uncertainty, and providing the alert if the object path uncertainty is greater than an object threshold uncertainty.

12. The method of claim 1, further comprising:

creating a statistical object model for the object in the vehicle environment using the sensor data, the object path being predicted using the statistical object model.

13. The method of claim 1, wherein said sensor data is obtained via a communication link with an outside source.

14. A method of facilitating safe operation of a vehicle, the vehicle having a vehicle operator and a vehicle environment including an object, the method comprising:

collecting sensor data representative of the vehicle environment;

identifying the object within the vehicle environment using the sensor data;

creating a world model, the world model including an object model for the object and a vehicle model for the vehicle;

detecting changes in the world model at intervals;

predicting a vehicle path for the vehicle from changes in the world model, the vehicle path having a vehicle path uncertainty;

predicting an object path from changes in the world model, the object path having an object path uncertainty; and

providing a collision risk alert if a collision between the vehicle and the object is predicted from the vehicle path and the object path, providing a vehicle risk alert if the vehicle path uncertainty exceeds a vehicle uncertainty threshold, and providing an object risk alert if the object path uncertainty exceeds an object uncertainty threshold, so as to facilitate the safe operation of the vehicle; and

wherein providing the vehicle risk includes;

determining vehicle parameters from the changes in the world model;

determining long-term statistics for the vehicle parameters; and

identifying the vehicle risk using the long-term statistics of the vehicle parameters.

15. An apparatus for reducing an accident risk for a vehicle, the vehicle having a vehicle operator and a vehicle environment including an object, the apparatus comprising:

a world modeler, the world modeler receiving object data correlated with objects in the vehicle environment and creating a world model including object parameters and vehicle parameters;

a world model change detector, detecting changes in the world model at intervals, the world model change detector identifying vehicle parameter changes and object parameter changes;

a vehicle path predictor, using the vehicle parameters to predict a vehicle path;

an object path predictor, using the object parameters to predict an object path;

a behavior evaluator, providing a collision alert on establishing a collision risk from the vehicle path and the object path, providing a vehicle risk alert if the vehicle path has an uncertainty greater than a first threshold value, and providing an object risk alert if the object path has an uncertainty greater than a second threshold value; and

wherein providing the vehicle risk includes;

determining vehicle parameters from the changes in the world model;

determining long-term statistics for the vehicle parameters; and

identifying the vehicle risk using the long-term statistics of the vehicle parameters.

16. The apparatus of claim 15, further comprising:

a sensor system, receiving sensor data correlated with the vehicle environment; and

an object identifier, operative to identify objects within the vehicle environment, and to determine object data associated with the objects.

17. The apparatus of claim 16, wherein the sensor system includes at least one imaging sensor.

18. The apparatus of claim 16, wherein the object identifier comprises an image processor receiving sensor signals from the imaging sensor.

19. The apparatus of claim 16, wherein the world modeler comprises a multiple-input multiple-output (MIMO) algorithm.

20. The apparatus of claim 15, wherein the multiple-input multiple-output (MTMO) algorithm is a spiking neuron algorithm.

21. The apparatus of claim 15, further comprising a communication link with an outside source, the communication link providing sensor data from the outside source with respect to the vehicle environment.