Multi-sensor probabilistic object detection and automated braking

• US 10,315,649 B2
• Filed: 11/29/2016
• Issued: 06/11/2019
• Est. Priority Date: 11/29/2016
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Kadetotad, Sneha, Jain, Jinesh J., Murali, Vidya Nariyambut, Liu, Dongran, Castro, Marcos Paul Gerardo, Siddiqui, Adil Nizam
• Primary Examiner(s): Holloway, Jason
• Application Number:
  US15/363,763
• Publication Number:
  US 20180148047A1
• Time in Patent Office: 924 Days
• CPC Class Codes: B60W 10/04 including control of propulsion units
• CPC Class Codes: B60W 10/18 including control of braking systems
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2420/54 Audio sensitive means, e.g. ultrasound
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 30/08 Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: B60W 40/04 Traffic conditions

Abstract

A controller receives outputs from a plurality of sensors such as a camera, LIDAR sensor, RADAR sensor, and ultrasound sensor, which may be rearward facing. A probability is updated each time a feature in a sensor output indicates presence of an object. The probability may be updated as a function of a variance of the sensor providing the output and a distance to the feature. Where the variance of a sensor is directional, directional probabilities may be updated according to these variances and the distance to the feature. If the probability meets a threshold condition, actions may be taken such as a perceptible alert or automatic braking. The probability may be decayed in the absence of detection of objects. Increasing or decreasing trends in the probability may be amplified by further increasing or decreasing the probability.

Claims

1. A method comprising performing, by a vehicle controller:

performing a plurality of iterations by;

receiving a plurality of sensor outputs each including an obstruction-indicating feature from a plurality of sensors having a plurality of different sensing modalities;

for a first iteration of the plurality of iterations, calculating a probability according to each sensor output of the plurality of sensor outputs and an estimated distance to the obstruction-indicating feature in the each sensor output, the probability indicating a probability of obstruction;

for subsequent iterations of the plurality of iterations, updating the probability according to each sensor output of the plurality of sensor outputs and the estimated distance to the obstruction-indicating feature in the each sensor output; and

when the probability is sufficiently high to perform collision avoidance, performing collision avoidance;

wherein at least one sensor of the plurality of sensors has multiple directional variances associated therewith;

wherein updating the probability comprises initiating for the first iteration of the plurality of iterations and updating for the subsequent iterations of the plurality of iterations each probability of a plurality of directional probabilities according to each sensor output of the at least one sensor according to both of the estimated distance and a directional variance of the directional variances corresponding to the each probability.

2. The method of claim 1, wherein the plurality of sensors include at least two of a two-dimensional camera image, a radio distancing and ranging (RADAR) sensor, a light distancing and ranging (LIDAR) sensor, and an ultrasonic sensor.

3. The method of claim 1, wherein each sensor of the plurality of the sensors has a variance associated therewith;

and

wherein updating the probability comprises updating the probability according to each sensor output of the plurality of sensor outputs according to both of the estimated distance and the variance of a sensor of the plurality of sensors that generated the each sensor output.

4. The method of claim 1, wherein updating the probability comprises performing a Bayesian probability update.

5. The method of claim 1, wherein the plurality of sensors are rearward facing on a vehicle including the vehicle controller.

6. The method of claim 1, further comprising:

reducing the probability in response to detecting a period of time in which no obstruction-indicating features are detected in outputs of the plurality of sensors.

7. The method of claim 6, further comprising:

detecting a decreasing trend in the probability; and

additionally reducing the probability in response to the decreasing trend.

8. The method of claim 7, further comprising:

detecting an increasing trend in the probability; and

additionally increasing the probability in response to the increasing trend.

9. The method of claim 1, further comprising, in response to determining that the probability is sufficiently high to perform collision avoidance, actuating at least one of a steering actuator, an accelerator actuator, and a brake actuator.

10. A system comprising:

a plurality of sensors having a plurality of sensing modalities;

a vehicle controller operably coupled to the plurality of sensors, the vehicle controller programmed to—

perform a plurality of iterations each including;

receiving a plurality of sensor outputs each including an obstruction-indicating feature indicating obstruction, the plurality of sensors having a plurality of different sensing modalities;

for a first iteration of the plurality of iterations, calculating a probability according to each sensor output of the plurality of sensor outputs and an estimated distance to the obstruction-indicating feature in the each sensor output, the probability indicating a probability of obstruction;

for iterations subsequent to the first iteration updating the probability according to each sensor output of the plurality of sensor outputs and the estimated distance to the obstruction-indicating feature in the each sensor output;

reducing the probability if a period of time occurs in which no obstruction-indicating features are detected in outputs of the plurality of sensors;

if a decreasing trend in the probability is detected, further reduce the probability in response to the decreasing trend;

if an increasing trend in the probability is detected, additionally increase the probability in response to the increasing trend; and

if the probability meets a threshold condition, perform collision avoidance.

11. The system of claim 10, wherein the plurality of sensors include at least two of a two-dimensional camera image, a radio distancing and ranging (RADAR) sensor, a light distancing and ranging (LIDAR) sensor, and an ultrasonic sensor.

12. The system of claim 10, wherein each sensor of the plurality of the sensors has a variance associated therewith;

wherein the vehicle controller is further programmed to update the probability according to each sensor output of the plurality of sensor outputs according to both of the estimated distance and the variance of a sensor of the plurality of sensors that generated the each sensor output.

14. The system of claim 10, wherein the vehicle controller is further programmed to update the probability by performing a Bayesian probability update.

15. The system of claim 10, wherein the plurality of sensors are rearward facing on a vehicle including the vehicle controller.

16. The system of claim 10, further comprising at least one of a steering actuator, an accelerator actuator, and a brake actuator;

wherein the vehicle controller is further programmed to, if the probability meets the threshold condition, activate at least one of the steering actuator, the accelerator actuator, and the brake actuator.

13. A system comprising:

a plurality of sensors having a plurality of sensing modalities;

a vehicle controller operably coupled to the plurality of sensors, the vehicle controller programmed to—

perform a plurality of iterations each including;

receiving a plurality of sensor outputs each including an obstruction-indicating feature indicating obstruction, the plurality of sensors having a plurality of different sensing modalities;

for a first iteration of the plurality of iterations, calculating a probability according to each sensor output of the plurality of sensor outputs and an estimated distance to the obstruction-indicating feature in the each sensor output, the probability indicating a probability of obstruction;

for iterations subsequent to the first iteration updating the probability according to each sensor output of the plurality of sensor outputs and the estimated distance to the obstruction-indicating feature in the each sensor output and

if the probability meets a threshold condition, perform collision avoidance

wherein at least one sensor of the plurality of sensors has multiple directional variances associated therewith; and

wherein the vehicle controller is further programmed to initiate for a first iteration of the plurality of iterations and update for subsequent iterations of the plurality of iterations each probability of a plurality of directional probabilities according to each sensor output of the at least one sensor according to both of the estimated distance and a directional variance of the directional variances corresponding to the each probability.