MULTIOBJECT FUSION MODULE FOR COLLISION PREPARATION SYSTEM

• US 20100191391A1
• Filed: 01/19/2010
• Published: 07/29/2010
• Est. Priority Date: 01/26/2009
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations Incorporated (Motors Liquidation Co. GUC Trust)
• Inventors: Zeng, Shuqing
• Granted Patent: US 8,812,226 B2
• Time in Patent Office: Days
• US Class Current: 701/1
• CPC Class Codes: G01S 13/723 by using numerical data
• CPC Class Codes: G01S 13/86 Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/87 Combinations of radar systems, e.g. primary radar and secondary radar
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 2013/9316 combined with communication equipment with other vehicles or with base stations
• CPC Class Codes: G01S 2013/932 using own vehicle data, e.g. ground speed, steering wheel direction
• CPC Class Codes: G01S 2013/93271 in the front of the vehicles
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

A method for controlling a vehicle operating during a dynamic vehicle event includes monitoring a first input image, monitoring a first tracked object within the first input image in a first tracking cycle, monitoring a second input image, monitoring a second tracked object within the second input image in a second tracking cycle, and determining a dissimilarity measure comparing the first tracked object to the second tracked object. The dissimilarity measure estimates whether the first tracked object and the second tracked object represent a single tracked object proximate to the vehicle. The method further includes associating the first tracked object and the second tracked object based upon the dissimilarity measure, and utilizing the associated objects in a collision preparation system to control operation of the vehicle.

Claims

1. Method for controlling a vehicle operating during a dynamic vehicle event, the method comprising:

monitoring a first input image;

monitoring a first tracked object within the first input image in a first tracking cycle;

monitoring a second input image;

monitoring a second tracked object within the second input image in a second tracking cycle;

determining a dissimilarity measure comparing the first tracked object to the second tracked object, the dissimilarity measure estimating whether the first tracked object and the second tracked object represent a single tracked object proximate to the vehicle;

associating the first tracked object and the second tracked object based upon the dissimilarity measure; and

utilizing the associated objects in a collision preparation system to control operation of the vehicle.

2. The method of claim 1, further comprising:

determining within the first input image the first tracked object;

determining a position within the first input image of the first tracked object;

defining features within the first input image proximate to the first tracked object;

determining a first label describing a likely classification of the first tracked object;

determining within the second input image the second tracked object;

determining a position within the second input image of the second tracked object;

defining features within the second input image proximate to the second tracked object;

determining a second label describing a likely classification of the second tracked object; and

wherein determining the dissimilarity measure comparing the first tracked object to the second tracked object comprises comparing the positions of each of the tracked objects, the features proximate to each of the tracked objects, and the labels.

3. The method of claim 2, wherein the likely classifications include a quickly moving object, a slowly moving object, and a stationary object.

4. The method of claim 2, further comprising:

monitoring range sensor data corresponding to each of the input images; and

wherein determining within the input images the tracked objects is based upon the range sensor data corresponding to each of the input images.

5. The method of claim 2, wherein the second label is based upon the first label.

6. The method of claim 2, wherein determining within the input images the tracked objects comprises indentifying salient image areas and areas associated with motion in each of the images.

7. The method of claim 6, wherein identifying salient image areas and areas associated with motion in each of the images comprises:

determining an edge map;

determining a corner map;

determining a Lucan-Kanade optical flow map;

determining an area of the input image indicating ground; and

identifying the salient image areas and the areas associated with motion based upon the edge map, the corner map, the Lucan-Kanade optical flow map, and the area of the input image indicating ground.

8. The method of claim 7, wherein indentifying salient image areas and areas associated with motion further comprises projecting object information from a previous input image upon the input image;

and

wherein identifying the salient image areas and the areas associated with motion is further based upon the projecting object information from the previous input image.

9. The method of claim 1, wherein the first object is selected a priori to compare to the second object.

10. The method of claim 1, further comprising:

analyzing the associated tracked objects with a bank of Kalman filters; and

wherein utilizing the associated tracked objects in the collision preparation system comprises utilizing the analyzed associated tracked objects.

11. The method of claim 10, wherein analyzing the associated objects with a bank of Kalman filters comprises:

monitoring a speed and a yaw rate of the vehicle;

applying the bank of Kalman filters to the associated objects and the speed and yaw rate of the vehicle; and

utilizing a Markov decision process upon the output from the bank of Kalman filters to generate the analyzed associated objects.

12. Method for controlling a vehicle operating during a dynamic vehicle event, the method comprising:

monitoring range sensor information and vision system information;

superimposing the range sensor information upon the vision system information to generate a modified image;

identifying salient image areas within the modified image using edge detection, corner detection, and optical flow detection;

excluding the salient image areas corresponding to ground;

iteratively identifying objects in a vehicle field-of-view using the salient image areas including identifying a first object in the vehicle field-of-view using the salient image areas in a first time step and identifying a second object in the vehicle field-of-view using the salient image areas in a subsequent time step;

predicting a position of the first object in the subsequent time step;

matching the first object and the second object based upon comparing the predicted position of the first object and the identified second object;

determining object tracks of the matched objects;

determining a likelihood for collision between each of the matched objects and the vehicle based upon the object tracks; and

controlling the vehicle based upon the likelihood for collision.

13. System for operating a collision preparation system in a vehicle, the system comprising:

a vision system;

a range sensor system; and

a control module

monitoring data from the range sensor and images from the vision system,

iteratively identifying objects in a vehicle field-of-view based upon the range sensor data and the images including identifying a first object in the vehicle field-of-view using the range sensor data and the images in a first time step and identifying a second object in the vehicle field-of-view using the range sensor data and the images in a subsequent time step,

predicting a position of the first object in the subsequent time step,

associating the first object and the second object based upon the predicted position of the first object,

determining an object track of the associated objects,

determining a likelihood for collision between the associated objects and the vehicle based upon the object track, and

controlling the vehicle based upon the likelihood for collision.

14. The system of claim 13, wherein determining the object track of the associated objects comprises:

processing the associated objects with a plurality of Kalman filters; and

determining the object track based upon selection of an output of the plurality of Kalman filters using a Markov decision process.

15. The system of claim 13, wherein the identifying objects in the vehicle field-of-view comprises:

superimposing the range sensor data onto one of the images to generate a modified image;

extracting image areas on the modified image associated with the range sensor information;

determining a feature descriptor of the image areas; and

identifying objects in the image areas based upon the feature descriptor.

16. The system of claim 15, wherein the feature descriptor is determined using at least one of a histogram of predetermined features, a histogram of gradient orientation, and principle component analysis.

17. The system of claim 16, wherein the predetermined features are determined using at least one of a Haar wavelet, a Gabor wavelet, and a Leung-and-Malik filter bank.

18. The system of claim 13, wherein the associating the first object and the second object based upon the predicted position of the first object comprises:

determining a dissimilarity measure between the first and second objects; and

matching the first and second objects when the dissimilarity is less than a predetermined threshold.

19. The system of claim 13, wherein determining the likelihood for collision between the associated objects and the vehicle based upon the object track comprises:

determining a Gaussian distribution for the object track;

randomly sampling the Gaussian distribution to obtain a set of particles;

determining a longitudinal and lateral displacement of each particle;

determining whether each particle is on a collision path with the vehicle;

determining a time-to-collision for each particle on a collision path;

designating each particle on a collision path to be a threat if the time-to-collision for the respective particle is less than a threshold time; and

determining a collision to be imminent when a predetermined percentage of the set of particles designated to be a threat is greater than a threshold percentage.