Detecting long objects by sensor fusion

• US 10,317,522 B2
• Filed: 03/01/2016
• Issued: 06/11/2019
• Est. Priority Date: 03/01/2016
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Tong, Wei, Zeng, Shuqing, Wang, Jinsong, Zhang, Wende
• Primary Examiner(s): Barker, Matthew M
• Application Number:
  US15/057,437
• Publication Number:
  US 20170254895A1
• Time in Patent Office: 1,197 Days
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/878 Combination of several spaced transmitters or receivers of known location for determining the position of a transponder or a reflector G01S13/874 takes precedence
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 2013/9322 using additional data, e.g. driver condition, road state or weather data
• CPC Class Codes: G01S 2013/93271 in the front of the vehicles
• CPC Class Codes: G01S 2013/93272 in the back of the vehicles
• CPC Class Codes: G01S 2013/93274 on the side of the vehicles
• CPC Class Codes: G01S 7/41 using analysis of echo signal for target characterisation; Target signature; Target cross-section

Abstract

A system and method are provided for detecting and identifying elongated objects relative to a host vehicle. The method includes detecting objects relative to the host vehicle using a plurality of object detection devices, identifying patterns in detection data that correspond to an elongated object, wherein the detection data includes data fused from at least two of the plurality of object detection devices, determining initial object parameter estimates for the elongated object using each of the plurality of object detection devices, calculating object parameter estimates for the elongated object by fusing the initial object parameter estimates from each of the plurality of object detection devices, and determining an object type classification for the elongated object by fusing the initial object parameter estimates from each of the plurality of object detection devices.

Claims

1. A method for detecting and identifying elongated objects relative to a host vehicle, the method comprising the steps of:

detecting objects relative to the host vehicle using a plurality of object detection devices, wherein the plurality of object detection devices include at least one of the following;

a radar detection device, a lidar detection device, or an imaging system;

identifying patterns in detection data that correspond to an elongated object, wherein the detection data includes data fused from at least two of the plurality of object detection devices;

determining initial object parameter estimates for the elongated object using each of the plurality of object detection devices;

calculating object parameter estimates for the elongated object by fusing the initial object parameter estimates from each of the plurality of object detection devices; and

determining an object type classification for the elongated object by fusing the initial object parameter estimates from each of the plurality of object detection devices.

2. The method of claim 1, wherein the plurality of object detection devices includes both the radar detection device and the imaging system.

3. The method of claim 2, wherein the step of identifying patterns includes identifying radar signal patterns in radar detection data generated by radar sensors.

4. The method of claim 3, wherein the step of identifying the radar signal patterns includes clustering an initial set of radar measurements into at least one group according to one or more metric(s).

5. The method of claim 4, wherein the one or metrics includes a location and signal strength for each radar measurement in the radar detection data.

6. The method of claim 5, wherein the one or more metric(s) further include a metric derived from captured image data from the imaging system.

7. The method of claim 6, wherein the clustering step further includes mapping radar detection data to the captured image data to determine corresponding image pixel data.

8. The method of claim 4, wherein identifying patterns in the detection data further includes determining, for each of the at least one group, a distribution of the radar measurements in a first dimension, and a concentration of radar measurement locations along a second dimension, which is perpendicular to the first dimension.

9. The method of claim 4, wherein identifying patterns in the detection data further includes applying a linear regression model to the radar measurements in each of the at least one group that satisfy the one or more metric(s).

10. The method of claim 4, wherein identifying patterns in the detection data further includes sorting each radar measurement in each of the at least one group according to an azimuth angle measured relative to the host vehicle.

11. The method of claim 10, further including determining if a range rate associated with each of the sorted radar measurements changes gradually with respect to one another according to a range rate threshold.

12. The method of claim 11, further including estimating a relative speed for a group of radar measurements that have a gradual change in range rate according to the range rate threshold, wherein estimating the relative speed includes applying a linear regression speed model to the group of radar measurements.

13. The method of claim 2, wherein detection data generated by the imaging system includes one or more frame(s) of image data relating to captured images, and wherein the step of identifying patterns includes identifying regions of interest in each frame and tracking points of interest in each of the regions of interest from a previous frame to a current frame.

14. The method of claim 1, wherein calculating the object parameter estimates further includes determining a parameter weight for each of the initial object parameter estimates computed by each of the plurality of object detection devices, and summing weighted object parameters for each of the plurality of object detection devices.

15. The method of claim 1, wherein determining the object type classification for the elongated object includes determining a maximum probability of a particular class of objects as a function of the initial object parameter estimates from each of the plurality of object detection devices.

16. A method for detecting and identifying elongated objects relative to a host vehicle, the method comprising the steps of:

obtaining detection data relating to objects detected using a plurality of object detection devices, wherein the plurality of object detection devices include at least one of the following;

a radar detection device, a lidar detection device, or an imaging system;

fusing the detection data at a first processing level by referencing complementary detection data from each of the plurality of object detection devices;

classifying a detected object as an elongated object at each of the plurality of object detection devices by identifying patterns in the fused detection data that correspond to an elongated object;

calculating initial object parameter estimates for the elongated object using each of the plurality of object detection devices;

fusing the detection data at a second processing level by;

weighting the initial object parameter estimates for each of the plurality of object detection devices and summing the weighted initial object parameter estimates to determine object parameter estimates for the elongated object, and

calculating a maximum probability of a particular class of objects as a function of the initial object parameter estimates from each of the plurality of object detection devices to determine an object type classification for the elongated object.

17. The method of claim 16, wherein the detection data includes radar measurements from the radar detection device and captured image data from the imaging system, and wherein classifying the detected object as an elongated object includes using the radar detection device to apply position and velocity linear regression models to groups of radar measurements clustered according to a location, a signal strength, or both, of the radar measurements and corresponding image pixel data from the captured image data, and using the imaging system to identify regions of interest in each frame of image data and tracking points of interest in each of the regions of interest from a previous frame to a current frame.

18. A system for detecting and identifying elongated objects relative to a host vehicle, the system comprising:

at least two object detection devices selected from the following;

a radar detection device, a lidar detection device, or an imaging system, and wherein the at least two object detection devices are configured to;

obtain detection data relating to objects detected in a field-of-view relating to each of the at least two object detection devices;

fuse the detection data by referencing complementary detection data from each of the at least two object detection devices;

classify a detected object as an elongated object by identifying patterns in the fused detection data that correspond to an elongated object; and

calculate initial object parameter estimates for the elongated object using each of the at least two object detection devices;

at least one vehicle control module configured to;

receive the initial object parameter estimates from each of the at least two object detection devices and fuse the detection data to determine object parameter estimates and to classify the type of elongated object.

19. The system claim 18, wherein the at least two object detection devices include the radar detection device and the imaging system, and wherein the at least one vehicle control module is a fusion module.

20. The system of claim 18, wherein the at least one vehicle control module is configured to fuse the detection data by:

weighting the initial object parameter estimates for each of the plurality of object detection devices and summing the weighted initial object parameter estimates to determine object parameter estimates for the elongated object, and

calculating a maximum probability of a particular class of objects as a function of the initial object parameter estimates from each of the plurality of object detection devices to determine an object type classification for the elongated object.