Enhanced data association of fusion using weighted Bayesian filtering

• US 8,705,797 B2
• Filed: 03/07/2012
• Issued: 04/22/2014
• Est. Priority Date: 03/07/2012
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Zeng, Shuqing, Shi, Lufeng, Gandhi, Daniel, Nickolaou, James N.
• Primary Examiner(s): Tabatabai, Abolfazl
• Application Number:
  US13/413,861
• Publication Number:
  US 20130236047A1
• Time in Patent Office: 776 Days
• Field of Search: 382/103, 382/107, 382/236, 382/287, 348/152, 348/154, 348/155, 348/169, 348/170, 348/171, 348/172, 348/352, 348/94, 348/208.1, 348/208.14, 348/208.16
• US Class Current: 382/103
• CPC Class Codes: G01S 13/726 Multiple target tracking
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 2013/93271 in the front of the vehicles
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A method of associating targets from at least two object detection systems. An initial prior correspondence matrix is generated based on prior target data from a first object detection system and a second object detection system. Targets are identified in a first field-of-view of the first object detection system based on a current time step. Targets are identified in a second field-of-view of the second object detection system based on the current time step. The prior correspondence matrix is adjusted based on respective targets entering and leaving the respective fields-of-view. A posterior correspondence matrix is generated as a function of the adjusted prior correspondence matrix. A correspondence is identified in the posterior correspondence matrix between a respective target of the first object detection system and a respective target of the second object detection system.

Claims

1. A method of associating targets from at least two object detection systems, the method being executed by a processor, comprising the steps of:

generating an initial prior correspondence matrix based on prior target data from a first object detection system and a second object detection system;

identifying targets in a first field-of-view of the first object detection system based on a current time step;

identifying targets in a second field-of-view of the second object detection system based on the current time step;

adjusting the prior correspondence matrix based on respective targets entering and leaving the respective fields-of-view;

generating a posterior correspondence matrix as a function of the adjusted prior correspondence matrix; and

identifying a correspondence in the posterior correspondence matrix between a respective target of the first object detection system and a respective target of the second object detection system.

2. The method of claim 1 further comprising the steps of minimizing offsets in a transformation from a coordinate frame of the first field-of-view to a coordinate frame of the second field-of-view.

3. The method of claim 2 wherein the transformation is performed utilizing a rigid transformation, the rigid transformation being determined by the following formula:

T′

=(1−

η

)T+η

T*

where T′

is an updated transformation, T is a past transformation, T* is a current transformation, and η

is a learning factor 0<

η

<

1.

4. The method of claim 3 wherein the current transformation T* is represented by a coordinate offset (t*_x, t*_y, ε

*) where (t*_x) is an x-coordinate offset, (t*_y) is a y-coordinate offset, and (ε

*) is an angular displacement offset between the coordinate frame of the first field-of-view and the coordinate frame of the second field-of-view.

5. The method of claim 4 wherein x-coordinate offset (t*_x) is represented by the following formula:

6. The method of claim 4 wherein the y-coordinate offset (t*_y) is represented by the following formula:

7. The method of claim 4 wherein the angular displacement offset (ε

*) is determined by the following formula;

8. The method of claim 1 wherein generating the initial prior correspondence matrix includes initializing the prior correspondence matrix as non-informative by setting all correspondence between the targets in the first field-of-view and the targets in the second field-of-view as equal.

9. The method of claim 1 further comprising the steps of:

setting the prior correspondence matrix equal to the posterior correspondence matrix;

determining that a new frame of data is available from the first object detection system and the second object detection system for a next time step;

adjusting the prior correspondence matrix based on targets entering and the leaving the respective field-of-views at the next time step;

generating a next posterior correspondence matrix as a function of the adjusted prior correspondence matrix for the next time step; and

identifying a correspondence within the next posterior correspondence matrix between a respective target of the first object detection system and a respective target of the second object detection system for the next time step.

10. The method of claim 1 wherein the steps of adjusting the prior correspondence matrix based on targets entering and leaving the respective field-of-views further comprises the step of:

determining that a respective target is in the field-of-view of the first object detection system in a previous time step;

determining that the respective target is not in the field-of-view of the first object detection system in the current time step; and

removing a respective row in the prior correspondence matrix associated with the respective target of the first object detection system.

11. The method of claim 1 wherein the step of adjusting the prior correspondence matrix based on targets entering and leaving the respective fields-of-view further comprises the steps of:

determining that a respective target is in the field-of-view of the first object detection system in the current time step;

determining that the respective target is not in the field-of-view of the first object detection system in a previous time step; and

appending a respective row in the prior correspondence matrix associated with the respective target of the first object detection system.

12. The method of claim 11, wherein the respective row is initialized as non-informative in the prior correspondence matrix by setting all correspondence between the first set of vision targets and the second set vision targets in the respective row as equal.

13. The method of claim 1 wherein the step of adjusting the prior correspondence matrix based on targets entering and leaving the respective fields-of-view further comprises the steps of:

determining that a respective target is in the field-of-view of the second object detection system in a previous time step;

determining that the respective target is not in the field-of-view of the second object detection system in the current time step; and

removing a respective column in the prior correspondence matrix associated with the respective target of the second object detection system.

14. The method of claim 1 wherein the step of adjusting the prior correspondence matrix based on targets entering and leaving the respective fields-of-view further comprises the steps of:

determining that a respective target is in the field-of-view of the second object detection system in the current time step;

determining that the respective target is not in the field-of-view of the second object detection system in a previous time step; and

appending a respective row in the prior correspondence matrix that is associated with the respective target of the second object detection system.

15. The method of claim 1 wherein the posterior correspondence matrix is determined by the formula:

16. The method of claim 1 wherein the first object detection system uses radar for detecting targets and the second object detection system uses vision for detecting objects.

17. The method of claim 1 wherein the first object detection system uses radar for detecting targets and the second object detection system uses a predicted target determined from a target motion model, the target motion model determining a predicted position of a target as a function of a last computed posterior correspondence matrix.

18. The method of claim 1 wherein the first object detection system uses vision for detecting targets and the second object detection system uses a predicted target determined from a target motion model, the target motion model determining a predicted position of a target as a function of a last computed posterior correspondence matrix.

19. The method of claim 1 further comprising the steps of:

estimating target positions at a next time step based on a target motion model using corresponding targets identified in the posterior correspondence matrix;

setting the prior correspondence matrix equal to the posterior correspondence matrix;

applying the prior correspondence matrix to a first data association module and a second data association module;

adjusting the prior correspondence matrix applied to the first data association module based on targets leaving and entering the first object detection system at the next time step and based on the predicted targets output from the target motion model;

generating a first posterior correspondence matrix as a function of the adjusted correspondence matrix of the first data association module;

adjusting the prior correspondence matrix applied to the second data association module based on targets leaving and entering the second object detection system at the next time step and based on the predicted targets output from the target motion model;

generating a second posterior correspondence matrix as a function of the adjusted correspondence matrix of the second data association module;

fusing the first posterior correspondence matrix and the second posterior correspondence matrix; and

identifying correspondence within the fused posterior correspondence matrix between respective targets identified from the first data association module and the second data association module.

20. The method of claim 19 further comprising the steps of:

generating a time delay after identifying corresponding targets within the fused correspondence matrix; and

applying target motion analysis to the corresponding targets for estimating a target position for each of the corresponding targets at a next time step.