System and method of target tracking using sensor fusion

• US 20070073473A1
• Filed: 09/26/2005
• Published: 03/29/2007
• Est. Priority Date: 09/26/2005
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations Incorporated (Motors Liquidation Co. GUC Trust)
• Inventors: Zeng, Shuqing, Altan, Osman
• Granted Patent: US 7,460,951 B2
• Time in Patent Office: Days
• US Class Current: 701/207
• CPC Class Codes: G01S 13/726 Multiple target tracking
• CPC Class Codes: G01S 13/862 Combination of radar systems with sonar systems
• CPC Class Codes: G01S 13/865 Combination of radar systems with lidar systems
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 2013/93185 Controlling the brakes
• CPC Class Codes: G01S 2013/9321 Velocity regulation, e.g. cruise control
• CPC Class Codes: G01S 2013/9322 using additional data, e.g. driver condition, road state or weather data
• CPC Class Codes: G01S 2013/93271 in the front of the vehicles
• CPC Class Codes: G06F 18/253 of extracted features
• CPC Class Codes: G06V 10/806 of extracted features
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle

Abstract

A target tracking and sensory fusion system is adapted for use with a vehicle, and configured to observe a condition of at least one object during a cycle. The system includes a plurality of sensors, and a novel controller communicatively coupled to the sensors and configured to more accurately estimate the condition based on sensory fusion. In a preferred embodiment, Kalman filtering is utilized to produce a fused estimate of the object location. The preferred controller is further configured to match each new sensory observation with a track in a track list, and remove the track from the track list, when a matching observation is not determined, during a subsequent cycle.

Claims

1. A target tracking and sensor fusion system for estimating a condition of at least one object, said system comprising:

a first sensor configured to determine a first estimate of a condition of the object;

a second sensor configured to determine a second estimate of the condition, and

a controller communicatively coupled to the sensors, and configured to determine a third estimate of the condition, wherein said third estimate is based in part on the first and second estimates,

each of said first and second estimates including a measured value and a standard of deviation,

said third estimate presenting a calculated value and a standard of deviation less than each of the first and second standards of deviation.

2. The system as claimed in claim 1,

said controller being configured to apply Kalman filtering (KF) to determine the third estimate.

3. The system as claimed in claim 1,

said first or second sensor utilizing a mode of detection selected from the group consisting essentially of radar, Sonar, Lidar, video imaging, and ultrasonic sensors.

4. The system as claimed in claim 1,

said first or second sensor including a data processor configured to determine a first estimate of a second condition of the object, wherein said first condition is the object range relative to the measuring sensor, and said second condition is the range rate of the object, based in part on the object range.

5. The system as claimed in claim 1,

said first condition being the azimuth angle of the object location relative to the measuring sensor,

said controller being further configured to estimate the azimuth angle rate based in part on the azimuth angle.

6. The system as claimed in claim 1,

said first sensor being configured to determine first estimates of a plurality of object conditions,

said second sensor being configured to determine second estimates of the plurality of conditions,

said controller being further configured to determine third estimates of the plurality of conditions, wherein said third estimates are based in part on the corresponding first and second estimates,

said controller being further configured to generate a fused track based on the third estimates.

7. A target tracking and sensor fusion system for more accurately estimating the location of an object over a plurality of cycles, said system comprising:

first and second sensors configured to determine separate first and second sensor estimates of the object location during each cycle, wherein each of said first and second estimates includes a measured value and a standard of deviation; and

a controller communicatively coupled to the sensors, and configured to determine during a cycle a third estimate of the object location, wherein said third estimate is based in part on the first and second sensor estimates, and presents a calculated value and a standard of deviation less than each of the first and second standards of deviation,

said controller being further configured to match the third estimate to a track in a track list, project a future location of the object based on the third estimate, and remove the track from the track list, when no sensor estimates match the projected future location, during a subsequent cycle.

8. A computer program for execution by at least one electronic device associated with a plurality of sensors, wherein each of said sensors are configured to estimate at least one condition of at least one object, and said program is configured to receive initial estimate data of said at least one condition from the sensors, and apply a sensory fusion algorithm to the initial estimate data, so as to determine a state estimate for said at least one condition, wherein said state estimate presents a higher probability and smaller standard of deviation than the initial estimate data.

9. The program as claimed in claim 8, wherein initial and state estimates are determined for a plurality of conditions, and said state estimates are stored in a track (y_k(t)).

10. The program as claimed in claim 9, wherein initial and state estimates for a plurality of conditions are determined for a plurality of objects, so as to determine a plurality of tracks, and a state vector (Y(t)) comprising the tracks at a discrete time (t).

11. The program as claimed in claim 9, wherein the plurality of conditions include at least one rate condition, and each of said tracks is dynamically modeled at a time increment (t+1) by applying to y_k(t) a vector multiplier (F) that assumes a constant rate condition, and adding a white Guassian noise vector (v_k).

12. The program as claimed in claim 11, wherein initial and state estimates are determined for a plurality of conditions including object range (r), range rate ({dot over (r)}), azimuth angle (θ

) and azimuth angle rate ({dot over (θ

)}), and the modeled track (y_k(t+1)) is determined according to the formula;

y_k(t+¹)=Fy_k(t)+v_k, where

$F=\left[\begin{array}{cccc}1& \mathrm{\Delta <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>t& \text{\hspace{1em}<br><br>}& \text{\hspace{1em}<br><br>}\\ \text{\hspace{1em}<br><br>}& 1& \text{\hspace{1em}<br><br>}& \text{\hspace{1em}<br><br>}\\ \text{\hspace{1em}<br><br>}& \text{\hspace{1em}<br><br>}& 1& \mathrm{\Delta <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>t\\ \text{\hspace{1em}<br><br>}& \text{\hspace{1em}<br><br>}& \text{\hspace{1em}<br><br>}& 1\end{array}\right].$

13. The program as claimed in claim 9, wherein state estimates are determined for at least one new object, and compared to y_k(t) to determine a difference parameter for each of said conditions.

14. The program as claimed in claim 13,

each of said difference parameters being passed through a function based on the characteristics of the sensor, further multiplied by a constant coefficient based on the robustness of individual sensor measurements, and then combined to determine a merit value (L_k,i), wherein said state estimates of said at least one new object is assigned to y_k(t) where L_k,i is not less than a pre-determined threshold.

15. The program as claimed in claim 14,

said function being a fuzzy logic membership function.

16. The program as claimed in claim 14,

said function being a symmetric kernel function.

17. The program as claimed in claim 16,

said function being

$f_R$

(Δ

⁢

 

⁢

R)=K⁡

(Δ

⁢

 

⁢

RhR),

where h_R denotes a scaling factor based on the characteristics of the sensor.

18. The program as claimed in claim 17,

said function being a Gaussian Kernel function.

19. The program as claimed in claim 18,

said function being:

K(u)=exp(−

u²).

20. The program as claimed in claim 17,

said function being a quadratic Kernel function.

21. The program as claimed in claim 20,

said function being

$K$

(u)={(1-d2)if⁢

 

⁢



d

<

1,0otherwise.

22. The program as claimed in claim 14, wherein a plurality of observations (O_i(t)) are assigned to y_k(t), and each of said observations are defined by the formula:

O_i(t)=Gy(t)+w_i,

where

$G=\left[\begin{array}{cccc}1& \text{\hspace{1em}<br><br>}& \text{\hspace{1em}<br><br>}& \text{\hspace{1em}<br><br>}\\ \text{\hspace{1em}<br><br>}& 1& \text{\hspace{1em}<br><br>}& \text{\hspace{1em}<br><br>}\\ \text{\hspace{1em}<br><br>}& \text{\hspace{1em}<br><br>}& 1& 0\end{array}\right],$

w_i is a 3-by-1 white Gaussian noise vector derived in part from the accuracy of the sensor system, and the observations are combined to determine a true track status.

23. The program as claimed in claim 22, wherein the observations are combined using Kalman filtering to determine the true track status.

24. The program as claimed in claim 23, wherein the true track status is determined in accordance with the formula(s):

25. The program as claimed in claim 24, wherein the true track status is further determined in accordance with the covariance formula(s):

26. The program as claimed in claim 24, wherein the KF gain for the i-th observer (K_i) is determined in accordance with the following formula:

K_i={circumflex over (P)}(t+1|t)G^T(R_i+G{circumflex over (P)}(t+1|t)G^T)<sup>−

1</sup>.

27. The program as claimed in claim 10,

each of said tracks y_k(t) being assigned a track strength (q_k(t)), wherein q_k(t)=1 upon the assignment of an observation to the track, and decays at a rate (λ

) over a subsequent period, if no observations are assigned to the track over the period, and the track y_k(t) is removed from Y(t), if q_k(t) falls below a pre-determined minimum threshold.

28. The program as claimed in claim 27, wherein a first portion of the tracks decays at a different rate than a second portion of the tracks.