System and method for vehicle detection and tracking

• US 8,098,889 B2
• Filed: 01/15/2008
• Issued: 01/17/2012
• Est. Priority Date: 01/18/2007
• Status: Active Grant

• Current Assignee: Continental Automotive GmbH (Continental AG), Siemens Corp. (Siemens AG)
• Original Assignee: Continental Automotive GmbH (Continental AG), Siemens Corp. (Siemens AG)
• Inventors: Zhu, Ying, Baratoff, Gregory, Köhler, Thorsten
• Primary Examiner(s): Fitzpatrick, Atiba O
• Application Number:
  US12/014,397
• Publication Number:
  US 20080273752A1
• Time in Patent Office: 1,463 Days
• Field of Search: 382/100, 382103-107, 340907-911, 340933-938, 600425-429, 378/4, 378/69, 348113-120, 348169-173
• US Class Current: 382/103
• CPC Class Codes: B60W 40/02 related to ambient conditions
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G08G 1/16 Anti-collision systems road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A method for vehicle detection and tracking includes acquiring video data including a plurality of frames, comparing a first frame of the acquired video data against a set of one or more vehicle detectors to form vehicle hypotheses, pruning and verifying the vehicle hypotheses using a set of course-to-fine constraints to detect a vehicle, and tracking the detected vehicle within one or more subsequent frames of the acquired video data by fusing shape template matching with one or more vehicle detectors.

Claims

1. A method for vehicle detection and tracking, comprising:

acquiring video data including a plurality of frames;

comparing a first frame of the acquired video data against a set of one or more vehicle detectors to form vehicle hypotheses;

pruning and verifying the vehicle hypotheses using a set of coarse-to-fine constraints to detect a vehicle; and

tracking the detected vehicle within one or more subsequent frames of the acquired video data by fusing shape template matching with one or more vehicle detectors,

wherein the set of coarse-to-fine constraints includes a first level in which an objective function is replaced by a course approximation and a second level in which mode locations obtained in the first level are refined by optimizing the objective function.

2. A method for vehicle detection and tracking, comprising:

acquiring video data including a plurality of frames;

comparing a first frame of the acquired video data against a set of one or more vehicle detectors to form vehicle hypotheses;

pruning and verifying the vehicle hypotheses using a set of coarse-to-fine constraints to detect a vehicle; and

tracking the detected vehicle within one or more subsequent frames of the acquired video data by fusing shape template matching with one or more vehicle detectors,

wherein the detected vehicle is tracked within subsequent frames by kernel-based probabilistic shape tracker, and

wherein the kernel-based probabilistic shape tracker estimates vehicle motion by matching feature points in an image region with one or more shape templates that have been extracted from a corresponding target of a previous frame.

3. The method of claim 2, wherein the video data is acquired using a camera mounted on a moving host vehicle.

4. The method of claim 2, wherein the set of vehicle detectors includes at least one class-specific detector.

5. The method of claim 4, wherein the set of vehicle detectors includes at least one car specific detector.

6. The method of claim 4, wherein the set of vehicle detectors includes at least one truck specific detector.

7. The method of claim 2, wherein the set of vehicle detectors includes an “

L”

shape corner detector and its mirrored shape corner detector.

8. The method of claim 2, wherein the set of coarse-to-fine constraints includes vehicle part detectors learned from examples of vehicle lower left part and vehicle lower right part.

11. The method of claim 2, wherein the vehicle hypotheses are pruned and verified with a second set of one or more vehicle detectors, different from the set of one or more vehicle detectors used to form the vehicle hypotheses.

12. The method of claim 2, wherein the responses of vehicle detectors are accumulated over time.

13. The method of claim 2, wherein the shape templates are composed of a set of zero-crossing points.

14. The method of claim 13, wherein the zero-crossing points are described by their locations and gradient vectors.

15. The method of claim 2, wherein matching feature points in an image region with a shape template is performed by optimizing a kernel-based nonparametric density estimation on zero-crossing points.

16. The method of claim 15, wherein optimizing a kernel-based nonparametric density estimate on zero-crossing points generates multiple modes of vehicle motion parameters.

17. The method of claim 16, wherein a single best mode of vehicle motion parameters is determined as the mode that has the highest function value of the kernel-based nonparametric density estimate and whose response from the vehicle detectors is above a pre-selected value.

18. The method of claim 2, wherein the shape templates are updated to reflect frame-to-frame variance in target appearance.

19. The method of claim 2, further including assembling results of the verifying step and the tracking step to determine whether tracking is maintained or terminated.

20. The method of claim 2, wherein multiple vehicles are tracked simultaneously.

21. The method of claim 2, wherein the first video frame is represented as an image pyramid of multiple resolutions.

22. The method of claim 21, wherein in the forming of the vehicle detection hypotheses, detection of a vehicle appearing to be relatively small is performed in a relatively high resolution and detection of a vehicle appearing to be relatively large is performed in a relatively low resolution.

23. The method of claim 21, wherein tracking is performed at multiple resolutions.

24. The method of claim 2, wherein in verifying the vehicle detection hypotheses using a set of coarse-to-fine constraints, one or more component detectors are used.

25. The method of claim 2, wherein verified vehicle hypotheses is additionally verified using a whole-appearance vehicle detector.

26. The method of claim 2, wherein template-based tracking is used to track the detected vehicle within the one or more frames.

27. The method of claim 26, wherein template-based tracking uses kernel-based probabilistic shape tracking.

9. A method for vehicle detection and tracking, comprising:

acquiring video data including a plurality of frames;

comparing a first frame of the acquired video data against a set of one or more vehicle detectors to form vehicle hypotheses;

pruning and verifying the vehicle hypotheses using a set of coarse-to-fine constraints to detect a vehicle; and

tracking the detected vehicle within one or more subsequent frames of the acquired video data by fusing shape template matching with one or more vehicle detectors,

wherein the set of coarse-to-fine constraints includes vehicle part detectors learned from examples of vehicle lower left part and vehicle lower right part, and

wherein the vehicle part detectors are followed by an orientation histogram constraint.

10. The method of claim 9, wherein the orientation histogram constraint is followed by a cascade car/truck detector.

28. A system for vehicle detection and tracking, comprising:

a camera mounted on a moving host vehicle for acquiring video data including a plurality of frames; and

a central tracker for comparing a first frame of the acquired video data against a set of one or more vehicle detectors to form a vehicle detection hypotheses, verifying the vehicle detection hypotheses to detect a vehicle, and tracking the detected vehicle within one or more subsequent frames of the acquired video data,

wherein the detected vehicle is tracked within subsequent frames by kernel-based probabilistic shape tracker, and

wherein the kernel-based probabilistic shape tracker estimates vehicle motion by matching feature points in an image region with one or more shape templates that have been extracted from a corresponding target of a previous frame.

29. The system of claim 28, wherein the set of vehicle detectors includes at least one car or truck specific detector.

30. The system of claim 28, wherein the first video frame is represented as an image pyramid of multiple resolutions and tracking is performed at multiple resolutions.

31. A computer system comprising:

a processor; and

a program storage device readable by the computer system, embodying a program of instructions executable by the processor to perform method steps for vehicle detection and tracking, the method comprising;

acquiring video data including a plurality of frames;

comparing a first frame of the acquired video data against a set of one or more vehicle detectors to form a vehicle detection hypotheses; and

tracking the detected vehicle within one or more subsequent frames of the acquired video data,

wherein the detected vehicle is tracked within subsequent frames by kernel-based probabilistic shape tracker, and

wherein the kernel-based probabilistic shape tracker estimates vehicle motion by matching feature points in an image region with one or more shape templates that have been extracted from a corresponding target of a previous frame.

32. The computer system of claim 31, wherein the vehicle detection hypotheses is verified using a set of coarse-to-fine constraints to detect a vehicle.

33. The computer system of claim 31, wherein the set of vehicle detectors includes at least one car or truck specific detector.

34. The computer system of claim 31, wherein the first video frame is represented as an image pyramid of multiple resolutions and tracking is performed at multiple resolutions.