METHOD AND APPARATUS FOR DETECTING AND TRACKING VEHICLES

• US 20120070034A1
• Filed: 03/04/2011
• Published: 03/22/2012
• Est. Priority Date: 03/05/2010
• Status: Active Grant

• Current Assignee: SRI International, Inc.
• Original Assignee: SRI International, Inc.
• Inventors: Sawhney, Harpreet Singh, Cheng, Hui, Xiao, Jiangjian
• Granted Patent: US 8,712,096 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 18/29 Graphical models, e.g. Bayesian networks
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 10/84 using probabilistic graphical models from image or video features, e.g. Markov models or Bayesian networks
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60

Abstract

The present invention relates to a method and apparatus for detecting and tracking vehicles. One embodiment of a system for detecting and tracking an object (e.g., vehicle) in a field of view includes a moving object indication stage for detecting a candidate object in a series of input video frames depicting the field of view and a track association stage that uses a joint probabilistic graph matching framework to associate an existing track with the candidate object.

Claims

1. A system for detecting and tracking an object in a field of view, the system comprising:

a moving object indication stage for detecting a candidate object in a series of input video frames depicting the field of view; and

a track association stage that uses a joint probabilistic graph matching framework to associate an existing track with the candidate object.

2. The system of claim 1, wherein the moving object indication stage comprises:

a short-term change detector for detecting fast-moving objects in the series of input video frames; and

a long-term change detector for detecting slow-moving and stopped objects in the series of input video frames.

3. The system of claim 1, wherein the track association stage comprises:

a repository for storing the existing track; and

a graph matching processor for matching the candidate object to the existing track using at least one vertex matching feature and at least one pairwise spatial matching feature.

4. A method for detecting and tracking an object in a field of view, the method comprising:

detecting a candidate object in a series of input video frames depicting the field of view; and

associating an existing track with the candidate object in accordance with a joint probabilistic graph matching framework.

5. The method of claim 4, wherein the detecting comprises:

identifying one or more fast-moving objects depicted in the series of input video frames;

identifying one or more slow-moving or stopped objects in the series of input video frames; and

identifying the candidate object in accordance with the one or more fast-moving objects and the one or more slow-moving objects.

6. The method of claim 5, wherein the identifying the one or more fast-moving objects comprises:

performing frame subtraction using at least three frames from the series of input video frames, thereby producing a bounding shape around the candidate object.

7. The method of claim 5, wherein the identifying the one or more slow-moving or stopped objects comprises:

performing background modeling and subtraction using a plurality of frames from the series of input video frames, thereby producing a background model.

8. The method of claim 5, wherein the identifying the candidate object comprises:

refining a predicted location of the candidate object as estimated in accordance with the identifying one or more fast-moving objects and the identifying the one or more slow-moving or stopped objects, wherein the refining maximizing a correlation score on a joint domain of an appearance of the candidate object, a shape of the candidate object, a movement behavior of the candidate object, and a difference map.

9. The method of claim 8, wherein the refining comprises:

maintaining an appearance patch surrounding the candidate object in the existing track, the appearance patch containing accumulated appearance information for the candidate object; and

maintaining a shape patch surrounding the candidate object in the existing track, the shape patch containing a soft object mask for the candidate object; and

selecting a number of locations around the predicted location, wherein the number of locations comprises a fixed number of locations that best maximize the correlation score.

10. The method of claim 9, wherein each location in the number of locations is added as a new candidate object to be tracked.

11. The method of claim 4, wherein the associating comprises:

obtaining a first graph that represents a set of existing tracks including the existing track in a previous frame of the series of input video frames;

obtaining a second graph that represents a set of candidate objects including the candidate object in a current frame of the series of input video frames; and

producing a plurality of mappings between the first graph and the second graph.

12. The method of claim 11, wherein the plurality of mappings comprises:

a first mapping that maps the existing track in the first graph to the candidate object in the second graph.

13. The method of claim 12, wherein the producing comprises:

matching the candidate object to the existing track using at least one vertex matching feature and at least one pairwise spatial matching feature.

14. The method of claim 13, wherein the at least one vertex matching feature comprises at least one of:

color, gradient, shape, or movement characteristic.

15. The method of claim 13, wherein the at least one pairwise spatial matching feature is a measure of difference between the candidate object and a object indicated by the existing track.

16. The method of claim 15, wherein the at least one pairwise spatial matching feature comprises at least one of:

a distance between the candidate object and the object indicated by the existing track or a difference in appearance between the candidate object and the object indicated by the existing track.

17. The method of claim 4, further comprising:

updating a track model for use in detecting a new candidate object in a new series of input video frames, wherein the updating is responsive to the associating.

18. The method of claim 4, a behavior of the candidate object is modeled as a set of constraints that regulate edge probability and vertex probability in the joint probabilistic graph matching framework.

19. The method of claim 18, wherein the set of constraints comprises at least one of:

a distance constraint, a road constraint, a speed constraint, or a collision constraint.

20. A computer readable storage device containing an executable program for detecting and tracking an object in a field of view, where the program performs steps comprising:

detecting a candidate object in a series of input video frames depicting the field of view; and

associating an existing track with the candidate object in accordance with a joint probabilistic graph matching framework.