Foreground object tracking

• US 8,374,393 B2
• Filed: 07/10/2012
• Issued: 02/12/2013
• Est. Priority Date: 09/01/2009
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: Behavioral Recognition Systems Incorporated
• Inventors: Cobb, Wesley Kenneth, Seow, Ming-Jung, Yang, Tao
• Primary Examiner(s): Tucker, Wesley
• Assistant Examiner(s): Bitar, Nancy
• Application Number:
  US13/545,950
• Publication Number:
  US 20120275649A1
• Time in Patent Office: 217 Days
• Field of Search: None
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position

Abstract

Techniques are disclosed for detecting foreground objects in a scene captured by a surveillance system and tracking the detected foreground objects from frame to frame in real time. A motion flow field is used to validate foreground objects(s) that are extracted from the background model of a scene. Spurious foreground objects are filtered before the foreground objects are provided to the tracking stage. The motion flow field is also used by the tracking stage to improve the performance of the tracking as needed for real time surveillance applications.

Claims

1. A computer-implemented method for tracking foreground objects depicted in a sequence of video frames captured by a video camera, the method comprising:

identifying a first foreground patch in a current video frame of the sequence of video frames;

determining a motion flow field for a first foreground patch relative to at least a previous video frame of the sequence of video frames;

upon determining the motion flow field for the first foreground patch corresponds to a motion history of a first foreground object tracked in the sequence of video frames, associating the first foreground patch with the first foreground object; and

upon determining the motion flow field for the first foreground patch corresponds to a motion history of the first foreground object and a motion history of least a second foreground object tracked in the sequence of video frames, computing a covariance matrix for the foreground patch in the current video frame and each of the first and second foreground objects as depicted in the previous video frame.

2. The method of claim 1, further comprising, selecting, based on the covariance matrices computed for the foreground patch and the first and second foreground objects, to associate the first foreground patch with one of the first and the second foreground objects.

3. The method of claim 2, further comprising, updating tracking data associated with the selected foreground object based on the first foreground patch.

4. The method of claim 1, further comprising, upon determining the motion flow field for the first foreground patch does not correspond to the motion history of the first or second foreground objects:

identifying the first foreground patch as a new foreground object appearing in the sequence of video frames; and

creating a motion history corresponding to the first foreground patch.

5. The method of claim 1, wherein the covariance matrices compare red, green, and blue color components for pixels of the first foreground patch with red, green, and blue color components for pixels of the first and second foreground objects in the previous video frame.

6. The method of claim 1, further comprising, upon determining the motion flow field for the first foreground patch corresponds to the motion history of the first foreground object, updating tracking data associated with the first foreground object based on the first foreground patch.

7. The method of claim 1, wherein the motion flow field for the current video frame includes an angle and magnitude value for the first detected foreground patch in the video frame relative to the first foreground object in the previous video frame.

8. A non-transitory computer-readable storage medium containing a program, which when executed by a processor, performs an operation for tracking foreground objects depicted in a sequence of video frames captured by a video camera, the operation comprising:

identifying a first foreground patch in a current video frame of the sequence of video frames;

determining a motion flow field for a first foreground patch relative to at least a previous video frame of the sequence of video frames;

upon determining the motion flow field for the first foreground patch corresponds to a motion history of a first foreground object tracked in the sequence of video frames, associating the first foreground patch with the first foreground object; and

upon determining the motion flow field for the first foreground patch corresponds to a motion history of the first foreground object and a motion history of least a second foreground object tracked in the sequence of video frames, computing a covariance matrix for the foreground patch in the current video frame and each of the first and second foreground objects as depicted in the previous video frame.

9. The computer-readable storage medium of claim 8, wherein the operation further comprises, selecting, based on the covariance matrices computed for the foreground patch and the first and second foreground objects, to associate the first foreground patch with one of the first and the second foreground objects.

10. The computer-readable storage medium of claim 9, wherein the operation further comprises, updating tracking data associated with the selected foreground object based on the first foreground patch.

11. The computer-readable storage medium of claim 8, wherein the operation further comprises, upon determining the motion flow field for the first foreground patch does not correspond to the motion history of the first or second foreground objects:

identifying the first foreground patch as a new foreground object appearing in the sequence of video frames; and

creating a motion history corresponding to the first foreground patch.

12. The computer-readable storage medium of claim 8, wherein the covariance matrices compare red, green, and blue color components for pixels of the first foreground patch with red, green, and blue color components for pixels of the first and second foreground objects in the previous video frame.

13. The computer-readable storage medium of claim 8, wherein the operation further comprises, upon determining the motion flow field for the first foreground patch corresponds to the motion history of the first foreground object, updating tracking data associated with the first foreground object based on the first foreground patch.

14. The computer-readable storage medium of claim 8, wherein the motion flow field for the current video frame includes an angle and magnitude value for the first detected foreground patch in the video frame relative to the first foreground object in the previous video frame.

15. A system, comprising:

a video input source configured to provide a sequence of video frames;

a processor; and

a memory containing a program, which, when executed by the processor, performs an operation for tracking foreground objects depicted in a sequence of video frames captured by the video input source, the operation comprising;

identifying a first foreground patch in a current video frame of the sequence of video frames,

determining a motion flow field for a first foreground patch relative to at least a previous video frame of the sequence of video frames,

upon determining the motion flow field for the first foreground patch corresponds to a motion history of a first foreground object tracked in the sequence of video frames, associating the first foreground patch with the first foreground object, and

upon determining the motion flow field for the first foreground patch corresponds to a motion history of the first foreground object and a motion history of least a second foreground object tracked in the sequence of video frames, computing a covariance matrix for the foreground patch in the current video frame and each of the first and second foreground objects as depicted in the previous video frame.

16. The system of claim 15, wherein the operation further comprises, selecting, based on the covariance matrices computed for the foreground patch and the first and second foreground objects, to associate the first foreground patch with one of the first and the second foreground objects.

17. The system of claim 16, wherein the operation further comprises, updating tracking data associated with the selected foreground object based on the first foreground patch.

18. The system of claim 15, wherein the operation further comprises, upon determining the motion flow field for the first foreground patch does not correspond to the motion history of the first or second foreground objects:

identifying the first foreground patch as a new foreground object appearing in the sequence of video frames; and

creating a motion history corresponding to the first foreground patch.

19. The system of claim 15, wherein the covariance matrices compare red, green, and blue color components for pixels of the first foreground patch with red, green, and blue color components for pixels of the first and second foreground objects in the previous video frame.

20. The system of claim 15, wherein the operation further comprises, upon determining the motion flow field for the first foreground patch corresponds to the motion history of the first foreground object, updating tracking data associated with the first foreground object based on the first foreground patch.

21. The system of claim 15, wherein the motion flow field for the current video frame includes an angle and magnitude value for the first detected foreground patch in the video frame relative to the first foreground object in the previous video frame.