FOREGROUND OBJECT DETECTION IN A VIDEO SURVEILLANCE SYSTEM

• US 20110052003A1
• Filed: 09/01/2009
• Published: 03/03/2011
• Est. Priority Date: 09/01/2009
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: Behavioral Recognition Systems Incorporated
• Inventors: YANG, TAO, COBB, WESLEY KENNETH, SEOW, MING-JUNG
• Granted Patent: US 8,218,819 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position

Abstract

Techniques are disclosed for detecting foreground objects in a scene captured by a surveillance system and tracking the detected foreground objects from frame to frame in real time. A motion flow field is used to validate foreground objects(s) that are extracted from the background model of a scene. Spurious foreground objects are filtered before the detected foreground objects are provided to the tracking stage. The motion flow field is also used by the tracking stage to improve the performance of the tracking as needed for real time surveillance applications.

Claims

1. A computer-implemented method for detecting a foreground patch that depicts a foreground object in a sequence of video frames captured by a video camera, the method comprising:

receiving a video frame in the sequence of video frame;

extracting a first foreground patch from the video frame to produce an extracted first foreground patch;

computing a motion flow field for the video frame that includes an angle and magnitude value corresponding to a feature in the video frame relative to the feature in a previous video frame;

filtering the motion flow field to produce a consistent motion flow field for the video frame that includes the extracted first foreground patch as a detected first foreground patch that includes the feature; and

tracking the first foreground object in the sequence of video frames based on the detected first foreground patch in order to follow the foreground object over time.

2. The method of claim 1, wherein the feature motion flow field includes only a single angle and only a single magnitude value for the feature.

3. The method of claim 2, wherein the single angle is an average of the angles in the motion flow field within the feature and the single magnitude value is an average of the magnitude values in the motion flow field within the feature.

4. The method of claim 1, wherein filtering the motion flow field includes determining that the extracted first foreground patch is detected for multiple video frames in the sequence of video frames.

5. The method of claim 1, further comprising generating motion history data for the video frame as a difference between a color component for each pixel in the video frame and the color component for each pixel in the previous video frame.

6. The method of claim 1, wherein the filtering includes merging an extracted second foreground patch with the extracted first foreground patch to produce the first detected foreground patch.

7. The method of claim 1, wherein the filtering includes splitting the extracted first foreground patch into a first portion that is the first detected foreground patch and a second portion that is a second detected foreground patch.

8. A computer-readable storage medium containing a program, which when executed on a processor, performs an operation for detecting a foreground patch that depicts a foreground object in a sequence of video frames captured by a video camera, the operation comprising:

receiving a video frame in the sequence of video frame;

extracting a first foreground patch from the video frame to produce an extracted first foreground patch;

computing a motion flow field for the video frame that includes an angle and magnitude value for each pixel corresponding to a feature in the video frame relative to the feature in a previous video frame;

filtering the motion flow field to produce a consistent motion flow field for the video frame that includes the extracted first foreground patch as a detected first foreground patch that includes the feature; and

tracking the first foreground object in the sequence of video frames based on the detected first foreground patch in order to follow the foreground object over time.

9. The computer-readable storage medium of claim 8, wherein the motion flow field includes only a single angle and only a single magnitude value for the feature.

10. The computer-readable storage medium of claim 9, wherein the single angle is an average of the angles within the feature and the single magnitude value is an average of the magnitude values within the feature.

11. The computer-readable storage medium of claim 8, wherein the filtering of the motion flow field includes determining that the extracted first foreground patch is detected for multiple video frames in the sequence of video frames.

12. The computer-readable storage medium of claim 8, further comprising generating motion history data for the video frame as a difference between a color component for each pixel in the video frame and the color component for each pixel in the previous video frame.

13. The computer-readable storage medium of claim 8, wherein the filtering includes merging an extracted second foreground patch with the extracted first foreground patch to produce the first detected foreground patch.

14. The computer-readable storage medium of claim 8, wherein the filtering includes splitting the extracted first foreground patch into a first portion that is the first detected foreground patch and a second portion that is a second detected foreground patch.

15. A system, comprising:

a video input source configured to provide a sequence of video frames;

a processor; and

a memory containing a program, which, when executed on the processor is configured to perform an operation for detecting a foreground patch that depicts a foreground object in the sequence of video frames, the operation comprising;

receiving a video frame in the sequence of video frame;

extracting a first foreground patch from the video frame to produce an extracted first foreground patch;

computing a motion flow field for the video frame that includes an angle and magnitude value for each pixel corresponding to a feature in the video frame relative to the feature in a previous video frame;

filtering the motion flow field to produce a consistent motion flow field for the video frame that includes the extracted first foreground patch as a detected first foreground patch that includes the feature; and

tracking the first foreground object in the sequence of video frames based on the detected first foreground patch in order to follow the foreground object over time.

16. The system of claim 15, wherein the motion flow field includes only a single angle and only a single magnitude value for the feature.

17. The system of claim 16, wherein the single angle is an average of the angles within the feature and the single magnitude value is an average of the magnitude values within the feature.

18. The system of claim 15, further comprising generating motion history data for the video frame as a difference between a color component for each pixel in the video frame and the color component for each pixel in the previous video frame.

19. The system of claim 15, wherein the filtering includes merging a second foreground patch with the extracted first foreground patch to produce the first detected foreground patch.

20. The system of claim 15, wherein the filtering includes splitting the extracted first foreground patch into a first portion that is the first detected foreground patch and a second portion that is a second detected foreground patch.