Detecting moving objects in videos with corner-based background model

• US 20060171594A1
• Filed: 02/01/2005
• Published: 08/03/2006
• Est. Priority Date: 02/01/2005
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Zhu, Qiang, Avidan, Shmuel
• Granted Patent: US 7,373,012 B2
• Time in Patent Office: Days
• US Class Current: 382/224
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
• CPC Class Codes: G06V 10/443 by matching or filtering

Abstract

A computer implemented method models a background in a sequence of frames of a video. For each frame, the method detects static corners using an array of pixels of the frame, and extracts, for each static corner, features from a window of pixels around the static corner. For each static corner, a descriptor is determined from the corresponding features. Each static corner and corresponding descriptor is stored in a memory, and each static corner is classified as a background or foreground according to the descriptor to model a background in the video.

Claims

1. A computer implemented method for modeling a background in a sequence of frames of a video, each frame including an array of pixels, comprising:

detecting, for each frame, static corners using the array of pixels;

extracting, for each static corner, features from a window of pixels around the static corner;

determining, for each static corner, a descriptor from the corresponding features;

storing each static corner and corresponding descriptor in a memory; and

classifying each static corner as a background or foreground according to the descriptor to model a background in the video.

2. The method of claim 1, further comprising:

subtracting each pair of successive frames to generate motion images;

detecting, for each motion image, motion corners;

filtering, spatially and temporally, the foreground corners according to the motion corners.

3. The method of claim 2, in which a Harris corner detector detects the static corners and motion corners.

4. The method of claim 2, in which a number of static corners is approximately ten times a number of motion corners.

5. The method of claim 2, in which a minimum distance between any two static corners is about five pixels and a minimum distance between any two motion corners is about fifteen pixels.

6. The method of claim 1, in which the features for each static corner form a 256-dimensional vector.

7. The method of claim 6, further comprising:

normalizing the vector to unit length.

8. The method of claim 1, in which the features include gradient orientations and colors of the pixels in the window.

9. The method of claim 1, in which the descriptor includes a frequency that the corner occurs at a particular location in successive frames.

10. The method of claim 9, in which the frequency decreases over time if the corner is not detected in subsequent frames.

11. The method of claim 9, in which the descriptor includes a mean of the descriptor over time.

12. The method of claim 9, further comprising:

determining a correlation between a pair of descriptors, the correlation including a mean and a variance, a relatively low correlation indicating that a particular static corner is classified as the foreground.

13. The method of claim 12, in which a threshold is applied according to the variance to classify the corner.

14. The method of claim 12, in which the correlation is

$\mathrm{Correlation}=\frac{\sum _{}}{}$

i ⁢

U i ×

V i ∑

i ⁢

U i 2 ×

V i 2 ,

where U and V are the pair of descriptors.

15. The method of claim 14, in which the correlation is one if the two descriptors are identical.

16. The method of claim 2, in which the filtering counts a number of moving corners in a block of pixels, and deletes the moving corners if the count is less than a predetermined threshold count, and otherwise indicating a moving object.