Background modeling with feature blocks

• US 8,150,103 B2
• Filed: 09/03/2008
• Issued: 04/03/2012
• Est. Priority Date: 09/04/2007
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: ObjectVideo Incorporated (Alarm.com Holdings, Inc.)
• Inventors: Zhang, Zhong, Lipton, Alan J., Venetianer, Peter L., Yin, Weihong
• Primary Examiner(s): Koziol, Stephen
• Assistant Examiner(s): SHAH, UTPAL D
• Application Number:
  US12/203,514
• Publication Number:
  US 20090060277A1
• Time in Patent Office: 1,308 Days
• Field of Search: 382/103, 382/195, 382/173
• US Class Current: 382/103
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G08B 13/19606 Discriminating between target movement or movement in an area of interest and other non-signicative movements, e.g. target movements induced by camera shake or movements of pets, falling leaves, rotating fan

Abstract

Video content analysis of a video may include: modeling a background of the video; detecting at least one target in a foreground of the video based on the feature blocks of the video; and tracking each target of the video. Modeling a background of the video may include: dividing each frame of the video into image blocks; determining features for each image block of each frame to obtain feature blocks for each frame; determining a feature block map for each frame based on the feature blocks of each frame; and determining a background feature block map to model the background of the vide based on at least one of the feature block maps.

Claims

1. A computer-implemented method for video content analysis of a video comprising:

modeling a background of the video, comprising;

dividing each frame of the video into image blocks;

determining features for each image block of each frame to obtain feature blocks for each frame;

determining a feature block map for each frame based on the feature blocks of each frame; and

determining a background feature block map to model the background of the video based on at least one of the feature block maps;

detecting at least one target in a foreground of the video based on the feature blocks of the video; and

tracking each target of the video.

2. The method of claim 1, wherein determining features for each image block of each frame to obtain feature blocks for each frame comprises:

determining a pixel intensity for each image block;

placing pixels in each image block into a category based on the pixel intensity;

determining a pixel intensity gradient for each image block;

determining pixels in the image block having a high intensity or a low intensity;

counting a number of pixels determined to have the high intensity or low intensity; and

determining the feature based on the pixel intensity gradient and the counting.

3. The method of claim 1, further comprising determining a best reference frame based on the feature blocks.

4. The method of claim 1, further comprising:

determining jitter for each frame based on the reference feature block map and the feature block map of the frame; and

adjusting each frame based on the determined jitter.

5. The method of claim 1, wherein detecting at least one target comprises:

performing motion detection for the feature block map for each frame to obtain a first motion mask representing a string motion area and a second motion mask representing a weak motion area;

performing feature block region segmentation;

detecting high confident foreground feature blocks and high confident background feature blocks based on the motion detection;

performing AGC detection and compensation based on the high confident background feature blocks;

performing feature block change detection to obtain a first change mask representing a string change area and a second change mask representing a weak change area; and

detecting the target using the first and a second motion masks and the first and second change masks.

6. The method of claim 5, wherein performing AGC detection and compensation based on the high confident background feature blocks comprises:

determining an intensity difference graph;

determining if there is any AGC effect for the frame;

if so, generating a look up table for different intensity levels; and

performing AGC compensation for the background model.

7. The method of claim 1, wherein the features of the feature blocks are selected from the group consisting of:

pixel intensity mean, pixel intensity gradient, low intensity centroid, high intensity centroid, low intensity pixel count, high intensity pixel count, and feature block label.

8. A computer-implemented method for video content analysis of a video comprising:

modeling a background of the video, comprising;

dividing each frame of the video into image blocks;

determining features for each image block of each frame to obtain feature blocks for each frame;

determining a feature block map for each frame based on the feature blocks of each frame;

determining a reference feature block map based on the feature blocks of frames of the video;

determining jitter for each frame based on the reference feature block map and the feature block map of the frame; and

adjusting each frame based on the determined jitter;

detecting at least one target in a foreground of the video based on the feature blocks of the video; and

tracking each target of the video.

9. The method of claim 8, wherein determining jitter comprises:

selecting a reference feature block maps;

selecting a weight for feature blocks in the reference feature block map;

estimating coarse camera jitter; and

and estimating fine camera jitter.

10. The method of claim 9, wherein estimating coarse camera jitter comprising:

selecting horizontal and vertical strong texture feature blocks from the reference feature block map;

generating a horizontal and vertical shift graph using the horizontal strong texture feature blocks and the vertical strong texture feature blocks; and

estimating coarse horizontal frame shifts and coarse vertical frame shifts.

11. The method of claim 10, wherein estimating fine camera jitter comprises:

estimating local iterative camera jitter to obtain an error surface;

searching the error surface analysis to locate a minimum of a surface; and

determining a jitter estimation based on the minimum of the surface.

12. The method of claim 8, wherein determining features for each image block of each frame to obtain feature blocks for each frame comprises:

determining a pixel intensity for each image block;

placing pixels in each image block into a category based on the pixel intensity;

determining a pixel intensity gradient for each image block;

determining pixels in the image block having a high intensity or a low intensity;

counting a number of pixels determined to have the high intensity or low intensity; and

determining the feature based on the pixel intensity gradient and the counting.

13. The method of claim 8, further comprising determining a best reference frame based on the feature blocks.

14. The method of claim 8, wherein detecting at least one target comprises:

performing motion detection for the feature block map for each frame to obtain a first motion mask representing a string motion area and a second motion mask representing a weak motion area;

performing feature block region segmentation;

detecting high confident foreground feature blocks and high confident background feature blocks based on the motion detection;

performing AGC detection and compensation based on the high confident background feature blocks;

performing feature block change detection to obtain a first change mask representing a string change area and a second change mask representing a weak change area;

detecting the target using the first and a second motion masks and the first and second change masks.

15. The method of claim 14, wherein performing AGC detection and compensation based on the high confident background feature blocks comprises:

determining an intensity difference graph;

determining if there is any AGC effect for the frame;

if so, generating a look up table for different intensity levels; and

performing AGC compensation for the background model.

16. A computer-implemented method for video content analysis of a video comprising:

determining feature blocks for a video, comprising;

dividing each frame of the video into image blocks;

determining features for each image block of each frame to obtain feature blocks for each frame; and

determining a feature block map for each frame based on the feature blocks of each frame;

determining a background model of the video based on the feature block maps;

detecting at least one target in a foreground of the video based on the feature blocks of the video, comprising

identifying feature blocks of each frame that are likely background in the video as high-confident background feature blocks;

determining whether each frame has a video camera automatic gain control effect based on the high-confident background feature blocks; and

compensating for the video camera automatic gain control effect in each frame by adjusting the background model based on the high-confident background feature blocks; and

tracking each target of the video.