Object density estimation in video

• US 9,240,051 B2
• Filed: 11/21/2006
• Issued: 01/19/2016
• Est. Priority Date: 11/23/2005
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: Avigilon Fortress Corporation (Motorola Solutions, Inc.)
• Inventors: Liu, Haiying, Venetianer, Peter L., Haering, Niels, Javed, Omar, Lipton, Alan J., Martone, Andrew, Rasheed, Zeeshan, Yin, Weihong, Yu, Li, Zhang, Zhong
• Primary Examiner(s): MAI, KEVIN S
• Application Number:
  US11/602,490
• Publication Number:
  US 20080106599A1
• Time in Patent Office: 3,346 Days
• Field of Search: 348/154
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/53 Recognition of crowd images, e.g. recognition of crowd congestion
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

A video camera may overlook a monitored area from any feasible position. An object flow estimation module monitor the moving direction of the objects in the monitored area. It may separate the consistently moving objects from the other objects. A object count estimation module may compute the object density (e.g. crowd). A object density classification module may classify the density into customizable categories.

Claims

1. A non-transitory computer-readable medium comprising software for video surveillance, which when executed by a computer system, causes the computer system to perform operations comprising:

receiving video from a video camera;

detecting features in the video;

accumulating feature strength for each pixel in a window, wherein the feature strength is proportional to a number of times a feature is present over time;

classifying those pixels whose accumulated feature strength over time is larger than a threshold as background features;

comparing the background features to extracted features to determine foreground features, the foreground features corresponding to those features not classified as the background features;

estimating object count based on the foreground features detected;

computing a current object density based on the object count;

determining a current range of object densities by adjusting a maximum object density and a minimum object density, wherein adjusting the maximum object density and the minimum object density comprises;

subtracting a recovery density from the maximum object density if the maximum object density is greater than a first normal value; and

adding the recovery density to the minimum object density if the minimum object density is less than a second normal value;

replacing the minimum object density with the current object density if the current object density is less than the minimum object density;

replacing the maximum object density with the current object density if the current object density is greater than the maximum object density;

dynamically adjusting one or more object density thresholds based on the determined current range of object densities;

dividing the determined current range of object densities into a plurality of classes based on the one or more object density thresholds;

classifying the current object density based on the one or more dynamically adjusted object density thresholds; and

generating an alert based on determining that a classification of the current object density is associated with the alert.

2. The computer-readable medium of claim 1, wherein the detecting features comprises:

removing background features from the extracted features.

3. The computer-readable medium of claim 1, wherein the operations further comprise sub-sampling the foreground features into a lower resolution than a current resolution.

4. The computer readable medium of claim 1, wherein the threshold is computed by determining a difference between a maximum and a minimum feature strength and the threshold is a multiple of the difference plus the minimum feature strength.

5. The computer-readable medium of claim 1, wherein the estimating object count comprises weighting the object count at different locations to correct for a perspective of the video camera.

6. The computer-readable medium of claim 1, wherein computing the current object density comprises:

defining an area of interest in the video; and

dividing the object count over the area of interest.

7. The computer-readable medium of claim 1, wherein computing the current object density further comprises correcting areas from the current object density calculation that have a density close to zero for a predetermined time.

8. The computer-readable medium of claim 1, wherein dynamically adjusting the one or more object density thresholds further comprises:

adjusting the one or more object density thresholds based on at least one of the maximum object density and the minimum object density before the adjusting the maximum object density and the minimum object density.

18. The computer readable medium of claim 1, wherein the method performed further comprises:

generating an alert if the current object density is above or below a predetermined level for a prescribed period of time.

9. An apparatus to perform video surveillance, comprising:

at least one video camera; and

a video surveillance system coupled to the at least one video camera and comprising;

a processing system comprising one or more processors; and

a memory system comprising one or more computer-readable media, wherein the one or more computer-readable media contain instructions that when executed by the processing system, cause the processing system to perform operations comprising;

accumulating feature strength for each pixel in a window from a video, wherein the feature strength is proportional to a number of times a feature is present over time;

classifying those pixels whose accumulated feature strength over time is larger than a threshold as background features;

comparing the background features to extracted features to determine foreground features, the foreground features corresponding to those features not classified as the background features; and

estimating object count based on the foreground features detected;

computing a current object density based on the object count;

determining a current range of object densities by adjusting a maximum object density and a minimum object density, wherein adjusting the maximum object density and the minimum object density comprises;

subtracting a recovery density from the maximum object density if the maximum object density is greater than a first normal value; and

adding the recovery density to the minimum object density if the minimum object density is less than a second normal value;

replacing the minimum object density with the current object density if the current object density is less than the minimum object density;

replacing the maximum object density with the current object density if the current object density is greater than the maximum object density;

dynamically adjusting one or more object density thresholds based on the determined current range of object densities;

dividing the determined current range of object densities into a plurality of classes based on the one or more object density thresholds;

classifying the current object density based on the one or more dynamically adjusted object density thresholds; and

generating an alert based on determining that a classification of the current object density is associated with the alert.

10. The apparatus according to claim 9, wherein the video surveillance system further comprises:

a pre-defined rules database.

11. The apparatus according to claim 9, the operations further comprising:

removing background features from the extracted features.

12. The apparatus according to claim 9, the operations further comprising:

sub-sampling the foreground features into a lower resolution than a current resolution.

13. The apparatus according to claim 9, the operations further comprising determining a difference between consecutive snapshots from the video.

14. The apparatus according to claim 9, the operations further comprising weighting the object count at different locations to correct for a perspective of the video camera.

15. The apparatus according to claim 9, the operations further comprising:

defining an area of interest in the video; and

dividing the object count over the area of interest.

16. The apparatus according to claim 9, the operations further comprising correcting areas from the current object density calculation that have a density close to zero for a predetermined time.

17. The apparatus according to claim 9, the operations further comprising:

adjusting the one or more object density thresholds based on at least one of the maximum object density and the minimum object density before the adjusting the maximum object density and the minimum object density.

19. The apparatus of claim 9, the operations further comprising monitoring a moving direction of objects.