Object density estimation in video

• US 20080106599A1
• Filed: 11/21/2006
• Published: 05/08/2008
• Est. Priority Date: 11/23/2005
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: ObjectVideo Incorporated (Alarm.com Holdings, Inc.)
• Inventors: Martone, Andrew, Yu, Li, Haering, Niels, Zhang, Zhong, Yin, Weihong, Liu, Haiying, Rasheed, Zeeshan, Javed, Omar, Lipton, Alan, Venetianer, Peter
• Granted Patent: US 9,240,051 B2
• Time in Patent Office: Days
• US Class Current: 348/143
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/53 Recognition of crowd images, e.g. recognition of crowd congestion
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

A video camera may overlook a monitored area from any feasible position. An object flow estimation module monitor the moving direction of the objects in the monitored area. It may separate the consistently moving objects from the other objects. A object count estimation module may compute the object density (e.g. crowd). A object density classification module may classify the density into customizable categories.

Claims

1. A computer-readable medium comprising software for video surveillance, which when executed by a computer system, causes the computer system to perform operations comprising a method of:

receiving video from a video camera;

detecting features in the video;

estimating object count based on the features detected;

computing object density based on the object count; and

classifying the object density.

2. A computer-readable medium of claim 1, wherein the detecting features comprises:

extracting features from the video; and

removing background features from the features extracted.

3. A computer-readable medium of claim 1, wherein the detecting features further comprises:

building a background feature model; and

comparing the background feature model to the extracted features to determine foreground features, the foreground features corresponding to the detected features not in the background feature model.

4. A computer-readable medium of claim 3 further comprising software for video surveillance, which when executed by a computer system, causes the computer system to perform operations comprising a method of sub-sampling the foreground features into a lower resolution than a current resolution.

5. A computer-readable medium of claim 3, wherein building the background model comprises:

accumulating feature strength for each pixel in a window;

determining a minimum and maximum feature strength for the accumulated feature strength; and

classifying those pixels whose accumulated feature strength is larger than a threshold as background features.

6. A computer readable medium of claim 5, wherein the threshold is computed by determining a difference between the maximum and the minimum feature strength and the threshold is a multiple of the difference plus the minimum feature strength.

7. A computer-readable medium of claim 1, wherein the estimating object count comprises weighting the object count at different locations.

8. A computer-readable medium of claim 1, wherein computing the object density comprises:

defining an area of interest in the video; and

dividing the object count over the area of interest.

9. A computer-readable medium of claim 1, wherein classifying the object density comprises comparing the object density to object density thresholds and classifying the object density accordingly.

10. A computer-readable medium of claim 9, adaptively determining the object density thresholds.

11. A computer-readable medium of claim 1, wherein classifying the object density comprises:

determining a maximum and a minimum object density; and

dividing a range between the minimum and maximum into the classes based on the object density thresholds.

12. A computer-readable medium of claim 1, wherein computing the object density further comprises correcting areas from the object density calculation that have a density close to zero for a predetermined time.

13. A computer-readable medium of claim 11, wherein determining the object density thresholds further comprises:

recovering minimum and maximum object densities; and

adjusting the object density threshold based on the recovered minimum and maximum densities.

14. An apparatus to perform video surveillance, comprising:

at least one video camera; and

a video surveillance system coupled to the at least one video camera and comprising;

an object flow estimation module;

an object count estimation module; and

an object density classification module.

15. The apparatus according to claim 14, further comprising:

a object density control module coupled to the video surveillance system.

16. The apparatus according to claim 14, wherein the video surveillance system further comprises:

a pre-defined rules database.

17. The apparatus according to claim 14, wherein the object count module is configured to:

extracting features from the video; and

removing background features from the features extracted.

18. The apparatus according to claim 14, wherein the object count module is configured to:

building a background feature model; and

comparing the background feature model to the extracted features to determine foreground features, the foreground features corresponding to the detected features.

19. The apparatus according to claim 14, wherein the object count module is configured to:

sub-sample the foreground features into a lower resolution than a current resolution.

20. The apparatus according to claim 18, wherein the object count module is configured to:

accumulating pixel strength for each pixel in a window;

determining a minimum and maximum strength for the accumulated strength;

determining a difference between the maximum and the minimum;

determining for each pixel if the accumulated strength is larger than the threshold; and

classifying those pixels whose accumulated strength is larger than the threshold as background features.

21. The apparatus according to claim 14, wherein the object count module is configured to determining a difference between consecutive snapshots from the video.

22. The apparatus according to claim 14, wherein the object count module is configured to weighting the object count at different locations.

23. The apparatus according to claim 14, wherein the object count module is configured to:

defining an area of interest in the video; and

dividing the object count over the area of interest.

24. The apparatus according to claim 14, wherein the object density classification module is configured to:

comparing the object density to object density thresholds and classifying the object density accordingly.

25. The apparatus according to claim 24, wherein the object density classification module is configured to adaptively determining the object density thresholds.

26. The apparatus according to claim 14, wherein the object density classification module is configured to:

determining a maximum and a minimum object densities; and

dividing a range between the minimum and maximum into the classes based on the object density thresholds.

27. The apparatus according to claim 14, wherein the object count module is configured to correcting areas from the object density calculation that have a density close to zero for a predetermined time.

28. The apparatus according to claim 14, wherein the object density classification module is configured to:

recovering minimum and maximum object densities;

adjusting the object density threshold based on the recovered minimum and maximum densities.

29. A method of video surveillance, comprising:

estimating object flow based on video input, to obtain an estimated object flow;

estimating a count of objects based on the video input, to obtain an estimated object count; and

performing a classification of object density based on the estimated object count, to obtain a object density classification.

30. The method of claim 29, wherein at least one of the estimating object flow and the estimating a count of objects is performed based on a set of pre-defined rules.

31. The method of claim 29, further comprising:

performing object density control based on the estimated object flow, the object density classification, or both.

32. The method according to any one of claims 29, including at least one feature selected from the group consisting of:

detecting an approximate density of objects in a video scene of the video input;

detecting an approximate density of objects in a part of a video scene of the video input; and

creating at least one rule based on the object density classification.

33. The method of claim 29, further comprising:

utilizing the object density information as an input to a system selected from the group consisting of;

an alarm system;

a gating/access control system;

a crowd control system;

a traffic light system;

a traffic control system;

a transportation logistic system;

a work flow control system;

a inventory control system; and

a data analysis system.

34. The method according to claim 33, wherein the monitoring comprises an operation selected from the group consisting of:

detecting overcrowding/undercrowding on a railway platform;

detecting traffic flow on a road;

detection traffic flow on or in a traffic-carrying structure; and

determining crowd movement patterns.