CLUSTERING-BASED OBJECT CLASSIFICATION

• US 20130170696A1
• Filed: 12/28/2011
• Published: 07/04/2013
• Est. Priority Date: 12/28/2011
• Status: Active Grant

• Current Assignee: Pelco Incorporated (Motorola Solutions, Inc.)
• Original Assignee: Pelco Incorporated (Motorola Solutions, Inc.)
• Inventors: Zhu, Hongwei, Aghdasi, Farzin, Millar, Greg
• Granted Patent: US 8,744,125 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06V 10/75 Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries

Abstract

An example of a method for identifying objects in video content according to the disclosure includes receiving video content of a scene captured by a video camera, detecting an object in the video content, identifying a track that the object follows over a series of frames of the video content, extracting object features for the object from the video content, and classifying the object based on the object features. Classifying the object further comprises: determining a track-level classification for the object using spatially invariant object features, determining a global-clustering classification for the object using spatially variant features, and determining an object type for the object based on the track-level classification and the global-clustering classification for the object.

Claims

1. A method for identifying objects in video content comprising:

receiving video content of a scene captured by a video camera;

detecting an object in the video content;

identifying a track that the object follows over a series of frames of the video content;

extracting object features for the object from the video content; and

classifying the object based on the object features, wherein classifying the object further comprises;

determining a track-level classification for the object using spatially invariant object features;

determining a global-clustering classification for the object using spatially variant features; and

determining an object type for the object based on the track-level classification and the global-clustering classification for the object.

2. The method of claim 1 wherein determining a track-level classification for the object further comprises determining the track-level classification for the object based on aspect ratios associated with the object.

3. The method of claim 2 wherein determining the track-level classification for the object further comprises determining the track-level classification for the object based on the aspect ratio associated with the object and a directional aspect ratio associated with the object.

4. The method of claim 2 wherein determining the track-level classification for the object further comprises:

constructing directional clusters associated with the aspect ratio for the tracked object;

constructing directional clusters associated with directional aspect ratio for the tracked object;

determining the track-level classification for the object based on the directional clusters associated with the aspect ratio and the directional clusters associated with directional aspect ratio; and

updating a histogram of track-level classification results for the tracked object based on the track-level classification.

5. The method of claim 1 wherein determining the global-clustering classification for the object further comprises determining the global-clustering classification for the object based on a size of the object.

6. The method of claim 5 wherein determining the global-clustering classification for the object further comprises:

updating local models of object size for locations visited by a persistently tracked object;

updating global clusters by associating local models with the global clusters, the local models having an object size matching that associated with the global cluster and are visited by the persistently tracked object, and wherein updating the global clusters further comprises;

updating a histogram of track-level classification results associated with matched objects;

updating directional clusters based on the matched objects, the directional clusters being associated with directional aspect ratio and aspect ratio of tracked objects;

creating a feature map of object size for each global cluster to resolve conflicts in global cluster merges; and

creating cluster-to-type mapping to convert a global cluster to a determined object type.

7. The method of claim 1 further comprising:

determining whether the object has moved consistently in one direction for at least a predetermined threshold distance; and

if the object has not moved more than the predetermined threshold distance, determining the object type for the object based on the global-clustering classification and not the track-level classification of the object.

8. A surveillance system configured to identify objects in video content captured by a video camera, the system comprising:

means for receiving video content of a scene captured by a video camera;

means for detecting an object in the video content;

means for identifying a track that the object follows over a series of frames of the video content;

means for extracting object features for the object from the video content; and

means for classifying the object based on the object features, wherein classifying the object further comprises;

means for determining a track-level classification for the object using spatially invariant object features;

means for determining a global-clustering classification for the object using spatially variant features; and

means for determining an object type for the object based on the track-level classification and the global-clustering classification for the object.

9. The system of claim 8 wherein the means for determining a track-level classification for the object further comprises means for determining the track-level classification for the object based on aspect ratios associated with the object.

10. The system of claim 9 wherein the means for determining the track-level classification for the object further comprises means for determining the track-level classification for the object based on the aspect ratio associated with the object and a directional aspect ratio associated with the object.

11. The system of claim 9 wherein the means for determining the track-level classification for the object further comprises:

means for constructing directional clusters associated with the aspect ratio for the tracked object; and

means for constructing directional clusters associated with directional aspect ratio for the tracked object; and

means for determining the track-level classification for the object based on the directional clusters associated with the aspect ratio and the directional clusters associated with directional aspect ratio; and

means for updating a histogram of track-level classification results for the tracked object based on the track-level classification.

12. The system of claim 8 wherein the means for determining the global-clustering classification for the object further comprises means for determining the global-clustering classification for the object based on a size of the object.

13. The system of claim 12 wherein the means for determining the global-clustering classification for the object further comprises:

means for updating local models of object size for locations visited by a persistently tracked object;

means for updating global clusters by associating local models with the global clusters, the local models having an object size matching that associated with the global cluster and are visited by the persistently tracked object, and wherein the means for updating the global clusters further comprises;

means for updating a histogram of track-level classification results associated with matched objects;

means for updating directional clusters based on the matched objects, the directional clusters being associated with directional aspect ratio and aspect ratio of tracked objects;

means for creating a feature map of object size for each global cluster to resolve conflicts in global cluster merges; and

means for creating cluster-to-type mapping to convert a global cluster to a determined object type.

14. The system of claim 8 further comprising:

means for determining whether the object has moved consistently in one direction for at least a predetermined threshold distance; and

means for determining the object type for the object based on the global-clustering classification and not the track-level classification of the object if the object has not moved more than the predetermined threshold distance.

15. A surveillance system for identifying objects in video content captured by a video camera, the system comprising:

a tangible, non-transitory computer-readable memory;

a plurality of modules comprising processor executable code stored in the memory;

a processor connected to the memory and configured to access the plurality of modules stored in the memory; and

a video processing module configured to;

receive video content of a scene captured by a video camera;

detect an object in the video content;

identify a track that the object follows over a series of frames of the video content;

extract object features for the object from the video content; and

classify the object based on the object features, wherein to classify the object the video processing module is further configured to;

determine a track-level classification for the object using spatially invariant object features;

determine a global-clustering classification for the object using spatially variant features; and

determine an object type for the object based on the track-level classification and the global-clustering classification for the object.

16. The system of claim 15 wherein the video processing module being configured to determine a track-level classification for the object further is further configured to determine the track-level classification for the object based on aspect ratios associated with the object.

17. The system of claim 16 wherein the video processing module being configured to determine the track-level classification for the object is further configured to determine the track-level classification for the object based on the aspect ratio associated with the object and a directional aspect ratio associated with the object.

18. The system of claim 16 wherein the video processing module being configured to determine the track-level classification for the object is further configured to:

construct directional clusters associated with the aspect ratio for the tracked object;

construct directional clusters associated with directional aspect ratio for the tracked object;

determine the track-level classification for the object based on the directional clusters associated with the aspect ratio and the directional clusters associated with directional aspect ratio; and

update a histogram of track-level classification results for the tracked object based on the track-level classification.

19. The system of claim 15 wherein the video processing module being configured to determine the global-clustering classification for the object is further configured to determine the global-clustering classification for the object based on a size of the object.

20. The system of claim 19 wherein the video processing module being configured to determine the global-clustering classification for the object is further configured to:

update local models of object size for locations visited by a persistently tracked object;

update global clusters by associating local models with the global clusters, the local models having an object size matching that associated with the global cluster and are visited by the persistently tracked object, and wherein the video processing module being configured to update the global clusters is further configured to;

update a histogram of track-level classification results associated with matched objects;

update directional clusters based on the matched objects, the directional clusters being associated with directional aspect ratio and aspect ratio of tracked objects;

create a feature map of object size for each global cluster to resolve conflicts in global cluster merges; and

create cluster-to-type mapping to convert a global cluster to a determined object type.

21. The system of claim 15 wherein the video processing module is further configured to:

determine whether the object has moved consistently in one direction for at least a predetermined threshold distance; and

determine the object type for the object based on the global-clustering classification and not the track-level classification of the object if the object has not moved more than the predetermined threshold distance.

22. A tangible computer-readable medium, having stored thereon computer-readable instructions identifying objects in video content, comprising instructions configured to cause a computer to:

receive video content of a scene captured by a video camera;

detect an object in the video content;

identify a track that the object follows over a series of frames of the video content;

extract object features for the object from the video content; and

classify the object based on the object features, wherein the instruction to cause the computer to classify the object further comprise instructions to cause the computer to;

determine a track-level classification for the object using spatially invariant object features;

determine a global-clustering classification for the object using spatially variant features; and

determine an object type for the object based on the track-level classification and the global-clustering classification for the object.

23. The medium of claim 22 wherein the code to cause the computer to determine a track-level classification for the object further comprises code to cause the computer to determine the track-level classification for the object based on aspect ratios associated with the object.

24. The medium of claim 23 wherein the code to cause the computer to determine the track-level classification for the object further comprises code to cause the computer to determine the track-level classification for the object based on the aspect ratio associated with the object and a directional aspect ratio associated with the object.

25. The medium of claim 23 wherein the code to cause the computer to determine the track-level classification for the object further comprises code to cause the computer to:

construct directional clusters associated with the aspect ratio for the tracked object;

construct directional clusters associated with directional aspect ratio for the tracked object;

determine the track-level classification for the object based on the directional clusters associated with the aspect ratio and the directional clusters associated with directional aspect ratio; and

update a histogram of track-level classification results for the tracked object based on the track-level classification.

26. The medium of claim 22 wherein the code to cause the computer to determine the global-clustering classification for the object further comprises code to cause the computer to determine the global-clustering classification for the object based on a size of the object.

27. The medium of claim 26 wherein the code to cause the computer to determine the global-clustering classification for the object further comprises code to cause the computer to:

update local models of object size for locations visited by a persistently tracked object; and

update global clusters by associating local models with the global clusters, the local models having an object size matching that associated with the global cluster and are visited by the persistently tracked object, and wherein the code to update the global clusters further comprises code to cause the computer to;

update a histogram of track-level classification results associated with matched objects;

update directional clusters based on the matched objects, the directional clusters being associated with directional aspect ratio and aspect ratio of tracked objects;

create a feature map of object size for each global cluster to resolve conflicts in global cluster merges; and

create cluster-to-type mapping to convert a global cluster to a determined object type.

28. The medium of claim 22 further comprising code to cause the computer to determine:

determine whether the object has moved consistently in one direction for at least a predetermined threshold distance; and

determine the object type for the object based on the global-clustering classification and not the track-level classification of the object if the object has not moved more than the predetermined threshold distance.