Abnormal behavior detection system and method using automatic classification of multiple features

• US 8,885,929 B2
• Filed: 11/29/2010
• Issued: 11/11/2014
• Est. Priority Date: 06/08/2010
• Status: Active Grant

• Current Assignee: Gorilla Technology UK Limited
• Original Assignee: Gorilla Technology, Inc. (Gorilla Science & Technology, Inc.)
• Inventors: Ni, Sze-Yao, Lan, Yuang-Tzong, Chen, Yi-Wei, Lin, Tom C. I.
• Primary Examiner(s): Koziol, Stephen R
• Assistant Examiner(s): LE, TOTAM HA
• Application Number:
  US12/955,049
• Publication Number:
  US 20120134532A1
• Time in Patent Office: 1,443 Days
• Field of Search: 382/158, 382/159, 382/190, 382/209
• US Class Current: 382/159
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G08G 1/0175 by photographing vehicles, e.g. when violating traffic rules

Abstract

Described herein are a system and a method for abnormal behavior detection using automatic classification of multiple features. Features from various sources, including those extracted from camera input through digital image analysis, are used as input to machine learning algorithms. These algorithms group the features and produce models of normal and abnormal behaviors. Outlying behaviors, such as those identified by their lower frequency, are deemed abnormal. Human supervision may optionally be employed to ensure the accuracy of the models. Once created, these models can be used to automatically classify features as normal or abnormal. This invention is suitable for use in the automatic detection of abnormal traffic behavior such as running of red lights, driving in the wrong lane, or driving against traffic regulations.

Claims

1. A non-transitory computer storage medium storing a behavior detection program using an automatic classification of multiple features, comprising:

extracting a set of features associated with each of a plurality of objects respectively from monitoring data;

establishing at least one behavioral model for classifying at least one behavior type of objects after a cluster analysis based on similarities among the separated sets of features;

determining a behavior type of an object according to an extracted set of features of said object and the at least one behavior model; and

outputting the determined behavior type of the said object.

2. The non-transitory computer storage medium of claim 1, wherein the set of features further comprises:

spatial information extracted by a spatial feature extraction module and temporal information extracted by a temporal feature extraction module.

3. The non-transitory computer storage medium of claim 2, wherein the behavior detection program is applied to detect a traffic violation, and the spatial feature extraction module comprises:

a traffic signal status detection subunit, for obtaining the status of the traffic signal by an image analysis process; and

a vehicle position detection subunit, for obtaining one or more positions of one or more vehicles through the image analysis process.

4. The non-transitory computer storage medium of claim 2, wherein the behavior detection program is applied to detect the traffic violation, and the spatial feature extraction module comprises a vehicle type detection subunit for detecting the type of each vehicle by an image analysis technique.

5. The non-transitory computer storage medium of claim 2, wherein the behavior detection program is applied to detect the traffic violation, and the temporal feature extraction module comprises a vehicle tracking subunit for obtaining position distribution of one or more vehicles along a continuous timeline and the track of each vehicle.

6. The non-transitory computer storage medium of claim 1, wherein the monitoring data is received from each of a plurality of sensors distributed in the monitoring environment.

7. The non-transitory computer storage medium of claim 1, wherein the outputting comprises at least one of a storage device, a display and a transmitter.

16. The non-transitory computer storage medium of claim 1, wherein the set of features associated with the object comprises an object feature and an environmental feature which is extracted from where the object is in.

8. A behavior detection method which performs automatic classification based on multiple features, comprising:

extracting a set of features associated with each of a plurality of objects respectively from the monitoring data;

performing a cluster analysis based on similarities among the separated sets of features;

establishing at least one behavior model for classifying at least one behavior type of objects based on the result of the cluster analysis;

determining a behavior type of an object according to an extracted set of features of said object and the at least one behavior model; and

outputting the determined behavior of the said object.

9. The method of claim 8, wherein the set of features of an object includes both spatial and temporal information.

10. The method of claim 8, wherein performing the cluster analysis comprises grouping objects that have similar environmental features, trajectory, or object features.

11. The method of claim 8, wherein the monitoring data is extracted from each of a plurality of sensors distributed in the monitoring environment.

12. The method of claim 11, wherein the monitoring data includes a road monitoring image and the status of a traffic signal.

13. The method of claim 8, wherein establishing the at least one behavior model comprises using a supervised learning scheme to define a behavior type as an abnormal behavior.

14. The method of claim 8, wherein the behavior type of each of the plurality of the objects comprises a normal behavior and an abnormal behavior, and the at least one behavior mode comprises a normal behavior model, wherein the behavior type of the said object is regarded as normal, if the extracted set of features of the said object fits the normal behavior model;

and the behavior type of the said object is regarded as abnormal, if the extracted set of features of the said object does not fit any normal behavior model.

15. The method of claim 8, wherein performing the cluster analysis further comprises dividing the plurality of objects into a plurality of groups, wherein each group has a similar set of features;

and the behavior model of each group is established based on the probability of the group among the plurality of groups.

17. The method of claim 8, wherein the set of features associated with the object comprises an object feature and an environmental feature which is extracted from where the object is in.