SCENE ACTIVITY ANALYSIS USING STATISTICAL AND SEMANTIC FEATURES LEARNT FROM OBJECT TRAJECTORY DATA

• US 20120170802A1
• Filed: 12/30/2010
• Published: 07/05/2012
• Est. Priority Date: 12/30/2010
• Status: Active Grant

• Current Assignee: Pelco (Schneider Electric SE)
• Original Assignee: Pelco (Schneider Electric SE)
• Inventors: Millar, Greg, Aghdasi, Farzin, Zhu, Hongwei
• Granted Patent: US 8,855,361 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats

Abstract

Trajectory information of objects appearing in a scene can be used to cluster trajectories into groups of trajectories according to each trajectory'"'"'s relative distance between each other for scene activity analysis. By doing so, a database of trajectory data can be maintained that includes the trajectories to be clustered into trajectory groups. This database can be used to train a clustering system, and with extracted statistical features of resultant trajectory groups a new trajectory can be analyzed to determine whether the new trajectory is normal or abnormal. Embodiments described herein, can be used to determine whether a video scene is normal or abnormal. In the event that the new trajectory is identified as normal the new trajectory can be annotated with the extracted semantic data. In the event that the new trajectory is determined to be abnormal a user can be notified that an abnormal behavior has occurred.

Claims

1. A method for providing trajectory information about a video scene, the method comprising:

obtaining video scene metadata that includes information regarding trajectories for a plurality of objects moving through the video scene;

clustering each of the plurality of trajectories into trajectory groups according to their relative distance from each other;

extracting statistical features about each trajectory group;

classifying each trajectory in the plurality of trajectories as a normal trajectory or an abnormal trajectory based at least in part on the clustering;

extracting semantic data about each trajectory group or each normal trajectory; and

annotating each trajectory with the semantic data.

2. The method according to claim 1, wherein the clustering comprises a plurality of clustering iterations using a plurality of different distance measures.

3. The method according to claim 1, wherein the distance measure comprises a distance measure selected from the group consisting of a minimum distance between trajectories, a maximum distance between trajectories, and an average distance between trajectories.

4. The method according to claim 1, wherein the statistical features include features selected from the listing consisting of:

average and standard deviation of object height, object width, object size, object speed, object appearance color, object shape, trajectory length, trajectory duration, as well as moving direction probability, spatial occurrence probably, temporal occurrence probably.

5. The method according to claim 1, wherein the semantic data includes data selected from the group consisting of entry zones, exit zones, representative trajectories, pivotal points and directions of trajectory patterns, and crossing zones between different trajectory patterns.

6. The method according to claim 1, in the event a trajectory is classified as a normal trajectory further comprising annotating a normal trajectory based on the trajectories association with a classified group of trajectories.

7. The method according to claim 1, wherein a normal trajectory is a trajectory with statistical features similar to other trajectories within a group.

8. The method according to claim 1, wherein an abnormal trajectory is a trajectory that is unassociated with a group of trajectories.

9. A computer program product residing on a non-transitory processor-readable medium and comprising instructions configured to cause a processor to:

cluster each of a plurality of trajectories into trajectory groups according to their relative distance from each other;

extract statistical features about each trajectory group;

classify each trajectory in the plurality of trajectories as a normal trajectory or an abnormal trajectory based at least in part on the clustering;

extract semantic data from each trajectory group or each normal trajectory; and

annotate each normal trajectory with the semantic data.

10. The computer program product of claim 9, wherein the instructions to cause the processor to cluster each of plurality of trajectories into trajectory groups comprise instructions configured to cause the processor to iteratively cluster the plurality of trajectories using different distance measures.

11. The computer program product of claim 9, wherein in the event a trajectory is classified as a normal trajectory further comprising annotating a normal trajectory based on the trajectories association with a classified group of trajectories.

12. A trajectory analysis system comprising:

an input configured to receive video scene metadata that includes information regarding the trajectories of objects as the objects move within the video scene;

a memory; and

a processing unit communicatively coupled to the input and the memory and configured to;

cluster each trajectory into a trajectory group according to the trajectory'"'"'s relative distance from previously analyzed trajectories;

identify each trajectory in the plurality of trajectories as a normal trajectory or an abnormal trajectory based at least in part on the clustering; and

annotate each normal trajectory as a normal trajectory or an abnormal trajectory.

13. The trajectory analysis system according to claim 12 wherein the processing unit is configured to extract statistical features for each trajectory group, extract semantic data from each trajectory group or each normal trajectory and annotate the normal trajectory with the semantic data.

14. The trajectory analysis system according to claim 12 further comprises a user interface wherein the processing unit is configured to notify the user through the user interface in the event a trajectory is an abnormal trajectory.

15. The trajectory analysis system according to claim 12, wherein the trajectory analysis system comprises a camera.

16. A method for determining abnormal behavior in a video scene, the method comprising:

creating a database of normal trajectories from video data of a scene that includes a plurality of objects moving through the scene along a trajectory, wherein a normal trajectory comprises a trajectory that is clustered with a plurality of previous trajectories within the video scene;

identifying a new trajectory within the video scene;

determining whether the new trajectory is a normal trajectory; and

in the event the new trajectory is not a normal trajectory, notifying a user.

17. The method according to claim 16, wherein a normal trajectory comprises a trajectory whose distance is near a plurality of previous trajectories within a group in terms of statistical feature similarities.

18. The method according to claim 16, wherein the creating a database of normal trajectories comprises:

clustering each of the plurality of trajectories into trajectory groups according to their relative distance from each other;

classifying each trajectory in the plurality of trajectories as a normal trajectory or an abnormal trajectory based at least in part on the clustering; and

saving the results of the classification in a database.

19. The method according to claim 16, wherein the determining whether the new trajectory is a normal trajectory comprises determining the relative distance in terms of statistical feature similarities between the new trajectory and one or more of the normal trajectories stored within the database.

20. A trajectory analysis system comprising:

an input configured to receive video scene metadata that includes information regarding the trajectories of objects as the objects move within the video scene;

a user interface;

a memory configured to store normal trajectories from video data of a scene that includes a plurality of objects moving through the scene along a trajectory, wherein a normal trajectory comprises a trajectory that is clustered with a plurality of previous trajectories within the video scene; and

a processing unit communicatively coupled to the input and the memory and configured to;

identify a new trajectory within the video scene;

determine whether the new trajectory is a normal trajectory; and

in the event the new trajectory is not a normal trajectory, notify a user through the user interface.

21. The trajectory analysis system according to claim 20, wherein a new trajectory is identified as a normal trajectory when the new trajectory is near a plurality of previous trajectories.

22. The trajectory analysis system according to claim 21, wherein the new trajectory is identified as being near one of a plurality of trajectories based on the relative distance from the new trajectory and the plurality of trajectories.

23. The trajectory analysis system according to claim 20, wherein a new trajectory is identified as a normal trajectory when the new trajectory is similar to trajectories in a cluster of trajectories based on similarities between any of the direction, speed, or appearing time of the new trajectory and any trajectory in the cluster of trajectories.

24. The trajectory analysis system according to claim 21, wherein the relative distance comprises a distance measure selected from the group consisting of a minimum distance between trajectories, a maximum distance between trajectories, and an average distance between trajectories.