Video surveillance system that detects predefined behaviors based on predetermined patterns of movement through zones

• US 7,088,846 B2
• Filed: 08/12/2004
• Issued: 08/08/2006
• Est. Priority Date: 11/17/2003
• Status: Active Grant

• Current Assignee: Axis AB (Canon Inc.)
• Original Assignee: Vidient Systems, Inc. (Agilence Incorporated)
• Inventors: Tao, Hai, Gong, Yihong, Han, Mei
• Primary Examiner(s): Couso, Jose L.
• Application Number:
  US10/916,870
• Publication Number:
  US 20050104727A1
• Time in Patent Office: 726 Days
• Field of Search: 382/103, 348152-156, 348/159, 348/169, 348/211.8, 340/541, 340/825.37
• US Class Current: 382/103
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G08B 13/19604 involving reference image or background adaptation with time to compensate for changing conditions, e.g. reference image update on detection of light level change
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19615 wherein said pattern is defined by the user
• CPC Class Codes: G08B 13/19652 Systems using zones in a single scene defined for different treatment, e.g. outer zone gives pre-alarm, inner zone gives alarm

Abstract

A video surveillance system uses rule-based reasoning and multiple-hypothesis scoring to detect predefined behaviors based on movement through zone patterns. Trajectory hypothesis spawning allows for trajectory splitting and/or merging and includes local pruning to managed hypothesis growth. Hypotheses are scored based on a number of criteria, illustratively including at least one non-spatial parameter. Connection probabilities computed during the hypothesis spawning process are based on a number of criteria, illustratively including object size. Object detection and probability scoring is illustratively based on object class.

Claims

1. A method for use in a video surveillance system, the method comprising

generating at least one hypothesis comprising a set of hypothesized trajectories of objects identified, based on their appearance, as being in a particular object class and hypothesized to have been moving through an area under surveillance at a particular time, said system capturing an image of said area under surveillance, said image of said area under surveillance including at least two zones within said area under surveillance,

utilizing said hypothesis to analyze said area under surveillance to determine whether at least one predefined alert condition has occurred within the area under surveillance, said at least one alert condition including

a predetermined pattern of movement of at least a first one of said objects between at least two of said zones, said predetermined pattern of movement including that a) said first object was in a first one of said zones within said area being analyzed, and b) said first object thereafter moved from said first one of said zones within said area being analyzed to a second one of said zones within said area being analyzed, andat least one time criterion defining a time relationship between the presence of a) said first object in at least one of said first and second zones within said area being analyzed and b) at least a second one of said objects in that same one of said first and second zones.

2. The method of claim 1 wherein said area under surveillance includes a door, wherein at least one of said zones is a door zone that includes said door, and wherein said predetermined pattern of movement includes movement of said first object a) from said door zone to another one of said zones, or b) to said door zone from another one of said zones.

3. The method of claim 2 wherein the determination of whether said alert condition has occurred is made independent of any non-video signal indicating the open or closed status of said door.

4. The method of claim 2 wherein said area under surveillance includes an access control device, wherein at least one of said zones is an access entry zone that includes said access control device, and wherein said predetermined pattern of movement includes movement of said first object a) from said access entry zone to another one of said zones, or b) to said access entry zone from another one of said zones,

said access control device being a card reader mounted near said door and operative to cause said door to become unlocked.

5. The method of claim 4 wherein the determination of whether said alert condition has occurred is made independent of any non-video signal indicating that said access control device has been used.

6. The method of claim 4 wherein said area under surveillance further includes a normally locked door, wherein said access control device is operative to cause said door to become unlocked, wherein at least one of said zones is a door zone that includes said door, and wherein said predetermined pattern of movement further includes movement of said first object a) from said door zone to another one of said zones, or b) to said door zone from another one of said zones.

7. The method of claim 1 wherein said area under surveillance is a non-secure area from which objects enter into a secure area.

8. The method of claim 7 wherein said area under surveillance includes a normally locked door separating said secure area from said non-secure area, wherein at least one of said zones is a door zone that includes said door, and wherein said predetermined pattern of movement includes movement of said first object a) from said door zone to another one of said zones, or b) to said door zone from another one of said zones.

9. The method of claim 8 wherein the determination of whether said alert condition has occurred is made independent of any non-video signal indicating the open or closed status of said door.

10. The method of claim 9 wherein said non-secure area includes a card reader mounted near said door and operative to cause said door to become unlocked.

11. The method of claim 10 wherein said reader is located in the area under surveillance, wherein at least one of said zones is an access entry zone that includes said card reader, and wherein said predetermined pattern of movement includes movement of said first object a) from said access entry zone to another one of said zones, or b) to said access entry zone from another one of said zones.

12. The method of claim 11 wherein the determination of whether said alert condition has occurred is made independent of any non-video signal indicating that said access control device has been used.

13. The invention of claim 1 wherein said alert condition further includes a predetermined pattern of movement of said second object within the area under surveillance, said predetermined pattern of movement of said second object including that a) said second object was in a first one of said zones within said area being analyzed, and b) said second object thereafter moved from said first one of said zones within said area under surveillance to a second one of said zones within said area being analyzed.

16. Au electronic surveillance system adapted to carry out the method defined by claim 1.

17. A computer-readable medium on which are stored instructions that are executable by a processor to carry out the method defined by claim 1.

20. The invention of claim 1 wherein said at least one time criterion is that, within a predetermined period of time, said first object and said second object are in said same one of said first and second zones.

21. The invention of claim 1 wherein said same one of said first and second zones within said area under surveillance is said second zone.

22. The invention of claim 1 wherein said alert condition further includes at least one other time criterion, said at least one other time criterion being that said first object is within said second zone for at least a minimum period of time.

23. The invention of claim 2 wherein said predetermined pattern of movement is that within a predetermined period of time, said first and second objects enter said door zone.

24. The invention of claim 2 wherein said predetermined pattern of movement is that within a predetermined period of time, said first object enters said door zone and said second object leaves said door zone.

25. The invention of claim 11 wherein said predetermined pattern of movement is that within a predetermined period of time, said first object enters said door zone from one side of said door and said second object enters said access entry zone from another side of said door.

14. A method for use in a video surveillance system, the method comprising

generating at least one hypothesis comprising a set of hypothesized trajectories of objects identified, based on their appearance, as being in a particular object class and hypothesized to have been moving through an area under surveillance at a particular time, said area under surveillance being divided into at least two zones,

utilizing said at least one of said hypotheses to determine whether at least one predefined alert condition has occurred within the area under surveillance, said at least one alert condition including

a predetermined pattern of movement of at least one of said objects relative to at least one of said zones, said predetermined pattern of movement including that a) said object was in a first one of said zones within said area under surveillance, and b) said object thereafter moved from said first one of said zones within said area under surveillance to a second one of said zones within said area under surveillance, andat least one time criterion, said at least one time criterion being that said object was in said second zone within said area under surveillance for more than a predetermined period of time.

15. The method of claim 14 wherein said area under surveillance includes a door, wherein said second zone is a door zone that includes said door, and wherein said alert condition further includes that said object did not go through said door.

18. An electronic surveillance system adapted to carry out the method defined by claim 14.

19. A computer-readable medium on which are stored instructions that are executable by a processor to carry out the method defined by claim 14.