Video surveillance system with rule-based reasoning and multiple-hypothesis scoring

• US 20050104962A1
• Filed: 08/12/2004
• Published: 05/19/2005
• Est. Priority Date: 11/17/2003
• Status: Active Grant

• Current Assignee: Axis AB (Canon Inc.)
• Original Assignee: Vidient Systems, Inc. (Agilence Incorporated)
• Inventors: Gong, Yihong, Han, Mei, Tao, Hai
• Granted Patent: US 7,136,507 B2
• Time in Patent Office: Days
• US Class Current: 348/143
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position

Abstract

A video surveillance system uses rule-based reasoning and multiple-hypothesis scoring to detect predefined behaviors based on movement through zone patterns. Trajectory hypothesis spawning allows for trajectory splitting and/or merging and includes local pruning to managed hypothesis growth. Hypotheses are scored based on a number of criteria, illustratively including at least one non-spatial parameter. Connection probabilities computed during the hypothesis spawning process are based on a number of criteria, illustratively including object size. Object detection and probability scoring is illustratively based on object class.

Claims

1. A method for use in a video surveillance system, the method comprising

generating a plurality of hypotheses, each hypothesis comprising a respective different set of hypothesized trajectories of objects hypothesized to have been moving through an area under surveillance at a particular time,

identifying at least one of said hypotheses as being more likely than others of said hypotheses to represent the actual trajectories of the actual objects moving through the area under surveillance at said particular time, and

utilizing said at least said one of said hypotheses to determine whether at least one predefined alert condition has occurred within the area under surveillance.

2. The method of claim 1 wherein said identifying includes computing a likelihood for each of said hypotheses, said likelihood being a function of connection probabilities each associated with a respective trajectory of that hypothesis, each connection probability being a measure of the probability that an object that terminates said associated trajectory is, in fact, an object in the area under surveillance that is on that trajectory.

3. The method of claim 2 wherein said at least one of said hypotheses is the one having the largest computed likelihood.

4. The method of claim 1 wherein each of said hypotheses is generated by

detecting particular objects in the area under surveillance at said particular time, and

extending a previously generated hypothesis that comprises a particular set of hypothesized trajectories of objects hypothesized to have been moving through the area under surveillance at a previous time,

said extended hypothesis comprising 1) at least ones of the trajectories of the previously generated hypothesis, 2) one or more of the detected objects, and 3) a particular set of connections extending at least ones of the trajectories of the previously generated hypothesis to at least ones of the detected objects.

5. The method of claim 1 wherein said objects are people and wherein said alert condition includes a predetermined pattern of movement of one or more people relative to at least one of a) at least one other person, b) one or more fixed objects, or c) one or more predefined zones within said area under surveillance.

6. The method of claim 5 wherein said predetermined pattern of movement includes at least one time criterion.

16. An electronic surveillance system adapted to carry out the method defined by claim 1.

18. A tangible medium on which are stored instructions that are executable by a processor to carry out the method defined by claim 1.

7. A method for use in an electronic surveillance system, the method comprising

establishing a plurality of hypotheses associated with a particular point in time, each hypothesis comprising a set of objects hypothesized to be in an area under surveillance by said system at said particular point in time and further comprising hypothesized prior trajectories through said area under surveillance for at least ones of the hypothesized objects,

detecting objects present in the area under surveillance at a subsequent point in time,

establishing a plurality of extended hypotheses associated with said subsequent point in time, each extended hypothesis being associated with one of the previously established hypotheses and comprising 1) at least ones of the trajectories of the associated previously established hypothesis, 2) one or more of the detected objects, and 3) a particular set of connections extending at least ones of the trajectories to said one or more of the detected objects, each extended hypothesis thereby comprising a set of objects hypothesized to be in the area under surveillance at the subsequent point in time and further comprising hypothesized prior trajectories through said area under surveillance for at least ones of said set of hypothesized objects,

repeating said detecting and said establishing extended hypotheses multiple times,

identifying at least one of the hypotheses associated with said subsequent point in time as being more likely than others of the hypotheses associated with that point in time as comprising the actual trajectories of the actual objects in the area under surveillance at that point in time, and

utilizing the identified hypothesis to determine whether at least one predefined alert condition has occurred within the area under surveillance.

8. The method of claim 7 wherein said identifying includes computing a likelihood for each of said hypotheses, said likelihood being a function of connection probabilities each associated with a respective trajectory of that hypothesis, each connection probability being a measure of the probability that an object that terminates said associated trajectory is, in fact, an object in the area under surveillance that is on that trajectory.

9. The method of claim 8 wherein said identified hypothesis is the one having the largest computed likelihood.

10. The method of claim 7 wherein said objects are people and wherein said alert condition includes a predetermined pattern of movement of at least one person relative to a fixed object in said area under surveillance.

11. The method of claim 7 wherein said objects are people and wherein said alert condition includes a predetermined pattern of movement that includes at least one time criterion.

12. The method of claim 7 wherein said objects are people and wherein said alert condition includes a predetermined pattern of movement between two or more people.

13. The method of claim 12 wherein said alert condition further includes a predetermined pattern of movement between at least one of said people and a fixed object in said area under surveillance.

14. The method of claim 7 wherein said objects are people and wherein said alert condition includes a predetermined pattern of movement of one or more people relative to at least one of a) at least one other person, b) one or more fixed objects, or c) one or more predefined zones within said area under surveillance.

15. The method of claim 14 wherein said predetermined pattern includes at least one time criterion.

17. An electronic surveillance system adapted to carry out the method defined by claim 7.

19. A tangible medium on which are stored instructions that are executable by a processor to carry out the method defined by claim 7.