Video surveillance system with rule-based reasoning and multiple-hypothesis scoring

• US 7,136,507 B2
• Filed: 08/12/2004
• Issued: 11/14/2006
• Est. Priority Date: 11/17/2003
• Status: Active Grant

• Current Assignee: Axis AB (Canon Inc.)
• Original Assignee: Vidient Systems, Inc. (Agilence Incorporated)
• Inventors: Tao, Hai, Gong, Yihong, Han, Mei
• Primary Examiner(s): Johns, Andrew W.
• Assistant Examiner(s): EDWARDS, PATRICK L
• Application Number:
  US10/917,985
• Publication Number:
  US 20050104962A1
• Time in Patent Office: 824 Days
• Field of Search: 382/103, 340/541, 348/143, 348/169
• US Class Current: 382/103
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position

Abstract

A video surveillance system uses rule-based reasoning and multiple-hypothesis scoring to detect predefined behaviors based on movement through zone patterns. Trajectory hypothesis spawning allows for trajectory splitting and/or merging and includes local pruning to managed hypothesis growth. Hypotheses are scored based on a number of criteria, illustratively including at least one non-spatial parameter. Connection probabilities computed during the hypothesis spawning process are based on a number of criteria, illustratively including object size. Object detection and probability scoring is illustratively based on object class.

Claims

1. A method for use in a video surveillance system, the method comprising

generating a first hypothesis comprising a set of hypothesized trajectories of respective objects hypothesized to have been moving through an area under surveillance at a previous time,

generating a second hypothesis comprising a different set of hypothesized trajectories of respective ones of said objects hypothesized to have been moving through the area under surveillance at said previous time,

generating a first extended hypothesis comprising a set of hypothesized trajectories of respective objects hypothesized to have been moving through the area under surveillance at a present time, the trajectories of said first extended hypothesis comprising 1) at least ones of the trajectories of the first hypothesis, 2) one or more of the objects hypothesized to have been moving through the area under surveillance at said present time, and 3) a particular set of connections extending at least ones of the trajectories of said first hypothesis to at least ones of the objects hypothesized to have been moving through the area under surveillance at said present time,

generating a second extended hypothesis comprising a set of hypothesized trajectories of respective ones of said objects hypothesized to have been moving through the area under surveillance at said present time, the trajectories of said second extended hypothesis comprising 1) at least ones of the trajectories of the first hypothesis, 2) one or more of said objects hypothesized to have been moving through the area under surveillance at said present time, and 3) a particular set of connections extending at least ones of the trajectories of said first hypothesis to at least ones of the objects hypothesized to have been moving through the area under surveillance at said present time, the first and second extended hypotheses being different from one another,

generating a third extended hypothesis comprising a set of hypothesized trajectories of respective ones of said objects hypothesized to have been moving through the area under surveillance at said present time, the trajectories of said third extended hypothesis comprising 1) at least ones of the trajectories of the second hypothesis, 2) one or more of the objects hypothesized to have been moving through the area under surveillance at said present time, and 3) a particular set of connections extending at least ones of the trajectories of said second hypothesis to at least ones of the objects hypothesized to have been moving through the area under surveillance at said present time,

generating a fourth extended hypothesis comprising a set of hypothesized trajectories of respective ones of said objects hypothesized to have been moving through the area under surveillance at said present time, the trajectories of said fourth extended hypothesis comprising 1) at least ones of the trajectories of the second hypothesis, 2) one or more of the objects hypothesized to have been moving through the area under surveillance at said present time, and 3) a particular set of connections extending at least ones of the trajectories of said second hypothesis to at least ones of the objects hypothesized to have been moving through the area under surveillance at said present time, the third and fourth extended hypotheses being different from one another,

identifying at least one of said first, second, third and fourth extended hypotheses as being more likely than others of said first, second, third and fourth extended hypotheses to represent the actual trajectories of the actual objects moving through the area under surveillance at said present time, and

utilizing said at least said one of said first, second, third and fourth extended hypotheses to determine whether at least one predefined alert condition has occurred within the area under surveillance.

2. The method of claim 1 wherein said identifying includes computing a likelihood for each of said first, second, third and fourth extended hypotheses, the likelihoods for said first, second, third and fourth extended hypotheses each being a function of connection probabilities each associated with a respective trajectory of that particular hypothesis, each connection probability being a measure of the probability that an object that terminates said associated trajectory is, in fact, an object in the area under surveillance that is on that trajectory.

3. The method of claim 2 wherein said at least one of said first, second, third and fourth extended hypotheses is the one having the largest computed likelihood.

4. The method of claim 1 wherein said objects are people and wherein said alert condition includes a predetermined pattern of movement of one or more people relative to at least one of a) at least one other person, b) one or more fixed objects, or c) one or more predefined zones within said area under surveillance.

5. The method of claim 4 wherein said predetermined pattern of movement includes at least one time criterion.

6. An electronic surveillance system adapted to carry out the method defined by claim 1.

7. A computer readable medium on which are stored instructions that are executable by a processor to carry out the method defined by claim 1.