Video surveillance system with trajectory hypothesis spawning and local pruning

• US 20050104960A1
• Filed: 08/12/2004
• Published: 05/19/2005
• Est. Priority Date: 11/17/2003
• Status: Abandoned Application

• Current Assignee: Vidient Systems, Inc. (Agilence Incorporated)
• Original Assignee: Vidient Systems, Inc. (Agilence Incorporated)
• Inventors: Gong, Yihong, Han, Mei, Tao, Hai
• Application Number:
  US10/917,009
• Publication Number:
  US 20050104960A1
• Time in Patent Office: Days
• US Class Current: 348/143
• CPC Class Codes: G06T 7/269 using gradient-based methods
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A video surveillance system uses rule-based reasoning and multiple-hypothesis scoring to detect predefined behaviors based on movement through zone patterns. Trajectory hypothesis spawning allows for trajectory splitting and/or merging and includes local pruning to managed hypothesis growth. Hypotheses are scored based on a number of criteria, illustratively including at least one non-spatial parameter. Connection probabilities computed during the hypothesis spawning process are based on a number of criteria, illustratively including object size. Object detection and probability scoring is illustratively based on object class.

Claims

1. A method for use in a video surveillance system, the method comprising

generating at least one hypothesis, said hypothesis comprising a set of hypothesized trajectories of objects hypothesized to have been moving through an area under surveillance at a previous time,

identifying objects present in the area under surveillance at a present time,

computing connection probabilities each associated with a respective object/trajectory pair, each object/trajectory pair comprising a particular one of said identified objects and a particular one of said hypothesized trajectories, each connection probability being indicative of a likelihood that said particular trajectory is the actual trajectory of said particular object, and

generating one or more extended hypotheses, each extended hypothesis comprising a respective different combination of at least one of

a) one or more of said trajectories extended to one or more of said identified objects,

b) one or more of said trajectories not extended to any of said identified objects,

c) one or more of said objects to which none of said trajectories is extended,

d) one or more of said objects regarded as having been identified in error, and

repeating said identifying and computing for at least one of said extended hypotheses to generate at least one further extended hypothesis, said further extended hypothesis not including

any trajectory defined in b) having at least one associated connection probability that meets a first predetermined criterion, nor

any object defined in c) or d) having at least one associated connection probability that meets said first predetermined criterion.

2. The method of claim 1 wherein said first predetermined criterion is that said associated connection probability is at least equal to a predetermined value.

3. The method of claim 1 wherein said further extended hypothesis does not include any trajectory/object pair defined in a) if its associated connection probability meets a second predetermined criterion.

4. The method of claim 3 wherein said second predetermined criterion is that said associated connection probability is less than a predetermined value.

5. A method for use in a video surveillance system, the method comprising

generating a plurality of first hypotheses, each said first hypothesis comprising a respective set of hypothesized trajectories of objects hypothesized to have been moving through an area under surveillance at a first time,

hypothesizing the presence of particular objects in the surveillance space at a subsequent time,

generating, for at least one of said first hypotheses, at least two extended hypotheses associated with that one first hypothesis, each extended hypothesis including said hypothesized trajectories, and each said extended hypothesis differing from each other extended hypothesis in at least one of a) which said trajectories are extended to which of said particular objects, b) the number of said trajectories that are extended to any of said particular objects, c) the number of said particular objects to which no trajectory is extended, and d) the number of said particular objects that are included in said each extended hypothesis, and

repeating said hypothesizing and generating using only selected ones of all possible extended hypotheses having different combinations of b), c) and d).

6. An electronic surveillance system adapted to

generate at least one hypothesis, said hypothesis comprising a set of hypothesized trajectories of objects hypothesized to have been moving through an area under surveillance at a previous time,

identify objects present in the area under surveillance at a present time,

compute connection probabilities each associated with a respective object/trajectory pair, each object/trajectory pair comprising a particular one of said identified objects and a particular one of said hypothesized trajectories, each connection probability being indicative of a likelihood that said particular trajectory is the actual trajectory of said particular object, and

generate one or more extended hypotheses, each extended hypothesis comprising a respective different combination of at least one of

a) one or more of said trajectories extended to one or more of said identified objects,

b) one or more of said trajectories not extended to any of said identified objects,

c) one or more of said objects to which none of said trajectories is extended,

d) one or more of said objects regarded as having been identified in error, and

repeat the identifying and computing for at least one of said extended hypotheses to generate at least one further extended hypothesis, said further extended hypothesis not including

any trajectory defined in b) having at least one associated connection probability that meets a first predetermined criterion, nor

any object defined in c) or d) having at least one associated connection probability that meets said first predetermined criterion.

7. The method of claim 6 wherein said first predetermined criterion is that said associated connection probability is at least equal to a predetermined value.

8. The method of claim 6 wherein said further extended hypothesis does not include any trajectory/object pair defined in a) if its associated connection probability meets a second predetermined criterion.

9. The method of claim 8 wherein said second predetermined criterion is that said associated connection probability is less than a predetermined value.

10. An electronic surveillance system adapted to

generate a plurality of first hypotheses, each said first hypothesis comprising a respective set of hypothesized trajectories of objects hypothesized to have been moving through an area under surveillance at a first time,

hypothesize the presence of particular objects in the surveillance space at a subsequent time,

generate, for at least one of said first hypotheses, at least two extended hypotheses associated with that one first hypothesis, each extended hypothesis including said hypothesized trajectories, and each said extended hypothesis differing from each other extended hypothesis in at least one of a) which said trajectories are extended to which of said particular objects, b) the number of said trajectories that are extended to any of said particular objects, c) the number of said particular objects to which no trajectory is extended, and d) the number of said particular objects that are included in said each extended hypothesis, and

repeat the hypothesizing and generating using only selected ones of all possible extended hypotheses having different combinations of b), c) and d).

11. A tangible medium on which are stored instructions that are executable by a processor to carry out a method in a video surveillance system, the method comprising

generating at least one hypothesis, said hypothesis comprising a set of hypothesized trajectories of objects hypothesized to have been moving through an area under surveillance at a previous time,

identifying objects present in the area under surveillance at a present time,

computing connection probabilities each associated with a respective object/trajectory pair, each object/trajectory pair comprising a particular one of said identified objects and a particular one of said hypothesized trajectories, each connection probability being indicative of a likelihood that said particular trajectory is the actual trajectory of said particular object, and

generating one or more extended hypotheses, each extended hypothesis comprising a respective different combination of at least one of

a) one or more of said trajectories extended to one or more of said identified objects,

b) one or more of said trajectories not extended to any of said identified objects,

c) one or more of said objects to which none of said trajectories is extended,

d) one or more of said objects regarded as having been identified in error, and

repeating said identifying and computing for at least one of said extended hypotheses to generate at least one further extended hypothesis, said further extended hypothesis not including

any trajectory defined in b) having at least one associated connection probability that meets a first predetermined criterion, nor

any object defined in c) or d) having at least one associated connection probability that meets said first predetermined criterion.

12. The tangible medium of claim 11 wherein said first predetermined criterion is that said associated connection probability is at least equal to a predetermined value.

13. The tangible medium of claim 11 wherein said further extended hypothesis does not include any trajectory/object pair defined in a) if its associated connection probability meets a second predetermined criterion.

14. The method of claim 13 wherein said second predetermined criterion is that said associated connection probability is less than a predetermined value.

15. A tangible medium on which are stored instructions that are executable by a processor to carry out a method in a video surveillance system, the method comprising

generating a plurality of first hypotheses, each said first hypothesis comprising a respective set of hypothesized trajectories of objects hypothesized to have been moving through an area under surveillance at a first time,

hypothesizing the presence of particular objects in the surveillance space at a subsequent time,

generating, for at least one of said first hypotheses, at least two extended hypotheses associated with that one first hypothesis, each extended hypothesis including said hypothesized trajectories, and each said extended hypothesis differing from each other extended hypothesis in at least one of a) which said trajectories are extended to which of said particular objects, b) the number of said trajectories that are extended to any of said particular objects, c) the number of said particular objects to which no trajectory is extended, and d) the number of said particular objects that are included in said each extended hypothesis, and

repeating said hypothesizing and generating using only selected ones of all possible extended hypotheses having different combinations of b), c) and d).