System and method for modeling movement of objects using probabilistic graphs obtained from surveillance data

• US 8,149,278 B2
• Filed: 02/05/2007
• Issued: 04/03/2012
• Est. Priority Date: 11/30/2006
• Status: Expired due to Fees

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Wren, Christopher R., Ivanov, Yuri A., Sorokin, Alexander, Banga, Ishwinder Kaur
• Primary Examiner(s): Boutah, Alina N
• Assistant Examiner(s): NGUYEN, STEVEN C
• Application Number:
  US11/671,016
• Publication Number:
  US 20080130951A1
• Time in Patent Office: 1,884 Days
• Field of Search: 348/143, 348/154
• US Class Current: 348/143
• CPC Class Codes: G06F 18/29 Graphical models, e.g. Bayesian networks
• CPC Class Codes: G06V 10/84 using probabilistic graphical models from image or video features, e.g. Markov models or Bayesian networks
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A method and system tracks models and predict object movement in an environment. Sequences of temporally and spatially adjacent events sensed by the set of sensors are linked to form a set of tracklets. Each tracklet has an associated starting and terminating location. The tracklets are used to construct a directed graph including starting nodes, terminating nodes, and, intermediate nodes connected by edges. The intermediate nodes can be split nodes where tracklets diverge onto different tracks, and join nodes where multiple tracklets converge onto a single path. Probabilities are assigned to the edges to model and predict movement of the objects in the environment.

Claims

1. A computer implemented method for modeling movement of objects in an environment, comprising:

detecting discrete events caused by objects moving in an environment with a set of sensors arranged at known locations in the environment;

linking sequences of temporally and spatially adjacent discrete events to form a set of tracklets;

constructing a set of directed graphs from the set of tracklets, the graphs including at least one starting node, at least one terminating node, and one or more intermediate nodes where multiple tracklets connect giving rise to ambiguities in the movement of the objects, in which the starting node is A and a terminating node is Z and a graph γ

₀ models the movement of a particular object from node A to node Z, and a probability P(AZ|γ

₀) =1 if a particular object starting at node A always terminated at node Z in the graph γ

₀, where z,900 indicates follows; and

assigning probabilities to the nodes, the probabilities indicating likelihoods that the objects were at particular known locations to model the movement of the objects in the environment.

2. The method of claim 1, in which the sensors are binary motion sensors.

3. The method of claim 1, in which the sensors using wireless transmitters for transmitting the events.

4. The method of claim 1, in which the linking is performed according to temporal and spatial constraints.

5. The method of claim 4, in which the temporal and spatial constraints are learned over time.

6. The method of claim 1, in which a particular probability is associated with a particular object.

7. The method of claim 1., in which the intermediate nodes include split nodes where multiple tracklets diverge, and join nodes where multiple tracklets converge.

8. The method of claim 1, in which a starting node is A, a terminating node is Z, and Γ

is a set of graphs γ

_i, and

9. The method of claim 8, in which a probability:

10. The method of claim 9, in which a probability

11. The method of claim 10, in which the sets Φ

and Θ

can be discontinuous.

12. The method of claim 1, further comprising:

predicting the movement of the objects from the set of graphs with the assigned probabilities.

13. The method of claim 1, further comprising:

rank ordering the probabilities of the terminating nodes.

14. The method of claim 1, in which the probability P is 0 if a particular object is not at a particular known location, and 1 if the particular object is at the known location, and 0<

P<

1 otherwise.