Method and system for tracking and behavioral monitoring of multiple objects moving through multiple fields-of-view

• US 20040130620A1
• Filed: 11/12/2003
• Published: 07/08/2004
• Est. Priority Date: 11/12/2002
• Status: Active Grant

• Current Assignee: Johnson Controls Tyco IP Holdings LLP (Johnson Controls International plc)
• Original Assignee: Sensormatic Electronics LLC (Johnson Controls International plc)
• Inventors: Sobalvarro, Patrick, Buehler, Christopher J., Brock, Neil, Edulbehram, Jehanbux
• Granted Patent: US 8,547,437 B2
• Time in Patent Office: Days
• US Class Current: 348/143
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19641 Multiple cameras having overlapping views on a single scene
• CPC Class Codes: G08B 13/19671 Addition of non-video data, i.e. metadata, to video stream
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A computerized method of video analysis that includes receiving several series of video frames generated by a number of image sensors. Each image sensor has its own field-of-view which can, but does not have to, overlap with another image sensor'"'"'s field-of-view. The image sensors monitor a portion of a monitored environment. The computerized method also includes concurrently tracking, independent of calibration, multiple objects within the monitored environment as the objects move between fields-of-view, and multiple objects within one field-of-view. The tracking is based on the plurality of received series of video frames

Claims

1. A computerized method of video analysis comprising:

receiving a plurality of series of video frames generated by a plurality of image sensors, each having a field-of-view, which monitor portions of a monitored environment; and

concurrently tracking, independent of calibration (i) a plurality of objects within the monitored environment as the objects move between fields-of-view, and ii) plurality of objects within one field-of-view based on the plurality of received series of video frames.

2. The method of claim 1 wherein the image sensors are cameras.

3. The method of claim 1 further comprising tracking objects based on a probability that an object included in one video frame generated by a first image sensor at a first point in time will be included in a video frame generated by a second image sensor a second point in time.

4. The method of claim 1 further comprising:

storing a plurality of blob states over time, each state including a number of objects included in the blob and a blob signature; and

storing a plurality of transition likelihood values representing the probability that objects within one blob at one instant in time correspond to objects within other blobs at other instants in time.

5. The method of claim 4 further comprising altering the stored transition probabilities upon analysis of additional video frames.

6. The method of claim 4 further comprising storing object data indicating correspondences between objects and blob states.

7. The method of claim 4 generating a tracking solution based on the blob states and transition probabilities.

8. The method of claim 1 generating tracking metadata including at least one of object track data, tracking solutions, object feature data and field-of-view data.

9. The method of claim 8 further comprising:

selecting a rule set to analyze generated tracking metadata; and

evaluating, using a rules engine, the tracking metadata based on the rule set.

10. The method of claim 9 further comprising selecting the rule set to monitor parking lot security.

11. The method of claim 9 further comprising selecting the rule set to detect property theft.

12. The method of claim 9 further comprising selecting the rule set to detect hazards to children.

13. The method of claim 9 further comprising selecting the rule set to monitor public safety.

14. The method of claim 9 further comprising selecting the rule set to determine merchandizing and operations statistics.

15. A computerized system for video analysis comprising:

a receiving module configured to receive a plurality of series of video frames, the series of video frames generated by a plurality of image sensors which monitor portions of a monitored environment and have a field-of-view; and

a calibration-independent tracking module in communication with the receiving module and configured to i) concurrently track a plurality of objects within the monitored environment as the objects move between fields-of-view and ii) concurrently track a plurality of objects within one field-of-view based on the plurality of received series of video frames, the tracking module outputting tracking metadata.

16. The system of claim 15 further comprising a rules engine in communication with the tracking module and receiving the tracking metadata.

17. A system for monitoring parking lot security comprising:

a receiving module configured to receive a plurality of series of video frames, the series of video frames generated by a plurality of image sensors which monitor portions of a monitored environment and have a field-of-view;

a calibration-independent tracking module in communication with the receiving module and configured to i) concurrently track a plurality of objects within the monitored environment as the objects move between fields-of-view and ii) concurrently track a plurality of objects within one field-of-view based on the plurality of received series of video frames, the tracking module outputting tracking metadata; and

a rules engine utilizing a parking lot security rule set configured to receive and evaluate the tracking metadata.

18. A system for property theft detection comprising:

a receiving module configured to receive a plurality of series of video frames, the series of video frames generated by a plurality of image sensors which monitor portions of a monitored environment and have a field-of-view;

a calibration-independent tracking module in communication with the receiving module and configured to i) concurrently track a plurality of objects within the monitored environment as the objects move between fields-of-view and ii) concurrently track a plurality of objects within one field-of-view based on the plurality of received series of video frames, the tracking module outputting tracking metadata; and

a rules engine utilizing a theft detection rule set configured to receive and evaluate the tracking metadata.

19. A system for child hazard detection comprising:

a receiving module configured to receive a plurality of series of video frames, the series of video frames generated by a plurality of image sensors which monitor portions of a monitored environment and have a field-of-view;

a calibration-independent tracking module in communication with the receiving module and configured to i) concurrently track a plurality of objects within the monitored environment as the objects move between fields-of-view and ii) concurrently track a plurality of objects within one field-of-view based on the plurality of received series of video frames, the tracking module outputting tracking metadata; and

a rules engine utilizing a child safety rule set configured to receive and evaluate the tracking metadata.

20. A system for property theft detection comprising:

a receiving module configured to receive a plurality of series of video frames, the series of video frames generated by a plurality of image sensors which monitor portions of a monitored environment and have a field-of-view;

a calibration-independent tracking module in communication with the receiving module and configured to i) concurrently track a plurality of objects within the monitored environment as the objects move between fields-of-view and ii) concurrently track a plurality of objects within one field-of-view based on the plurality of received series of video frames, the tracking module outputting tracking metadata; and

a rules engine utilizing a public safety monitoring rule set configured to receive and evaluate the tracking metadata.

21. A system for merchandizing and operations statistical analysis comprising:

a receiving module configured to receive a plurality of series of video frames, the series of video frames generated by a plurality of image sensors which monitor portions of a monitored environment and have a field-of-view;

a calibration-independent tracking module in communication with the receiving module and configured to i) concurrently track a plurality of objects within the monitored environment as the objects move between fields-of-view and ii) concurrently track a plurality of objects within one field-of-view based on the plurality of received series of video frames, the tracking module outputting tracking metadata; and

a rules engine utilizing a merchandizing and operations statistical rule set configured to receive and evaluate the tracking metadata.

22. A method of analyzing video data comprising:

receiving tracking metadata from a calibration-independent tracking module;

analyzing the metadata;

generating an event if a portion of the metadata exhibits a specified pattern; and

analyzing the metadata using a regular expression representation of the specified pattern.

23. The method of claim 22, further comprising:

comparing the regular expression of the specified pattern to the portion of the metadata by utilizing a software implemented representation of a finite state machine.

24. A system of video analysis comprising:

means for receiving tracking metadata from a calibration-independent tracking module;

means for analyzing the metadata;

means for generating an event if a portion of the metadata exhibits specified pattern; and

means for analyzing the metadata using a regular expression representation of the specified pattern.