Video surveillance system with object detection and probability scoring based on object class

• US 20050105765A1
• Filed: 08/12/2004
• Published: 05/19/2005
• Est. Priority Date: 11/17/2003
• Status: Active Grant

• Current Assignee: Axis AB (Canon Inc.)
• Original Assignee: Vidient Systems, Inc. (Agilence Incorporated)
• Inventors: Gong, Yihong, Han, Mei, Tao, Hai
• Granted Patent: US 7,127,083 B2
• Time in Patent Office: Days
• US Class Current: 382/100
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19604 involving reference image or background adaptation with time to compensate for changing conditions, e.g. reference image update on detection of light level change
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19615 wherein said pattern is defined by the user
• CPC Class Codes: G08B 13/19652 Systems using zones in a single scene defined for different treatment, e.g. outer zone gives pre-alarm, inner zone gives alarm
• CPC Class Codes: G08B 31/00 Predictive alarm systems characterised by extrapolation or other computation using updated historic data
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

A video surveillance system uses rule-based reasoning and multiple-hypothesis scoring to detect predefined behaviors based on movement through zone patterns. Trajectory hypothesis spawning allows for trajectory splitting and/or merging and includes local pruning to managed hypothesis growth. Hypotheses are scored based on a number of criteria, illustratively including at least one non-spatial parameter. Connection probabilities computed during the hypothesis spawning process are based on a number of criteria, illustratively including object size. Object detection and probability scoring is illustratively based on object class.

Claims

1. A method for use in a video surveillance system in which at least a first set of trajectories of a first set of objects of a particular object class is hypothesized to have been moving through an area under surveillance at a previous point in time, the method comprising

identifying objects of said particular class hypothesized to be in said area under surveillance at a current point in time, said particular class of objects being distinguishable from other objects based said particular class of objects'"'"' physical appearance, at least one of said objects in said area under surveillance at said current point in time being identified independent of the physical appearance of any objects hypothesized to have been in said area under surveillance at said previous point in time, and

extending at least ones of said first set of trajectories to at least ones of said identified objects to develop at least one set of extended trajectories, each of at least ones of said extended trajectories being a respective one of said first set of trajectories extended to at least of the identified objects.

2. The method of claim 1 wherein said identifying includes analyzing individual portions of a video image of said area under surveillance to determine if said portions have features that are characteristic of objects in said particular class.

3. The method of claim 2 further comprising selecting, as an individual one of said portions, a portion of said video image based on its proximity to a terminating point of at least one of the trajectories of said first set of trajectories.

4. The method of claim 2 wherein said analyzing includes applying said individual portions of said video image to a neural network that has been trained to recognize objects in said particular object class.

5. The method of claim 4 wherein said neural network generates a score in response to each said applied portion of said video image and wherein said each portion is identified as being an object in said particular object class if said score is as least as large as a predetermined threshold.

6. The method of claim 2 further comprising selecting, as an individual one of said portions, a portion of the foreground of said video image at said current point in time.

7. The method of claim 6 wherein said foreground of said video image comprises portions of said image at said current point in time whose content is substantially different from the content of the spatially corresponding portion of said video image at a previous point in time.

8. The method of claim 6 further comprising selecting, as an individual one of said portions, a portion of said video image based on its proximity to a terminating point of at least one of the trajectories of said first set of trajectories, said selecting being independent of whether said individual one of said portions is in said foreground.

9. The method of claim 7 further comprising selecting, as an individual one of said portions, a portion of said video image based on its proximity to a terminating point of at least one of the trajectories of said first set of trajectories, said selecting being independent of whether said individual one of said portions is in said foreground.

14. The method of claims 9 wherein said analyzing includes applying data representing areas to be analyzed to a neural network that has been trained to recognize an image as being that of a person.

18. An electronic surveillance system adapted to carry out the method defined by claim 1.

19. A tangible medium on which are stored instructions that are executable by a processor to carry out the method defined by claim 1.

10. A method of analyzing a video image, the method comprising

identifying particular areas of said video image as being more likely than other areas to contain the image of a person, said identifying being carried out independent of any prior analysis of said video image,

analyzing said particular areas to determine if individual ones of them appear to represent people, and

extending at least one previously hypothesized trajectory of at least one person to include the location of at least one of said particular areas of movement that appears to represent a person.

11. The method of claim 10 wherein at least one of said particular areas of said video image is an area representing a moving object.

12. The method of claim 10 wherein at least one of said particular areas of said video image is an area selected based on its relative proximity to a terminating end of one of said previously hypothesized trajectories.

13. The method of claim 12 wherein at least one of said particular areas of said video image is an area representing a moving object.

15. The method of claims 10 wherein said analyzing includes applying data representing areas to be analyzed to a neural network that has been trained to recognize an image as being that of a person.

16. The method of claim 11 wherein said analyzing includes applying data representing areas to be analyzed to a neural network that has been trained to recognize an image as being that of a person.

17. The method of claim 12 wherein said analyzing includes applying data representing areas to be analyzed to a neural network that has been trained to recognize an image as being that of a person.

20. An electronic surveillance system adapted to carry out the method defined by claim 10.

21. A tangible medium on which are stored instructions that are executable by a processor to carry out the method defined by claim 10.