Stereo camera intrusion detection system

• US 8,432,448 B2
• Filed: 08/10/2006
• Issued: 04/30/2013
• Est. Priority Date: 08/10/2006
• Status: Active Grant

• Current Assignee: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Sponsoring Entity: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Inventors: Hassapis, Chris, Nishihara, H. Keith
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): Aynalem, Nathnael
• APPLICATION: US 20080043106A1
• Time in USPTO: 2455 Days
• Field of Search: 348/143, 348152-155, 348/159, 348/169, 348/207.99, 348/286, 348/487, 348/135, 348/148, 348/149, 348/47, 382/103, 382/104, 382/107, 382/291, 342/104
• US Class Current: 348/153
• CPC Class Codes: G06K 9/00771 : Recognising scenes under surveillance, e.g. with Markovian modelling of scene activity G06K9/00785 takes precedence; recognition of movements or behaviour G06K9/00335; motion analysis using general image processing G06T7/20; intruder alarms using image scanning and comparing systems G08B13/194; circuitry for movement detection and estimation for pictorial communication H04N5/144; closed circuit television systems H04N7/18
• CPC Class Codes: G06T 7/285 : using a sequence of stereo image pairs
• CPC Class Codes: G08B 13/19604 : involving reference image or background adaptation with time to compensate for changing conditions, e.g. reference image update on detection of light level change
• CPC Class Codes: G08B 13/19606 : Discriminating between target movement or movement in an area of interest and other non-signicative movements, e.g. target movements induced by camera shake or movements of pets, falling leaves, rotating fan
• CPC Class Codes: G08B 13/19608 : Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position analysis of motion by image processing per se G06T7/20
• CPC Class Codes: G08B 13/1961 : Movement detection not involving frame subtraction, e.g. motion detection on the basis of luminance changes in the image
• CPC Class Codes: G08B 13/19641 : Multiple cameras having overlapping views on a single scene
• CPC Class Codes: H04N 5/144 : Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20

Abstract

A system and method is provided for an intrusion detection system. The intrusion detection system comprises a first camera configured to acquire first visual images of a monitored area and a second camera configured to acquire second visual images of the monitored area. The intrusion detection system also comprises a detection device configured to compare the first images with a background image of the monitored area. The detection device can mark differences between the first images and the background image as a potential intruder. The intrusion detection system further comprises a tracking device configured to evaluate each of the first images relative to each of the second images to determine three-dimensional characteristics associated with the potential intruder.

Claims

1. An intrusion detection system comprising:

a first camera configured to generate first images of a monitored area;

a second camera configured to generate second images of the monitored area;

a detection device configured to compare the first images with a background image of the monitored area, the detection device marking differences between the first images and the background image as a potential intruder; and

a tracking device configured to evaluate each of the first images relative to each of the second images to determine three-dimensional characteristics associated with the potential intruder, the tracking device comprising a threshold comparator configured to allow a user of the intrusion detection system to designate at least one three-dimensional space within the monitored area, the at least one three-dimensional space being one of a threshold space and a null zone, the tracking device activating the indicator in response to one of motion into and motion toward the threshold space, and the tracking device not activating the indicator in response to differences between the first images and the background image in the null zone.

2. The system of claim 1, wherein the three-dimensional characteristics comprise location, size, and motion, and further comprising an indicator, such that the tracking device is further configured to activate the indicator upon at least one of the location, the size, and the motion associated with the potential intruder exceeding a respective threshold.

3. The system of claim 1, wherein the first camera and the second camera are each mounted stationary at approximately ground level.

4. The system of claim 1, wherein the background image is a first background image generated from the first camera, and wherein the detection device is further configured to compare the second images with a second background image of the monitored area, the second background image being generated from the second camera, the detection device being further configured to mark differences between the second images and the second background image as the potential intruder.

5. The system of claim 4, wherein the first background image, the second background image, the first images, and the second images are filtered to display texture contrast patterns, such that the detection device is configured to compare the first filtered images with the first filtered background image and to compare the second filtered images with the second filtered background image, the detection device marking differences between the first filtered images and the first filtered background image and differences between the second filtered images and the second filtered background image as the potential intruder.

6. The system of claim 5, wherein the first background image is updated based on the differences between the first filtered images and the first filtered background image and the second background image is updated based on the differences between the second filtered images and the second filtered background image, and wherein the detection device is further configured to mark differences between the first images and the updated first background image as the potential intruder and the second images and the updated second background image as the potential intruder.

7. The system of claim 1, further comprising a background image generator configured to generate the background image of the monitored area, wherein the background image generator comprises a background stabilizer configured to correlate pixels associated with the first images with pixels associated with the background image, such that the pixels associated with the first images are vertically and horizontally aligned with the pixels associated with the background image prior to the detection device marking differences between the pixels associated with the first images and the pixels associated with the background image as the potential intruder.

8. The system of claim 1, wherein the tracking device comprises an image correlator configured to compare each of the first images to a respective one of the second images to determine a location and a size of the potential intruder in three-dimensional space based on a parallax separation of a two-dimensional location of the potential intruder in a given one of the first images relative to a two-dimensional location of the potential intruder in the respective one of the second images.

9. The system of claim 8, wherein the image correlator comprises:

an image overlay device configured to overlay the given one of the first images on the respective one of the second images, such that the two-dimensional location of the potential intruder on the given one of the first images intersects an approximate location of the two-dimensional location of the potential intruder on the respective one of the second images;

an iterative pixel shift device configured to repeatedly shift the given one of the first images relative to the respective one of the second images by a single pixel per shift; and

a correlation calculator configured to determine a correlation score for each shift, such that the parallax separation of the respective two-dimensional locations of the potential intruder is determined based on a given shift that produces a highest correlation score.

10. The system of claim 9, wherein the tracking device comprises a plurality of image filters configured to generate a plurality of first filtered images associated with each of the first images and a plurality of second filtered images associated with each of the second images, each of the plurality of first filtered images having a resolution that is matched to a respective one of the plurality of second filtered images, such that the image overlay device and the iterative pixel shift device are configured to overlay and repeatedly shift a plurality of matched pairs of the plurality of first filtered images and the plurality of second filtered images.

27. The system of claim 1, wherein the detection device is also configured to mark both absolute value pixel differences and filtered texture differences between the first images and the background image as the potential intruder.

28. The system of claim 1, wherein the detection device is also configured to implement a pixel filling algorithm to connect a plurality of pixel differences that are spaced apart by a predefined pixel distance on a difference image associated with one of the first images relative to the background image to generate a potential intruder token corresponding to the potential intruder.

29. The system of claim 1, wherein the detection device is also configured to implement a plurality of filtering algorithms to generate a respective plurality of filtered first images corresponding to each of the first images and a plurality of filtered background images, each of the plurality of filtered first images having a different resolution, the detection device being further configured to compare each of the plurality of filtered first images with a corresponding one of the plurality of filtered background images having a same resolution to mark the differences between the first images and the background image as the potential intruder.

11. A method for detecting intruders in a monitored area, the method comprising:

acquiring first images of the monitored area from a first camera and second images of the monitored area from a second camera;

generating a first background image of the monitored area associated with the first camera and a second background image of the monitored area associated with the second camera;

correlating first pixels associated with the first images with second pixels associated with the first background image of the monitored area, such that the first pixels are horizontally and vertically aligned with the second pixels;

correlating third pixels associated with the second images with fourth pixels associated with the second background image of the monitored area, such that the third pixels are horizontally and vertically aligned with the fourth pixels;

comparing the first images with the first background image of the monitored area and the second images with the second background image to determine the presence of a potential intruder;

determining three-dimensional characteristics of the potential intruder based on a relative comparison of the first images and the second images; and

activating an indicator upon the three-dimensional characteristics of the potential intruder exceeding at least one predetermined threshold.

12. The method of claim 11, wherein determining the three-dimensional characteristics of the potential intruder comprises determining a parallax separation of the potential intruder in each of the first images relative to the respective second images.

13. The method of claim 12, wherein determining the parallax separation comprises overlaying each of the first images onto a respective one of the second images at respective approximate two-dimensional locations of the potential intruder and shifting each of the first images relative to the respective one of the second images until a substantial correlation is found.

14. The method of claim 12, further comprising generating at least one first filtered image associated with each of the first images and at least one second filtered image associated with each of the second images, each of the at least one first filtered image forming a matched pair with a respective one of the at least one second filtered image having a substantially equal resolution.

15. The method of claim 14, wherein determining the three-dimensional characteristics of the potential intruder comprises equally correlating each of the at least one matched pair to determine a parallax separation of the potential intruder.

16. The method of claim 15, wherein the equally correlating comprises:

equally pixel shifting each of at least one first filtered image relative to the respective matched one of the at least one second filtered image;

generating an aggregate correlation score at each pixel shift, the aggregate correlation score comprising a correlation score of each of the at least one matched pair; and

comparing the aggregate correlation score at each pixel shift to determine the pixel shift having the highest aggregate correlation score.

17. The method of claim 11, further comprising filtering the first images, the second images, the first background image, and the second background image, wherein comparing the first images and the first background image comprises comparing the first filtered images and the first filtered background image, and wherein comparing the second images and the second background image comprises comparing the second filtered images and the second filtered background image.

18. The method of claim 17, further comprising:

updating the first background image based on the comparison between the first filtered images and the first filtered background image;

updating the second background image based on the comparison between the second filtered images and the second filtered background image;

comparing the first images with the updated first background image to determine the presence of the potential intruder; and

comparing the second images with the updated second background image to determine the presence of the potential intruder.

19. The method of claim 11, wherein determining the three-dimensional characteristics comprises determining a location, size, and motion associated with the potential intruder based on differences between the first images and the background image.

20. The method of claim 11, further comprising designating at least one three-dimensional space within the monitored area, such that the at least one three-dimensional space is one of a threshold space associated with activating the indicator and a null zone associated with activating the indicator.

30. The method of claim 11, wherein comparing the first images with the first background image and the second images with the second background image comprises marking both absolute value pixel differences and filtered texture differences between the first images and the first background image and between the second images and the second background image as the potential intruder.

31. The method of claim 11, further comprising implementing a pixel filling algorithm to connect a plurality of pixel differences that are spaced apart by a predefined pixel distance on a first difference image associated with one of the first images relative to the background image and a second difference image associated with one of the second images relative to the background image to generate a potential intruder token corresponding to the potential intruder on the first and second difference images.

21. An intrusion detection system comprising:

means for simultaneously acquiring first images and second images of a monitored area;

means for continuously generating a background image of the monitored area;

means for generating a plurality of filtered first images corresponding to each of the first images and a plurality of filtered background images, each of the plurality of filtered first images having a different resolution;

means for detecting a potential intruder based on differences between each of the plurality of filtered first images and a corresponding one of the plurality of filtered background images having a same resolution;

means for determining three-dimensional characteristics of the potential intruder based on the first images and the second images; and

means for activating an indicator based on the three-dimensional characteristics of the potential intruder.

22. The system of claim 21, wherein the three-dimensional characteristics comprise location, size, and motion.

23. The system of claim 21, wherein the means for determining the three-dimensional characteristics of the potential intruder comprises means for determining a parallax separation of a two-dimensional location of the potential intruder in a given one of the first images relative to a two-dimensional location of the potential intruder in the respective one of the second images.

24. The system of claim 23, wherein the means for determining a parallax separation comprises means for generating a plurality of first filtered images based on the first images and a plurality of second filtered images based on the second images, each of the plurality of first filtered images forming a matched pair with a respective one of the plurality of second filtered images having a substantially equal resolution.

25. The system of claim 24, wherein the means for determining a parallax separation further comprises means for pixel shifting each of the plurality of first filtered images relative to the respective matched one of the plurality of second filtered images and means for generating an aggregate correlation score for each pixel shift based on a correlation score associated with each of the plurality of matched pairs, the parallax separation being associated with the pixel shift having a highest aggregate correlation score.

26. The system of claim 21, further comprising means for horizontally and vertically aligning pixels associated with the plurality of filtered first images and the respective plurality of filtered background images prior to the operation of the means for detecting the potential intruder.

32. The system of claim 21, further comprising means for connecting a plurality of pixel differences that are spaced apart by a predefined pixel distance on a first difference image associated with one of the first images relative to the background image and a second difference image associated with one of the second images relative to the background image to generate a potential intruder token corresponding to the potential intruder on the first and second difference images.