Stereo camera intrusion detection system

• US 20080043106A1
• Filed: 08/10/2006
• Published: 02/21/2008
• Est. Priority Date: 08/10/2006
• Status: Active Grant

• Current Assignee: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Original Assignee: Northrop Grumman Corporation
• Inventors: Nishihara, H. Keith, Hassapis, Chris
• Granted Patent: US 8,432,448 B2
• Time in Patent Office: Days
• US Class Current: 348/153
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19604 involving reference image or background adaptation with time to compensate for changing conditions, e.g. reference image update on detection of light level change
• CPC Class Codes: G08B 13/19606 Discriminating between target movement or movement in an area of interest and other non-signicative movements, e.g. target movements induced by camera shake or movements of pets, falling leaves, rotating fan
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/1961 Movement detection not involving frame subtraction, e.g. motion detection on the basis of luminance changes in the image
• CPC Class Codes: G08B 13/19641 Multiple cameras having overlapping views on a single scene
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20

Abstract

A system and method is provided for an intrusion detection system. The intrusion detection system comprises a first camera configured to acquire first visual images of a monitored area and a second camera configured to acquire second visual images of the monitored area. The intrusion detection system also comprises a detection device configured to compare the first images with a background image of the monitored area. The detection device can mark differences between the first images and the background image as a potential intruder. The intrusion detection system further comprises a tracking device configured to evaluate each of the first images relative to each of the second images to determine three-dimensional characteristics associated with the potential intruder.

Claims

1. An intrusion detection system comprising:

a first camera configured to generate first images of a monitored area;

a second camera configured to generate second images of the monitored area;

a detection device configured to compare the first images with a background image of the monitored area, the detection device marking differences between the first images and the background image as a potential intruder; and

a tracking device configured to evaluate each of the first images relative to each of the second images to determine three-dimensional characteristics associated with the potential intruder.

2. The system of claim 1, wherein the three-dimensional characteristics comprise location, size, and motion, and further comprising an indicator, such that the tracking device is further configured to activate the indicator upon at least one of the location, the size, and the motion associated with the potential intruder exceeding a respective threshold.

3. The system of claim 1, wherein the tracking device comprises a threshold comparator configured to allow a user of the intrusion detection system to designate at least one three-dimensional space in the monitored area, the at least one three-dimensional space being one of a threshold space, such that one of motion into and motion toward the at least one three-dimensional space activates the indicator, and a null zone, such that differences between the first images and the background image in the three-dimensional space do not activate the indicator.

4. The system of claim 1, wherein the first camera and the second camera are each mounted stationary at approximately ground level.

5. The system of claim 1, wherein the background image is a first background image generated from the first camera, and wherein the detection device is further configured to compare the second images with a second background image of the monitored area, the second background image being generated from the second camera, the detection device being further configured to mark differences between the second images and the second background image as the potential intruder.

6. The system of claim 5, wherein the first background image, the second background image, the first images, and the second images are filtered to display texture contrast patterns, such that the detection device is configured to compare the first filtered images with the first filtered background image and to compare the second filtered images with the second filtered background image, the detection device marking differences between the first filtered images and the first filtered background image and differences between the second filtered images and the second filtered background image as the potential intruder.

7. The system of claim 6, wherein the first background image is updated based on the differences between the first filtered images and the first filtered background image and the second background image is updated based on the differences between the second filtered images and the second filtered background image, and wherein the detection device is further configured to mark differences between the first images and the updated first background image as the potential intruder and the second images and the updated second background image as the potential intruder.

8. The system of claim 1, further comprising a background image generator configured to generate the background image of the monitored area, wherein the background image generator comprises a background stabilizer configured to correlate pixels associated with the first images with pixels associated with the background image, such that the pixels associated with the first images are vertically and horizontally aligned with the pixels associated with the background image prior to the detection device marking differences between the pixels associated with the first images and the pixels associated with the background image as the potential intruder.

9. The system of claim 1, wherein the tracking device comprises an image correlator configured to compare each of the first images to a respective one of the second images to determine a location and a size of the potential intruder in three-dimensional space based on a parallax separation of a two-dimensional location of the potential intruder in a given one of the first images relative to a two-dimensional location of the potential intruder in the respective one of the second images.

10. The system of claim 9, wherein the image correlator comprises:

an image overlay device configured to overlay the given one of the first images on the respective one of the second images, such that the two-dimensional location of the potential intruder on the given one of the first images intersects an approximate location of the two-dimensional location of the potential intruder on the respective one of the second images;

an iterative pixel shift device configured to repeatedly shift the given one of the first images relative to the respective one of the second images by a single pixel per shift; and

a correlation calculator configured to determine a correlation score for each shift, such that the parallax separation of the respective two-dimensional locations of the potential intruder is determined based on a given shift that produces a highest correlation score.

11. The system of claim 10, wherein the tracking device comprises a plurality of image filters configured to generate at least one first filtered image associated with each of the first images and at least one second filtered image associated with each of the second images, each of the at least one first filtered image having a resolution that is matched to a respective one of the at least one second filtered image, such that the image overlay device and the iterative pixel shift device are configured to overlay and repeatedly shift the at least one matched pair of the at least one first filtered image and the at least one second filtered image.

12. A method for detecting intruders in a monitored area, the method comprising:

acquiring first images of the monitored area from a first camera and second images of the monitored area from a second camera;

generating a background image of the monitored area;

correlating first pixels associated with the first images with second pixels associated with the background image of the monitored area, such that the first pixels are horizontally and vertically aligned with the second pixels;

comparing the first images and the background image of the monitored area to determine the presence of a potential intruder;

determining three-dimensional characteristics of the potential intruder based on a relative comparison of the first images and the second images; and

activating an indicator upon the three-dimensional characteristics of the potential intruder exceeding at least one predetermined threshold.

13. The method of claim 12, wherein determining the three-dimensional characteristics of the potential intruder comprises determining a parallax separation of the potential intruder in each of the first images relative to the respective second images.

14. The method of claim 13, wherein determining the parallax separation comprises overlaying each of the first images onto a respective one of the second images at respective approximate two-dimensional locations of the potential intruder and shifting each of the first images relative to the respective one of the second images until a substantial correlation is found.

15. The method of claim 13, further comprising generating at least one first filtered image associated with each of the first images and at least one second filtered image associated with each of the second images, each of the at least one first filtered image forming a matched pair with a respective one of the at least one second filtered image having a substantially equal resolution.

16. The method of claim 15, wherein determining the three-dimensional characteristics of the potential intruder comprises equally correlating each of the at least one matched pair to determine a parallax separation of the potential intruder.

17. The method of claim 16, wherein the equally correlating comprises:

equally pixel shifting each of at least one first filtered image relative to the respective matched one of the at least one second filtered image;

generating an aggregate correlation score at each pixel shift, the aggregate correlation score comprising a correlation score of each of the at least one matched pair; and

comparing the aggregate correlation score at each pixel shift to determine the pixel shift having the highest aggregate correlation score.

18. The method of claim 12, wherein generating the background image comprises generating a first background image of the monitored area based on the first camera, and further comprising generating a second background image of the monitored area based on the second camera and comparing the second images and the second background image of the monitored area to determine the presence of the potential intruder.

19. The method of claim 18, further comprising filtering the first images, the second images, the first background image, and the second background image, wherein comparing the first images and the first background image comprises comparing the first filtered images and the first filtered background image, and wherein comparing the second images and the second background image comprises comparing the second filtered images and the second filtered background image.

20. The method of claim 19, further comprising:

updating the first background image based on the comparison between the first filtered images and the first filtered background image;

updating the second background image based on the comparison between the second filtered images and the second filtered background image;

comparing the first images with the updated first background image to determine the presence of the potential intruder; and

comparing the second images with the updated second background image to determine the presence of the potential intruder.

21. The method of claim 12, wherein determining the three-dimensional characteristics comprises determining a location, size, and motion associated with the potential intruder based on differences between the first images and the background image.

22. The method of claim 12, further comprising designating at least one three-dimensional space in the monitored area, such that the at least one three-dimensional space is one of a threshold space and a null zone associated with activating the indicator.

23. An intrusion detection system comprising:

means for simultaneously acquiring first images and second images of a monitored area;

means for continuously generating a background image of the monitored area;

means for detecting a potential intruder based on differences between the first images and the background image;

means for determining three-dimensional characteristics of the potential intruder based on the first images and the second images; and

means for activating an indicator based on the three-dimensional characteristics of the potential intruder.

24. The system of claim 23, wherein the three-dimensional characteristics comprise location, size, and motion.

25. The system of claim 23, wherein the means for determining the three-dimensional characteristics of the potential intruder comprises means for determining a parallax separation of a two-dimensional location of the potential intruder in a given one of the first images relative to a two-dimensional location of the potential intruder in the respective one of the second images.

26. The system of claim 25, wherein the means for determining a parallax separation comprises means for generating a plurality of first filtered images based on the first images and a plurality of second filtered images based on the second images, each of the plurality of first filtered images forming a matched pair with a respective one of the plurality of second filtered images having a substantially equal resolution.

27. The system of claim 26, wherein the means for determining a parallax separation further comprises means for pixel shifting each of the plurality of first filtered images relative to the respective matched one of the plurality of second filtered images and means for generating an aggregate correlation score for each pixel shift based on a correlation score associated with each of the plurality of matched pairs, the parallax separation being associated with the pixel shift having a highest aggregate correlation score.

28. The system of claim 23, further comprising means for horizontally and vertically aligning pixels associated with the first images and the background image prior to the operation of the means for detecting the potential intruder.