Method and apparatus for implementing multipurpose monitoring system

• US 8,111,289 B2
• Filed: 07/15/2003
• Issued: 02/07/2012
• Est. Priority Date: 07/15/2002
• Status: Active Grant

• Current Assignee: Magna B.S.P. Ltd.
• Original Assignee: Magna B.S.P. Ltd.
• Inventors: Zruya, Levi, Sibony, Haim, Nasanov, Viatcheslav, Stekel, Amit
• Primary Examiner(s): Kelley, Christopher S
• Assistant Examiner(s): FINDLEY, CHRISTOPHER G
• Application Number:
  US10/521,207
• Publication Number:
  US 20060049930A1
• Time in Patent Office: 3,129 Days
• Field of Search: None
• US Class Current: 348/144
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19604 involving reference image or background adaptation with time to compensate for changing conditions, e.g. reference image update on detection of light level change
• CPC Class Codes: G08B 13/1963 Arrangements allowing camera rotation to change view, e.g. pivoting camera, pan-tilt and zoom [PTZ]
• CPC Class Codes: G08B 13/19643 wherein the cameras play different roles, e.g. different resolution, different camera type, master-slave camera
• CPC Class Codes: G08B 13/1965 the vehicle being an aircraft
• CPC Class Codes: G08B 13/19691 Signalling events for better perception by user, e.g. indicating alarms by making display brighter, adding text, creating a sound

Abstract

Method for the monitoring of an environment, by procuring, adjourning and storing in a memory, files representing the background space. Programs for processing data obtained from the observation of objects are defined and stored in a memory, for identifying the objects and for determining whether they are dangerous. Parameters, according to which the observation of the controlled space is effected, are determined and stored. Photographic observation of the controlled space or sections thereof, is performed according to the aforesaid observation parameters. The digital data representing these photographs are processed to determine whether possible dangerous objects have been detected, and if so, these objects are classified according to the stored danger parameters.

Claims

1. Method for the monitoring of an environment by use of one or more pairs of imagers, each imager monitoring said environment from a different angle of view, comprising the steps of:

a) providing one or more pairs of imagers, wherein at least one of said pairs comprises an optical imager and at least one of said pairs comprises a Forward Looking Infra Red (FLIR) imager;

b) positioning both imagers of said one or more pairs of imagers on a common vertically oriented pole such that they are located at a distance of 0.5 to 50 meters from each other and capture a same object at a different angle of view, in order to perform a vertical stereoscopic observation of objects;

c) defining and storing in a memory, programs for processing, in real-time, photographic data to be obtained from the stereoscopic observation of objects by use of said one or more pairs of imagers;

d) determining and storing parameters according to which the observation of a controlled space or sections thereof is effected;

e) carrying out an unmanned, real-time vertical stereoscopic observation of said controlled space or sections thereof, according to the observation parameters;

f) determining the distance to said observed objects from said one or more pairs of imagers;

g) evaluating the size of each of said observed objects;

h) classifying a type and degree of danger of each of said observed objects by jointly processing said real-time data obtained from said stereoscopic observation with respect to the angle of view of each of said imagers, the path and size of, and distance from, said observed objects, and said stored danger parameters; and

i) providing an indication when one or more of said observed objects is approaching said controlled space and has been classified as having a sufficiently high degree of danger so as to be liable of damaging an authorized body within said controlled space.

2. Method according to claim 1, further comprising:

a) changing the sections of said stereoscopic observation so as to monitor the path of any detected dangerous objects;

b) receiving and storing the data defining the positions and the foreseen future path of all authorized bodies;

c) extrapolating the data obtained by monitoring the path of any detected dangerous objects to determine an assumed future path of said objects; and

d) comparatively processing said assumed future path with the foreseen future path of all authorized bodies, to determine the possible danger of collision or intrusion.

3. Method according to claim 2, further comprising determining an action on dangerous objects that will eliminate the danger of collision, intrusion or damage.

4. Method according to claim 3, wherein the action is the destruction of the dangerous object.

5. Method according to claim 3, wherein the action is change in their assumed future path the dangerous object.

6. Method according to claim 2, further comprising determining an action on an authorized body that will eliminate the danger of collision, intrusion or damage.

7. Method according to claim 6, wherein the action is a delay in the landing or take-off of an aircraft or a change of the landing or take-off path of said aircraft.

8. Method according to claim 1, wherein the indication is an alarm signaling the presence and nature of any dangerous objects, the danger of collisions and possible desirable preventive actions.

9. Method according to claim 1, wherein the stereoscopic observation is carried out by performing the steps of:

a) modifying the angle of one or more imagers;

b) capturing one or more images with said of one or more imagers;

c) processing said captured one or more images by a computerized system; and

d) repeating steps a) to c).

10. Method according to claim 9, wherein the stereoscopic observation is carried out as a continuous scan or segmental scan.

11. Method according to claim 1, wherein the processing of the real-time photographic data comprises the steps of:

a) setting initial definition for the stereoscopic observation and for the processing of the data of said stereoscopic observation;

b) storing in the memory the data that represent the last captured one or more images at a specific angle of the imagers; and

c) processing said data for detecting suspected objects, by performing, firstly, pixel processing and secondly, logical processing; and

d) deciding whether said suspected object is a dangerous object.

12. Method according to claim 11, wherein the pixel processing comprises the steps of:

a) mathematically processing each pixel in a current photo for detecting suspected objects; and

b) whenever a suspected object is detected, providing images at the same time period and of the same monitored section by means of both imagers of the one or more pairs of imagers for generating three-dimensional data related to said suspected object.

13. Method according to claim 12, wherein whenever the pixel processing detects a moving object, further comprising the steps of:

a) comparing the current images to an average image generated from the previous stored images, said previous stored images and said current image being captured at the same imager angle;

b) generating a comparison image from the difference in the pixels between said average image said current image, each pixel in said comparison image representing an error value;

c) comparing each error value to a threshold level, said threshold level being dynamically determined to each pixel in the image matrix statistically according the previous pixel values stored in the memory as a statistic database;

d) whenever a pixel value in said comparison image exceeds said threshold level, generating a logic matrix in which the location of said pixel value is set to a predetermined value; and

e) upon completing comparing each error value to said threshold level, for the entire current images, transferring said generated logic matrix to the logic process stage.

14. Method according to claim 12, wherein whenever the pixel processing detects a static object, further comprising the steps of:

a) generating an average image from the current one or more images;

b) generating a derivative matrix from said average image for emphasizing relatively small objects at each image from said one or more images, which might be potential dangerous objects;

c) storing said derivative matrix in the memory as part of an image database, and comparing said derived matrix with a previous derivative matrix stored in said memory as part of said image database, said previous derivative matrix being derived from one or more images that was taken from the exact imager angle as of said average image;

d) from the comparison, generating an error image, wherein each pixel in said error image represents the error value between said derivative matrix and said previous derivative matrix;

e) comparing the value of each pixel from said error image to a threshold level, said threshold level being dynamically determined to each pixel in the error image statistically according the previous pixel values stored in the memory as a part of a statistic database;

f) whenever a pixel value in said error image exceeds said threshold level, generating a logic matrix in which the location of said pixel value is set to a predetermined value; and

g) upon completing comparing each error value to said threshold level, for the entire current images, transferring said generated logic matrix to the logic process stage.

15. Method according to claim 11, wherein the logic processing comprises the steps of:

a) measuring parameters regarding the pixels in the logic matrix;

b) comparing said measured parameters to a predetermined table of values stored in the memory, whenever said measured parameters equal to one or more values in said table, the pixels that relates to said measurement are dangerous objects.

16. Method according to claim 15, wherein the parameters are selected from the group consisting of the dimension of an adjacent group of pixels, the track that one or more adjacent pixels created in the logic matrix, direction, speed, size and location of an object that is created from a group of pixels.

17. Method according to claim 1, further comprising rotating both imagers of a pair of imagers and simultaneously changing their corresponding angle of view.

18. Method according to claim 17, further comprising providing at least one encoder and at least one reset sensor for determining the angle of each imager, said encoder and reset sensor being provided to each axis that rotates an imager.

19. Method according to claim 18, wherein the reset sensor provides the initiation angle of the imager at the beginning of the scanning of a sector and the encoder provides the current angle of the imager during the scanning of the sector.

20. Method according to claim 1, further comprising the steps of:

a) generating a panoramic image and a map of the monitored area by scanning said area, said scanning being performed by rotating at least a pair of distinct and identical imagers around their central axis of symmetry;

b) obtaining the referenced location of a detected object by observing said object with said imagers, said location being represented by the altitude, range and azimuth parameters of said object; and

c) displaying the altitude value of said object on said panoramic image and displaying the range and the azimuth of said object on said map.

21. Method according to claim 20, wherein the imagers are selected from the group consisting of optical imagers, thermal imagers, CCD cameras, CMOS based cameras, and Forward Looking Infra Red (FLIR) cameras.

22. Method according to claim 20, wherein the distance, in an angle, between each two imagers is between 0.5 to 50 meters.

23. Method according to claim 20, wherein the imagers are not identical and do not share common central axis of symmetry or of optical magnification but have at least an overlapping part of their field of view.

24. Method according to claim 1, further comprising documenting the activities of the wildlife and other dangerous objects, for preventing and reducing from said wildlife and said other dangerous objects to appear at the monitored area.

25. Method according to claim 1, further comprising procuring, adjourning and storing in a memory file representing a background space.

26. Apparatus for the monitoring an environment by use of one or more pair of imagers, each imager monitoring said environment from a different angle of view, comprising:

a) one or more pairs of imagers for obtaining real-time photographic data of a controlled space or sections thereof, wherein both imagers of said one or more pairs of imagers are positioned along a common vertically oriented pole and are vertically spaced by a distance of 0.5 to 50 meters from each other, for performing an unmanned vertical stereoscopic observation of objects by capturing a same object at a different angle of view;

b) memory means in which are stored boundary parameters of a controlled space or sections thereof, danger parameters of observed objects and real-time photographic data processing instructions; and

c) a processing unit, operable to;

i. jointly process real-time photographic data obtained from said stereoscopic observation, with respect to the angle of view of each of said imagers and according to said instructions;

ii. determine the distance to said observed objects from said one or more pairs of imagers;

iii. evaluate the size of each of said observed objects;

iv. classify a type and degree of danger of each of said observed objects according to said processed photographic data, the path and size of, and distance from, said observed objects, and said stored danger parameters; and

v. provide an indication when one or more of said observed objects is approaching said controlled space and has been classified as having a sufficiently high degree of danger so as to be liable of damaging an authorized body within said controlled space,

wherein at least one of said pairs comprises an optical imager and at least one of said pairs comprises a Forward Looking Infra Red (FUR) imager.

27. Apparatus according to claim 26, wherein the imagers are photographic devices selected from the group consisting of:

CCD or CMOS cameras, infrared cameras, and Forward Looking Infra Red (FLIR) cameras.

28. Apparatus according to claim 26, wherein both imagers of a pair of imagers are identical.

29. Apparatus according to claim 26, wherein both imagers of a pair of imagers are provided with a different lens.

30. Apparatus according to claim 26, further comprising:

a) elaborator means for obtaining the referenced location of a detected object in said controlled space, said location being represented by the altitude, range and azimuth parameters of said object;

b) means for generating a panoramic image and a map of the monitored area;

c) means for displaying the altitude value of said object on said panoramic image and means for displaying the range and the azimuth of said object on said map.

31. Apparatus according to claim 30, wherein the means for displaying the monitored area comprise three-dimensional software graphics for indicating the location of each detected object as a three-dimensional image.

32. Apparatus according to claim 30, wherein the elaborator means comprise one or more dedicated algorithm installed within a computerized system.

33. Apparatus according to claim 26, further comprising a laser range finder being electrically connected to a computerized system for measuring the distance of a detected object from said laser range finder, said laser range finder transfers to said computerized system data representing the distance from a detected object, thereby aiding said computerized system to obtain the location of said detected object.

34. Apparatus according to claim 26, further comprising a set of motors for displacing the imagers, for changing the controlled space or sections thereof, and for thereby generating the real-time photographic data.