Method and apparatus for implementing multipurpose monitoring system

• US 20060049930A1
• Filed: 07/15/2003
• Published: 03/09/2006
• Est. Priority Date: 07/15/2002
• Status: Active Grant

• Current Assignee: Magna B.S.P. Ltd.
• Original Assignee: Magna B.S.P. Ltd.
• Inventors: Stekel, Amit, Nasanov, Viatcheslav, Sibony, Haim, Zruya, Levi
• Granted Patent: US 8,111,289 B2
• Time in Patent Office: Days
• US Class Current: 340/500
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19604 involving reference image or background adaptation with time to compensate for changing conditions, e.g. reference image update on detection of light level change
• CPC Class Codes: G08B 13/1963 Arrangements allowing camera rotation to change view, e.g. pivoting camera, pan-tilt and zoom [PTZ]
• CPC Class Codes: G08B 13/19643 wherein the cameras play different roles, e.g. different resolution, different camera type, master-slave camera
• CPC Class Codes: G08B 13/1965 the vehicle being an aircraft
• CPC Class Codes: G08B 13/19691 Signalling events for better perception by user, e.g. indicating alarms by making display brighter, adding text, creating a sound

Abstract

Method for the monitoring of an environment, by procuring, adjourning and storing in a memory, files representing the background space. Programs for processing data obtained from the observation of objects are defined and stored in a memory, for identifying the objects and for determining whether they are dangerous. Parameters, according to which the observation of the controlled space is effected, are determined and stored. Photographic observation of the controlled space or sections thereof, is performed according to the aforesaid observation parameters. The digital data representing these photographs are processed to determine whether possible dangerous objects have been detected, and if so, these objects are classified according to the stored danger parameters.

Claims

1. Method for the monitoring of an environment, comprising the steps of:

a) defining and storing in a memory programs for processing, in real-time, data obtained from the observation of objects by one or more pairs of optical and/or thermal imagers, relatively positioned along a common vertical line, for identifying said objects and determining whether they are dangerous;

b) determining and storing parameters according to which the observation of the controlled space is effected;

c) carrying out photographic observation of the controlled space or sections thereof, according to the aforesaid observation parameters; and

d) jointly processing the digital data representing said optical and thermal photographs, to determine whether possible dangerous objects have been detected, and if so, classifying said objects according to the stored danger parameters.

2. Method according to claim 1, further comprising:

a) changing the sections of the said photographic observation so as to monitor the path of any detected dangerous objects;

b)receiving and storing the data defining the positions and the foreseen future path of all authorized bodies;

c) extrapolating the data obtained by monitoring the path of any detected dangerous objects to determine an assumed future path of said objects; and

d) comparatively processing said assumed future path with the foreseen future path of all authorized bodies, to determine the possible danger of collision or intrusion.

3. Method according to claim 2, further comprising determining an action on dangerous objects that will eliminate the danger of collision, intrusion or damage.

4. Method according to claim 3, wherein the action is the destruction of the dangerous object.

5. Method according to claim 3, wherein the action is change in their assumed future path the dangerous object.

6. Method according to claim 2, further comprising determining an action on an authorized body that will eliminate the danger of collision, intrusion or damage.

7. Method according to claim 6, wherein the action is a delay in their landing or take-off of the aircraft or a change of their landing or take-off path.

8. Method according to claim 1, further comprising giving alarms signaling the presence and nature of any dangerous objects, the danger of collisions and possible desirable preventive actions.

9. Method according to claim 1, wherein the photographic observation is carried out by performing the steps of:

a) modifying the angle of one or more photographic devices;

b) photographing one or more photos with said photographic device;

c) processing said photographed one or more photos by a computerized system; and

d) repeating steps a) to c).

10. Method according to claim 9, wherein the photographic observation is carried out as a continuous scan or segmental scan.

11. Method according to claim 1, wherein the processing of the digital data comprises the step of:

a) setting initial definition for the photographic observation and for the processing of the data of said photographic observation;

b) storing in the memory the data that represent the last photographed one or more photos at a specific angle of the photographic devices; and

c) processing said data for detecting suspected objects, by performing, firstly, pixel processing and secondly, logical processing; and

d) deciding whether said suspected object is a dangerous object.

12. Method according to claim 11, wherein the pixel processing comprises the step of:

a) Mathematically processing each pixel in a current photo for detecting suspected objects; and

b) Whenever a suspected object is detected, at least two photographic devices, being positioned vertically one above the other in distance from each other, provides photos at same time period and same monitored section, generating data regarding said suspected object from at least said two photographic devices, said generated data is a 3-D data.

13. Method according to claim 12, wherein whenever the pixel processing detects moving object, it comprises the steps of:

a) comparing the current photo to an average photo generated from the previous stored photos, said previous stored photos and said current photo was photographed at the same photographic device angle;

b) generating a comparison photo from the difference in the pixels between said average photo said current photo, each pixel in said comparison photo represents an error value;

c) comparing each error value to a threshold level, said threshold level is dynamically determined to each pixel in the photo matrix statistically according the previous pixel values stored in the memory as a statistic database;

d) whenever a pixel value in said comparison photo exceeds said threshold level, generating a logic matrix in which the location of said pixel value is set to a predetermined value; and

e) upon completing comparing each error value to said threshold level, for the entire current photos, transferring said generated logic matrix to the logic process stage.

14. Method according to claim 12, wherein whenever the pixel processing detects static object, it comprises the steps of:

a) generating an average photo from the current one or more photos;

b) generating a derivative matrix from said average photo for emphasis relatively small objects at each photo from said one or more photo, which might be potential dangerous objects;

c) storing said derivative matrix in the memory as part of a photo database, and comparing said derived matrix with previous derivative matrix stored in said memory as part of said photo database, said previous derivative matrix is derived from one or more photos that was taken from the exact photographic device angle as of said average photo;

d) From the comparison, generating an error photo, wherein each pixel in said error photo represents the error value between said derivative matrix and said previous derivative matrix;

e) comparing the value of each pixel from said error photo to a threshold level, said threshold level is dynamically determined to each pixel in the error photo statistically according the previous pixel values stored in the memory as a part of a statistic database;

f) whenever a pixel value in said error photo exceeds said threshold level, generating a logic matrix in which the location of said pixel value is set to a predetermined value; and

g) upon completing comparing each error value to said threshold level, for the entire current photos, transferring said generated logic matrix to the logic process stage.

15. Method according to claim 11, wherein the logic processing comprises the steps of:

a) measuring parameters regarding the pixels in the logic matrix;

b) comparing said measured parameters to a predetermined table of values stored in the memory, whenever said measured parameters equal to one or more values in said table, the pixels that relates to said measurement are dangerous objects.

16. Method according to claim 15, wherein the parameters are selected from the group consisting of the dimension of an adjacent group of pixels, the track that one or more adjacent pixels created in the logic matrix, direction, speed, size and location of an object that is created from a group of pixels.

17. Method according to claim 1, wherein the photographic observation is taken from at least two cameras.

18. Method according to claim 17, wherein the cameras positioned with the same view angle are located at a distance of 0.5 to 50 meters from each other.

19. Method according to claim 18, wherein the cameras positioned with same view angle are installed on the same pole.

20. Method according to claim 18, wherein the cameras positioned with same view angle are being rotated thus their view angle is changed simultaneously.

21. Method according to claim 18, further comprising providing at least one encoder and at least one reset sensor for determining the angle of each camera, said encoder and reset sensor are provided to each axis that rotates a camera.

22. Method according to claim 21, wherein the reset sensor provides the initiation angle of the camera at the beginning of the scanning of a sector and the encoder provides the current angle of the camera during the scanning of the sector.

23. Method according to claim 1, further comprising the steps of:

a) generating a panoramic image and a map of the monitored area by scanning said area, said scanning being performed by rotating at least a pair of distinct and identical imagers around their central axis of symmetry;

b) obtaining the referenced location of a detected object by observing said object with said imagers, said location being represented by the altitude, range and azimuth parameters of said object; and

c) displaying the altitude value of said object on said panoramic image and displaying the range and the azimuth of said object on said map.

24. Method according to claims 23, wherein the imagers are photographic devices selected from the group consisting of:

CCD or CMOS based cameras or Forward Looking Infra Red (FLIR) cameras.

25. Method according to claim 23, wherein the distance, in an angle, between each two imagers is between 0.5 to 50 meters.

26. Method according to claim 23, wherein the imagers are not identical and do not share common central axis of symmetry or of optical magnification but have at least an overlapping part of their field of view.

27. Method according to claim 1, further comprising documenting the activities of the wildlife and other dangerous objects, for preventing and reducing from said wildlife and said other dangerous objects to appear at the monitored area.

37. Method according to claim 1, further comprising procuring, adjourning and storing in a memory files representing the background space.

28. Apparatus for the monitoring an environment, comprising:

a) one or more pairs of optical and/or thermal imagers, relatively positioned along a common vertical line for carrying out photographic/thermal observation of the controlled space or sections thereof;

b) a set of motors for changing the sections of the said photographic observation;

c) elaborator means for jointly processing the digital data representing said optical and thermal photographs, to determine whether possible dangerous objects have been detected, and if so, classifying said objects according to the stored danger parameters, processing the digital data representing the photographs taken by said photographic devices;

d) memory means for storing programs for processing, in real-time, data obtained from the observation of objects by said imagers, and for identifying objects and determining whether they are dangerous.

29. Apparatus according to claim 28, wherein the photographic devices comprise one or more CCD or CMOS cameras and/or one or more infrared cameras.

30. Apparatus according to claim 28, wherein the distance, in an angle, between each two cameras located on the same pole is between 0.5 to 50 meters.

31. Apparatus according to claim 28, in which the photographic devices are at least a pair of distinct and identical imagers.

32. Apparatus according to claim 28, in which each photographic device is provided with a different lens.

33. Apparatus according to claim 28, further comprising:

a) elaborator means for obtaining the referenced location of a detected object in said controlled space, said location being represented by the altitude, range and azimuth parameters of said object;

b) means for generating a panoramic image and a map of the monitored area;

c) means for displaying the altitude value of said object on said panoramic image and means for displaying the range and the azimuth of said object on said map.

34. Apparatus according to claim 33, in which the means for displaying the monitored area are using three-dimensional software graphics where the location of each detected object is indicated as a three-dimensional image.

35. Apparatus according to claim 33, in which the elaborator means are one or more dedicated algorithm installed within the computerized system.

36. Apparatus according to claim 28, further comprising a laser range finder being electrically connected to the computerized system for measuring the distance of a detected object from said laser range finder, said laser range finder transfers to said computerized system data representing the distance from a detected object, thereby aiding said computerized system to obtain the location of said detected object.