SURVEILLANCE SYSTEM AND A METHOD FOR DETECTING A FOREIGN OBJECT, DEBRIS, OR DAMAGE IN AN AIRFIELD

• US 20130329052A1
• Filed: 02/21/2012
• Published: 12/12/2013
• Est. Priority Date: 02/21/2011
• Status: Abandoned Application

• Current Assignee: Stratech Systems Limited (In Liquidation) (The Stratech Group Ltd.)
• Original Assignee: Stratech Systems Limited (In Liquidation) (The Stratech Group Ltd.)
• Inventors: Chew, Khien Meow David
• Application Number:
  US14/000,843
• Publication Number:
  US 20130329052A1
• Time in Patent Office: Days
• US Class Current: 348/159
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/0008 checking presence/absence
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08G 5/065 Navigation or guidance aids, e.g. for taxiing or rolling
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A surveillance system and method for detecting a foreign object, debris, or damage in an airfield, the surveillance system comprising: one or more cameras for capturing images of the airfield; a processing unit for detecting the foreign object, debris or damage in the airfield from the images captured by the one or more cameras; and a weapons impact surveillance system for detecting weapon impact in the airfield and directing the one or more cameras to capture images in an area of the detected weapon impact.

Claims

1. A surveillance system for detecting a foreign object, debris, or damage in an airfield, the surveillance system comprising:

one or more cameras for capturing images of the airfield;

a processing unit for detecting the foreign object, debris or damage in the airfield from the images captured by the one or more cameras; and

a weapons impact surveillance system for detecting weapon impact in the airfield and directing the one or more cameras to capture images in an area of the detected weapon impact.

2. The surveillance system as claimed in claim 1, wherein the one or more cameras comprise a combination of static and non-static cameras.

3. The surveillance system as claimed in claims 1, wherein the one or more cameras zoom in or focus on the area of the detected weapon impact to obtain images with details for detecting type of damage.

4. The surveillance system as claimed in claim 1, wherein the weapon impact is detected by images of explosion, smoke, dust or flash caused by the weapon impact.

5. The surveillance system as claimed in claim 1, wherein the surveillance system comprises a computation module for deriving a Minimum Operating Strip (MOS) for aircraft landing based on location of the foreign object, debris, or damage detected in the airfield.

6. The surveillance system of claim 1, wherein the processing unit detects cracks, crater, camouflet, spall, UXO, or an animal in the airfield.

7. The surveillance system as claimed in claim 6, wherein the processing unit determine size of a crater, camouflet or spall by detecting amount of debris around the crater, camouflet or spall respectively.

8. The surveillance system of claim 1, wherein the one or more cameras are equipped with wide angle lens to provide wide area image capture of the airfield.

9. The surveillance system of claim 1, wherein the one or more cameras is mounted on an aircraft arranged to fly over the airfield to capture images of the airfield.

10. The surveillance system as claimed in claim 1, wherein the surveillance system comprises one or more infrared illuminators for providing artificial illumination under low visibility or low ambient illumination conditions.

11. The surveillance system as claimed in claim 1, wherein the processing unit is configured for measuring size or physical attributes of the foreign object, debris, or damage.

12. The surveillance system as claimed in claim 1, wherein the damage detected in the airfield is mapped onto an airfield map.

13. The surveillance system as claimed in claim 1, wherein the surveillance system comprises

a region based detector for detecting a region in the images captured that can be a foreign object, debris or damage;

and

an edge based detector for detecting edges of all objects in the images captured, wherein the region detected by the region based detector in the images that is overlapping with the edges detected by the edge based detector in the images are stored.

14. The surveillance system of claim 1, wherein the images captured by the one or more cameras are stitched together and areas of differences between earlier captured stitched images and later captured stitched images are highlighted in the later captured stitched images.

15. The surveillance system of claim 1, wherein the weapons impact surveillance system is configured to trigger a visual or audio alert upon detecting weapon impact.

16. The surveillance system as claimed in claim 1, wherein the surveillance system comprises a repair estimation and planning module for estimating and planning repair work based on location of the weapon impact or information of the airfield damage.

17. The surveillance system of claim 1, wherein the one or more cameras are installed on opposite sides of a runway in the airfield and their field of views is overlapped.

18. The surveillance system as claimed in claim 1, wherein field of views of the one or more cameras are overlapped for providing coverage redundancy.

19. The surveillance system as claimed in claim 1, wherein the surveillance system comprises one or more mobile handheld devices for receiving remote alert and critical information to repair damage or remove foreign object in the airfield.

20. The surveillance system as claimed in claim 1, wherein the surveillance system comprises an image offset compensator for correcting offset between a currently captured image and a reference image taken by a camera based on position of a common object located in the currently captured image and a reference image.

21. The surveillance system as claimed in claim 20, wherein the image offset compensator uses more than one common object in the currently captured image and the reference image to determine the offset to correct between the currently captured image and the reference image.

22. The surveillance system as claimed in claim 21, wherein linear regression is used to determine the offset to correct between the currently captured image and the reference image, and an offset value calculated based on a common object in the currently captured image and the reference image is considered for linear regression if a score determined by matching the common object in the currently captured image and the reference image is greater than a predetermined threshold value.

23. The surveillance system as claimed in claim 1, wherein the surveillance system further comprises a computation module for deriving a Minimum Airfield Operating Surface (MAOS) for aircraft based on location of the foreign object, debris, or damage detected in the airfield.

24. The surveillance system as claimed in claim 1, wherein the surveillance system comprises one or more visible spectrum illuminators for providing artificial illumination under low visibility or low ambient illumination conditions.

25. The surveillance system as claimed in claim 1, wherein the weapon impact is detected by sound caused by the weapon impact.

26. The surveillance system as claimed in claim 1, wherein the processing unit detects the foreign object, debris or damage in the airfield based on adaptive image processing of the images captured by the cameras and

the surveillance system is adaptively operable for detecting the foreign object, debris or damage under both day and night ambient light conditions without assisted illumination such as infrared or laser illuminators.

27. The surveillance system as claimed in claim 1, wherein the processing unit applies image enhancement methods to enhance the captured images.

29. The surveillance system as claimed in claim 27, wherein the processing unit determines if the instant of processing is a day-time or night-time;

and detects an abnormal light condition, such as due to aircraft landing or aircraft taking off or ground vehicle movement, from the captured image during night-time.

30. The surveillance system as claimed in claim 29, wherein detecting of the abnormal light condition comprises global histogram and statistical analysis to compare each image with one or more preceding images and identifies the abnormal light condition based on a change in intensity with reference to a threshold value.

31. The surveillance system as claimed in claim 29, wherein images for which the abnormal light condition detected are ignored from further processing.

32. The surveillance system as claimed in claim 27, wherein the processing unit adaptively estimates one or more threshold values for optimal foreign object, debris or damage edge extraction for different environmental conditions;

and generates a pixel level edge map using a statistical method based on progressively learned background image edge map to determine the grayscale lookup table (LUT) to be used to generate pixel level threshold map.

33. The surveillance system as claimed in claim 32, wherein the processing unit further applies temporal filtering to a stack of pixel level edge maps to retain only the robust edge map which consists only of pixels that have accumulated to pass the threshold.

34. The surveillance system as claimed in claim 33, wherein the processing unit further subjects the robust edge map to adaptive background learning, the adaptive background learning comprising:

comparing background edge images obtained at previous instants with current image;

identifying slow-change features in the airfield; and

updating the background edge image with the slow changing features.

35. The surveillance system as claimed in claim 34, wherein the processing unit further generates a composite background edge map comprising an adaptive background edge map, a previously learned and saved day or night background edge map, and a seasonal marking map generated for a particular season or weather conditions.

36. The surveillance system as claimed in claim 35, wherein the processing unit further compares the composite background edge map and the robust edge map;

and removes background edges to extract a suspected edge map of the foreign object, debris or damage.

37. The surveillance system as claimed in claim 36, wherein the processing unit further performs edge filtering to filter unwanted edges related to environmental changes from the suspected edge map, and computes edge parameters of the foreign object, debris or damage from the suspected edge map,

38. The surveillance system as claimed in claim 37, wherein the environmental conditions include day to night transition, or night to day transition, weather conditions, rain, smoke, cloud or the like.

39. The surveillance system as claimed in claim 1, wherein image processing performed by the processing unit includes overlaying a foreign object, debris or damage graphic on a suspected region of the airfield on a video display to alarm an operator at a control tower or control room of foreign object, debris or damage detection.

40. The surveillance system as claimed in claim 1, wherein the processing unit further classifies the foreign object, debris or damage.

41. The surveillance system as claimed in claim 1, wherein the one or more cameras are placed on one side of an airfield runway.

42. The surveillance system as claimed in claim 1, wherein when one or more cameras fail to function, respective adjacent cameras are operable to cover the areas covered by the failed cameras.

43. The surveillance system as claimed in claim 1, wherein the one or more cameras comprises one or more monochrome cameras, one or more colour cameras or both.

44. The surveillance system as claimed in claim 1, further comprising one or more night vision cameras.

45. The surveillance system as claimed in claim 1, wherein a runway surface in the airfield is divided into a plurality of segments, and one or more non-static cameras sequentially scan the runway segment-by-segment for foreign object, debris or damage detection.

46. The surveillance system as claimed in claim 1, wherein a static camera detects respective locations of aircraft take off and landing on the runway such that a non-static camera is directed to first scan runway segments in the respective locations of aircraft landing or take off to reduce foreign object, debris or damage detection time.

47. The surveillance system as claimed in claim 1, wherein the processing unit applies temporal filtering to filter out rain clutter in runway scene images by recognizing rain-like characteristics of rain motion clutter and based on the motion clutter due to rain occurring across the entire runway.

48. The surveillance system as claimed in claim 1, wherein the processing unit applies temporal filtering to filter out snow clutter in runway scene images of a runway in the airfield by recognizing snow-like characteristics of snow motion clutter and based on the motion clutter due to snow occurring across the entire runway.

49. The surveillance system as claimed in claim 1, wherein the processing unit makes use of markers or runway edge lights located along a longitudinal direction on a runway in the airfield and on same vertical distance from a side of the runway for runway scene calibration to map pixels on images of the runway to precise co-ordinates on a real-world co-ordinate frame.

50. The surveillance system as claimed in claim 1, wherein the processing unit makes use of two parallel horizontal runway lines on each side of a runway middle line and the runway middle line to derive two vertical pixel mapping ratios for runway scene calibration to map pixels on the images on a runway in the airfield to precise co-ordinates on a real-world co-ordinate frame.

51. The surveillance system as claimed in claim 1, wherein the processing unit makes use of monoscopic vision and calibrated runway scene image captured by a monoscopic camera to determine the position and range of the foreign object, debris or damage on a runway in the airfield.

52. The surveillance system as claimed in claim 1, wherein the system makes use of the foreign object, debris or damage position and range determined by a static camera and a calibrated runway scene image to automatically control a non-static camera to pan, tilt, zoom or focus onto a foreign object, debris or damage to obtain images of the foreign object, debris or damage with details to enable verification of accuracy of the detected foreign object, debris or damage or to filter a false alarm.

53. The surveillance system as claimed in claim 1, wherein the surveillance system makes use of stereo vision including a pair of surveillance cameras to cover a same segment of a runway in the airfield so that foreign object, debris or damage range and position can be computed from differences in images obtained by comparing images respectively captured by the pair of surveillance cameras with overlapping field of views.

28. The surveillance system as claimed in 27, wherein said methods for enhancing the captured images comprises a high pass filter, a Sobel X from left_to_right filter and a Sobel X from right_to_left filter, or a Scharr X filter to the captured image.

54. A method for detecting a foreign object, debris, or damage in an airfield, the method comprising:

capturing images of the airfield;

detecting the foreign object, debris or damage in the airfield from the images captured;

detecting weapon impact in the airfield; and

directing one or more cameras to capture images in an area of the detected weapon impact.