RUNWAY SURVEILLANCE SYSTEM AND METHOD

• US 20110063445A1
• Filed: 08/25/2008
• Published: 03/17/2011
• Est. Priority Date: 08/24/2007
• Status: Active Grant

• Current Assignee: Changi Airport Group Pte Limited (Government of Singapore)
• Original Assignee: Stratech Systems Limited (In Liquidation) (The Stratech Group Ltd.)
• Inventors: Chew, Khien Meow David
• Granted Patent: US 9,483,952 B2
• Time in Patent Office: Days
• US Class Current: 348/159
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30184 Infrastructure
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 7/0008 checking presence/absence
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19652 Systems using zones in a single scene defined for different treatment, e.g. outer zone gives pre-alarm, inner zone gives alarm
• CPC Class Codes: G08B 13/19691 Signalling events for better perception by user, e.g. indicating alarms by making display brighter, adding text, creating a sound
• CPC Class Codes: G08G 5/0026 located on the ground
• CPC Class Codes: G08G 5/04 Anti-collision systems
• CPC Class Codes: G08G 5/06 for control when on the ground
• CPC Class Codes: G08G 5/065 Navigation or guidance aids, e.g. for taxiing or rolling

Abstract

A surveillance system and method for detecting a foreign object, debris, or damage (FOD) on a runway. The system comprises one or more cameras for capturing images of the runway; and an image processing system for detecting the FOD on the runway based on adaptive image processing of the images captured by the cameras; wherein the surveillance system is adaptively operable for FOD detection under both day and night ambient light conditions without assisted illumination such as infrared or laser illuminators.

Claims

1. A surveillance system for detecting a foreign object, debris, or damage (FOD) on a runway comprising:

one or more cameras for capturing images of the runway; and

an image processing system for detecting the FOD on the runway based on adaptive image processing of the images captured by the cameras;

wherein the surveillance system is adaptively operable for FOD detection under both day and night ambient light conditions without assisted illumination such as infrared or laser illuminators.

2. The surveillance system according to claim 1, wherein the image processing system applies image enhancement methods to enhance the captured images.

3. The surveillance system according to claim 2, wherein means for enhancing the captured images comprises a high pass filter, a Sobel X from left_to_right filter and a Sobel X from right_to_left filter, or a Scharr X filter to the captured image.

4. The surveillance system according to claim 2, wherein the image processing system determines if the instant of processing is a day-time or night-time;

and detects an abnormal light condition, such as due to aircraft landing or aircraft taking off or ground vehicle movement, from the captured image during night-time.

5. The surveillance system according to claim 4, wherein detecting of the abnormal light condition comprises global histogram and statistical analysis to compare each image with one or more preceding images and identifies the abnormal light condition based on a change in intensity with reference to a threshold value.

6. The surveillance system according to claim 4, wherein images for which the abnormal light condition are detected are ignored from further processing.

7. The surveillance system according to claim 2, wherein the image processing system adaptively estimates one or more threshold values for optimal FOD edge extraction for different environmental conditions;

and generates a pixel level edge map using a statistical method based on progressively learned background image edge map to determine the grayscale lookup table (LUT) to be used to generate pixel level threshold map.

8. The surveillance system according to claim 7, wherein the image processing system further applies temporal filtering to a stack of pixel level edge maps to retain only the robust edge map which consists only of pixels that have accumulated to pass the threshold.

9. The surveillance system according to claim 8, wherein the image processing system further subjects the robust edge map to adaptive background learning, the adaptive background learning comprising:

comparing background edge images obtained at previous instants with current image;

identifying slow-change features on the runway; and

updating the background edge image with the slow changing features.

10. The surveillance system according to claim 9, wherein the image processing system further generates a composite background edge map comprising an adaptive background edge map, a previously learned and saved day or night background edge map, and a seasonal marking map generated for a particular season or weather conditions.

11. The surveillance system according to claim 10, wherein the image processing system further compares the composite background edge map and the robust edge map;

and removes background edges to extract a suspected edge map of FOD.

12. The surveillance system according to claim 11, wherein the image processing system further performs edge filtering to filter unwanted edges related to environmental changes from the suspected edge map, and computes edge parameters of FOD from the suspected edge map,

13. The surveillance system according to claim 12, wherein the environmental conditions include day to night transition, or night to day transition, weather conditions, rain, smoke or cloud.

14. The surveillance system according to claim 1, wherein the image processing further overlays an FOD graphic on a suspected region of the runway on a video display to alarm an operator at a control tower or control room of FOD detection.

15. The surveillance system according to claim 14, wherein one or more of the cameras, or one or more additional cameras are arranged for zooming on to the suspected region for visual verification.

16. The surveillance system as claimed in claim 1, wherein the image processing system further classifies the FOD.

17. The surveillance system according to claim 1, wherein the one or more cameras comprises one or more static cameras, one or more non-static cameras or a combination of both static and non static cameras.

18. The surveillance system according to claim 1, wherein the cameras are placed on one side of the runway.

19. The surveillance system according to claim 1, wherein the cameras are placed on either sides of the runway in a staggered manner.

20. The surveillance system according to claim 1, wherein when one or more cameras fail to function, respective adjacent cameras are operable to cover the areas covered by the failed cameras.

21. The surveillance system according to claim 1, wherein the one or more cameras comprises one or more monochrome cameras, one or more colour cameras or both.

22. The surveillance system according to claim 1, further comprising one or more night vision cameras.

23. The surveillance system according to claim 1, wherein a runway surface is divided into a plurality of segments, and one or more non-static cameras sequentially scan the runway segment-by-segment for FOD detection.

24. The surveillance system according to claim 1, wherein a static camera detects respective locations of aircraft take off and landing on the runway such that a non-static camera is directed to first scan runway segments in the respective locations of aircraft landing or take off to reduce FOD detection time.

25. The surveillance system according to claim 1, wherein the image processing system applies temporal filtering to filter out rain clutter in runway scene images by recognising rain-like characteristics of rain motion clutter and based on the motion clutter due to rain occurring across the entire runway.

26. The surveillance system according to claim 1, wherein the image processing system applies temporal filtering to filter out snow clutter in runway scene images by recognising snow-like characteristics of snow motion clutter and based on the motion clutter due to snow occurring across the entire runway.

27. The surveillance system according to claim 1, wherein the image processing system makes use of markers or runway edge lights located along the longitudinal (horizontal) direction on the runway and on same vertical distance from the side of the runway for runway scene calibration to map pixels on the images of the runway to precise co-ordinates on the real-world co-ordinate frame (such as WGS84 or Airport Grid).

28. The surveillance system according to claim 1, wherein the image processing system makes use of two parallel horizontal runway lines on each side of a runway middle line and the runway middle line to derive two vertical pixel mapping ratios for runway scene calibration to map pixels on the images on the runway to precise co-ordinates on the real-world co-ordinate frame (such as WGS84 or Airport Grid).

29. The surveillance system according to claim 1, wherein the image processing system makes use of monoscopic vision and calibrated runway scene image captured by a monoscopic camera to determine the position and range of the FOD on the runway.

30. The surveillance system according to claim 1, wherein the system makes use of the FOD position and range determined by a static camera and a calibrated runway scene image to automatically control the non-static camera (such as a pan tilt zoom camera) to pan and/or tilt and/or zoom and/or focus onto a FOD to obtain telephoto images of the FOD with sufficient details to enable the verification of detected FOD or to filter a false alarm.

31. The surveillance system according to claim 1, wherein the system makes use of stereo vision using a pair of surveillance cameras to cover the same segment of the runway so that FOD range and position can be computed from the difference image obtained by comparing the two images as captured by the two cameras covering the same area of surveillance (field of view) on the runway.

32. A surveillance method for detecting a foreign object, debris, or damage (FOD) on a runway, the method comprising:

capturing images of the runway;

performing adaptive image processing of the images captured for detecting the FOD on the runway;

wherein the method is adaptively operable for FOD detection under both day and night ambient light conditions without assisted illumination such as infrared or laser illuminators.