Imaging and display system to aid helicopter landings in brownout conditions

• US 8,019,490 B2
• Filed: 09/17/2007
• Issued: 09/13/2011
• Est. Priority Date: 09/29/2006
• Status: Active Grant

• Current Assignee: Applied Minds LLC
• Original Assignee: Applied Minds LLC
• Inventors: Peterson, Michael Harvey, Hillis, W. Daniel, Ferren, Bran
• Primary Examiner(s): Tran; Khoi
• Assistant Examiner(s): FEI, JORDAN S
• Application Number:
  US11/856,557
• Publication Number:
  US 20090138138A1
• Time in Patent Office: 1,457 Days
• Field of Search: 701/3, 701/4, 701/16, 701/17, 340/945, 340/951, 345/419, 345/427, 345/653, 345/654, 345/672, 345/679, 345/680
• US Class Current: 701/3
• CPC Class Codes: G01C 23/00 Combined instruments indicating more than one navigational value, e.g. for aircraft; Combined measuring devices for measuring two or more variables of movement, e.g. distance, speed or acceleration
• CPC Class Codes: G05D 1/0676 specially adapted for landing

Abstract

An imaging and display system provides helicopter pilots with an unobstructed display of a landing area in a brownout or whiteout condition by capturing a high resolution image of the landing area prior to obscuration. Using inertial navigation information from the aircraft or an independent system, the system transforms the image to a desired viewpoint and overlays a representation of the helicopter'"'"'s current position relative to the landing area. The system thus greatly improves orientation and situational awareness, permitting safe and effective operation under zero visibility brownout conditions.

Claims

1. A system to enhance situational awareness in a vertical takeoff and landing (VTOL) aircraft, comprising:

an element for receiving data representing an unobscured two-dimensional image of a takeoff or landing zone before the formation of a brownout or whiteout condition, wherein said element for receiving data is selected from among a group of elements consisting of;

a digital camera;

an infrared visual imager;

an element for retrieving a stored digital camera image of said takeoff or landing zone; and

an element for retrieving digital camera imagery of said takeoff or landing zone transmitted from a source external to said aircraft;

an element for receiving navigational information;

a processing element programmed to generate one or more enhanced images representing said takeoff or landing zone as seen from an arbitrary position by overlaying said unobscured two-dimensional image onto a virtual geo-referenced 3D graphics space and by transforming said data representing said two-dimensional unobscured image based on said navigational information; and

an element for outputting said one or more enhanced images to a display device, wherein said display device displays one or more images depicting position of said aircraft relative to said landing zone.

2. The system of claim 1, wherein said display comprises either a standalone display or an existing aircraft display of sufficient resolution.

3. The system of claim 1, wherein said arbitrary position comprises said aircraft'"'"'s current position.

4. The system of claim 1, further comprising at least one additional sensor for any of:

enhancing accuracy of positioning for any element from said group of elements;

characterizing slope and other obstacles; and

detecting formation of a dust cloud to begin or stop image acquisition.

5. The system of claim 4, wherein said at least one additional sensor comprises any of:

one or more laser rangefinders aligned with said visual sensor to furnish slant range to a landing area;

a downward pointing laser rangefinder to give altitude information; and

a radar altimeter to give altitude information.

6. The system of claim 1, further comprising:

a triggering element to activate image acquisition by said system, wherein said triggering element is adjustable to trigger image acquisition at one or more of a specific airspeed, a specific altitude above ground level and proximity from a designated landing zone.

7. The system of claim 1, wherein image acquisition is manually triggered.

8. The system of claim 1, wherein image acquisition is either triggered or suspended by detection of formation of a brownout or a whiteout condition.

9. The system of claim 1, further comprising:

a component for directly measuring slope at a landing zone, wherein said component for directly measuring slope comprises one of;

means for projecting a pattern of tightly-focused light onto a landing zone, where distortion of a reflected pattern is analyzed by said processing element to provide 3D information about terrain and large obstacles;

a multi-beam laser wherein slope of a landing zone can be calculated by measuring return time of the multiple lasers;

a 3D imaging apparatus comprising two cameras separated by a sufficient baseline to characterize the landing zone in 3D; and

means for conducting pixel-by-pixel optical flow analysis to provide information on contours.

10. The system of claim 1, further comprising:

a navigation system transmitting said navigational information to said element for receiving navigational information, wherein said navigational system comprises one or both of;

said aircraft'"'"'s navigation system, wherein said navigational information is transmitted to said processing element via a data signal; and

a navigation system incorporated within said system for imaging and display, said incorporated navigation system including one or both of a GPS (global positioning system) sensor and an inertial measurement device.

11. The system of claim 10, further comprising:

a navigation database, wherein said navigation database stores navigation data regarding at least said aircraft'"'"'s flight plan, coordinates of crew-designated landing zones and terrain elevations.

12. The system of claim 1, wherein said navigational information comprises information regarding position and attitude of said aircraft, said navigation information including one or more of:

latitude and longitude;

altitude above ground;

aircraft attitude, including any of pitch, roll and heading;

distance to a landing zone; and

camera head pan and tilt angles, wherein pan and tilt axes permit a sensor to track a geo-referenced ground location or to remain aligned to a flight direction.

13. The system of claim 1, further comprising:

an intensifier to amplify light entering an image-capture device.

14. The system of claim 1, further comprising:

an illuminator for providing additional light for capturing image data in low-light situations.

15. The system of claim 14, wherein said illuminator comprises:

a pulsed illuminator synchronized to a visual sensor, wherein said illuminator has sufficient power to fully light said landing or takeoff zone, said illuminator further comprising a parabolic reflector to direct the light to the desired illumination angle and an IR-pass filter covering said reflector to prevent escape of visible light.

16. The system of claim 1, further comprising:

a steerable gimbal, said steerable gimbal housing said element for receiving data.

17. The system of claim 1, wherein said processing element programmed to generate one or more enhanced images representing said takeoff or landing zone as seen from an arbitrary position comprises means for:

visually enhancing said two-dimensional image data to improve contrast and detail; and

transforming said two-dimensional image data to a desired viewpoint.

18. A system to enhance situational awareness in a vertical takeoff and landing (VTOL) aircraft in a brownout or whiteout environment, comprising:

an imager for acquiring a two-dimensional digital image of a landing area before the formation of a brownout or whiteout condition, wherein said imager is selected from among a group of imagers consisting of;

a digital camera;

an infrared visual imager;

an element for retrieving a stored digital camera image of said takeoff or landing zone; and

an element for retrieving digital camera imagery of said takeoff or landing zone transmitted from a source external to said aircraft;

a system for acquiring, storing and transmitting navigational information;

a display device in view of one or more of the flight crew; and

a computer that inputs said navigational information, inputs and processes said two-dimensional digital image based on said navigational information by overlaying said two-dimensional image onto a virtual geo-referenced 3D graphics space and by transforming said data representing said two-dimensional image based on said navigational information, and outputs said processed image to said display device;

wherein said display device displays an image depicting said aircraft position relative to said landing area.

19. A method for enhancing situational awareness in a vertical takeoff and landing (VTOL) aircraft, comprising the steps of:

acquiring data representing an unobscured two-dimensional image of a takeoff or landing zone before the formation of a brownout or whiteout condition, wherein the step of acquiring is performed by a steps chosen from among a group of steps consisting of acquiring data representing an unobscured two-dimensional image from a digital camera;

acquiring data representing an unobscured two-dimensional image from an infrared visual imager;

acquiring data representing an unobscured two-dimensional image from an element for retrieving a stored digital camera image of said takeoff or landing zone; and

acquiring data representing an unobscured two-dimensional image from an element for retrieving digital camera imagery of said takeoff or landing zone transmitted from a source external to said aircraft;

acquiring navigational information;

generating one or more enhanced images representing said takeoff or landing zone as seen from an arbitrary position by overlaying said unobscured two-dimensional image onto a virtual geo-referenced 3D graphics space and by transforming said data representing said unobscured image based on said navigational information; and

displaying to a crew of said aircraft at least one of said one or more enhanced images representing said aircraft'"'"'s flight path.