Exterior aircraft vision system using a helmet-mounted display

• US 7,180,476 B1
• Filed: 06/30/2000
• Issued: 02/20/2007
• Est. Priority Date: 06/30/1999
• Status: Expired due to Term

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Leitch, Roderick, Schroeder, Steve E., Guell, Jeff J., Marstad, Norman, DeVogel, Michael
• Primary Examiner(s): LAO, LUNYI
• Application Number:
  US09/608,234
• Time in Patent Office: 2,426 Days
• Field of Search: 345 7- 9, 340/980, 348/53, 348115-117, 348/208, 348/262, 348563-565, 348/143, 348/144, 348/147, 348/148, 348/207.99, 348/208.6, 348/48, 359/464, 359/643, 359/648, 359630-631, 701/14, 250/334, 349/11
• US Class Current: 345/7
• CPC Class Codes: G01S 19/15 Aircraft landing systems
• CPC Class Codes: G01S 5/163 Determination of attitude using inertial means G01C9/00; control of attitude G05D1/49
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

An improved multiple-sensor vision system for use on vehicles or aircraft. The system utilizes a plurality of fixed sensors, such as infrared sensors, that provide signals that are recorded on tape and/or stored in memory. A processor digitally samples the stored images and provides output to a helmet-mounted display. Desirably, the sensors are arrayed in series such that their images may be juxtaposed and blended to provide a wider field-of-view image. A tracking system desirably monitors the head position of the operator, which position is then used to select various images from the processor. In this way, the operator can select various views by simply looking in that direction. In one embodiment, the array of sensors is forward-looking and positioned close to the head position of the operator to minimize parallax issues. In addition to forward-looking sensors, other sensors mounted around the vehicle/aircraft may provide rearward, hemispherical, or complete spherical coverage. Sensors that monitor the status of various instruments on board the vehicle can also provide input to the helmet-mounted display, including a moving map tile that is displayed below or to the side of the main image.

Claims

1. A conformal enhanced vision system for mobile vehicles, comprising:

an array of vision sensors mounted on a vehicle, each sensor being capable of generating image signals;

a recording medium for storing the image signals from the array of vision sensors;

a processor for sampling the stored image signals from the recording medium and producing an output signal therefrom;

a display connected to receive the output signal from the processor and superimpose images generated by the output signal on a see-through visor which also selectively permits an operator to view real images disposed in front of said visor; and

a tracking system associated with the display that monitors the movement of the head of the operator and transmits a tracking signal to the processor, the processor producing the output signal based on feedback from the tracking signal;

wherein both the images generated by the output signal and the real images are in conformity with one another, to create a seamless effect for the operator and wherein images generated by the output signal include an exterior view relative to the vehicle, a portion of the exterior view being partially blocked by the vehicle in the real images viewable by the operator, but the images generated by the output signal of the exterior view being displayable superimposed and in conformity with the real images on the see-through visor and including the portion of the exterior view that is partially blocked by the vehicle in the real images.

2. The system of claim 1, wherein the vehicle is an aircraft, and wherein the array of vision sensors is mounted close to the cockpit area such that the image signals originate from a location proximate the wearer of the display.

3. The system of claim 2, wherein the array of vision sensors is mounted in the upper radome area of the nose of the aircraft.

4. The system of claim 1, wherein the vision sensors are infrared sensors, and wherein one of the infrared sensors has higher resolution than the others and is forward-looking.

5. The system of claim 4, wherein the vehicle is an aircraft and wherein the higher resolution infrared sensor is located in the center of the array of vision sensors for a group including at least one of takeoff, air-to-air refueling and final approach and landing situations.

6. The system of claim 1, wherein the vehicle is an aircraft, and wherein the array of vision sensors is mounted in the nose area and has a downwardly-looking elevational field-of-view.

7. The system of claim 6, wherein the array of vision sensors has an elevational field-of-view of approximately 24°

.

8. The system of claim 1, wherein the vehicle is an aircraft, and wherein the array of vision sensors is mounted in the nose area and has a field-of-view straddling the horizontal horizon.

9. The system of claim 8, wherein the array of vision sensors has an elevational field-of-view of approximately 51°

.

10. The system of claim 1, wherein the array of vision sensors provides at least a hemispherical field-of-view.

11. The system of claim 10, wherein the array vision sensors provides a spherical field-of-view.

12. The system of claim 1, wherein at least one of the vision sensors additionally provides an infrared search and track function.

13. The system of claim 1, further including at least one other sensor separate from the array of vision sensors that provides a separate signal to the processor that then combines it with the output signal.

14. The system of claim 13, wherein the one other sensor is a vision sensor oriented differently than the array of vision sensors.

15. The system of claim 14, wherein the array of vision sensors is forward-looking, and wherein the one other vision sensor is rearward-looking.

16. The system of claim 14, wherein the array of vision sensors provide a series of adjacent image signals that are combined by the processor into a wide field-of-view output signal, and wherein the signal from the one other vision sensor is overlaid on the wide field-of-view output signal as a picture-in-picture image.

17. The system of claim 13, wherein the one other sensor generates a real-time map signal that is combined by the processor into the output signal and displayed on the display outside an image produced by the array of vision sensors.

18. The system of claim 13, wherein the one other sensor monitors an operational parameter of the vehicle and generates a corresponding signal.

19. The system of claim 18, wherein the operational parameter of the vehicle is selected from the group consisting of:

speed;

altitude;

attitude; and

engine status.

20. The system of claim 1, and further including a manual input device to the processor, wherein the output signal may be manually disabled in select areas on the helmet-mounted display.

21. The system of claim 1, wherein said display comprises a helmet-mounted display.

22. The system as recited in claim 1, wherein each vision sensor is fixedly mounted on the vehicle and comprises a non-turret mounted immovable sensor.

23. A conformal enhanced vision system for mobile vehicles, comprising:

an array of vision sensors mounted on a vehicle, each sensor being capable of generating image signals;

a processor for producing an output signal from a selected sampling of said image signals;

a display connected to receive the output signal from the processor and superimpose images generated by the output signal on a see-through screen which also selectively permits an operator to view actual images disposed in front of said screen; and

a controller for controlling an intensity of light permitted to pass through said screen and for alternatively selectively disabling selected regions of said screen so that light cannot pass through those selected regions;

wherein both the images generated by the output signal and the actual images are in conformity with one another, to create a seamless effect for the operator and wherein images generated by the output signal include an exterior view relative to the vehicle, a portion of the exterior view being partially blocked by the vehicle in the actual images viewable by the operator, but the images generated by the output signal of the exterior view being displayable superimposed and in conformity with the actual images on the see-through screen and including the portion of the exterior view that is partially blocked by the vehicle in the actual images.

24. The system of claim 23, wherein said display comprises a helmet-mounted display, and said screen comprises a helmet visor.

25. The system of claim 24, and further comprising a tracking system associated with the helmet-mounted display that monitors the movement of the head of the wearer of the display and transmits a tracking signal to the processor, the processor producing the output signal based on feedback from the tracking signal.

26. The system of claim 25, wherein said tracking system comprises an emitter fixedly mounted on a helmet of the operator and a single detector disposed in spaced relation to said emitter.

27. The system of claim 23, wherein said controller includes a manual override capability so that the operator can selectively manually control and select particular output images from various ones of said sensors.

28. The system as recited in claim 23, wherein said vision sensors are immovably mounted on the vehicle.

29. An enhanced vision system for an aircraft, comprising:

an array of sensors mounted on the aircraft;

an imaging system to process signals from each sensor of the array of sensors to produce an output signal;

a display to superimpose images generable from the output signal on an actual real-time scene being viewed by the operator, wherein the images generable by the output signal are in conformity with the actual real-time scene and wherein images generated by the output signal include an exterior view relative to the aircraft, a portion of the exterior view being partially blocked by the aircraft in the actual real-time scene being viewed by the operator, but the images generable by the output signal including the exterior view being displayable superimposed and in conformity with the real-time actual scene and including the portion of the exterior view that is partially blocked by the vehicle in the real-time actual scene.

30. The system of claim 29, further comprising a tracking system to monitor real-time movement of the operator'"'"'s head to alter which sensors of the array of sensors are sampled to generate the exterior view relative to the aircraft.

31. The system of claim 30, wherein a number of images produced by the sensors are scannable by the operator moving his head.

32. The system of claim 29, wherein the array of sensors comprise a sensor that has a higher resolution than the other sensors and is forward-looking for use in performing a group of operations including at least one of takeoff, air-to-air refueling, and final approach and landing.

33. The system of claim 29, wherein the images generable by the output signal comprise an enhanced exterior view relative to the aircraft that is substantially more extensive than that provided through a cockpit window of the aircraft.

34. The system of claim 29, wherein the images generable by the output signal are arrangeable in tiles that can be juxtaposed and viewed on the display, and wherein the system further comprises a picture-in-picture tile positionable relative to the other tiles to present an image generable by a sensor on another aircraft on the display.

35. The system of claim 29, further comprising a moving map tile presentable on the display.

36. The system of claim 35, wherein the moving map tile is adapted to be supplied information from at least one of a group comprising a global positioning system (GPS), an airborne warning and control system (AWACS), and other aircraft.