Autonomous electro-optical framing camera system with constant ground resolution, unmanned airborne vehicle therefor, and methods of use

• US 6,130,705 A
• Filed: 07/10/1998
• Issued: 10/10/2000
• Est. Priority Date: 07/10/1998
• Status: Expired due to Term

• Current Assignee: Goodrich Corporation (Rtx Corporation)
• Original Assignee: Recon/Optical, Inc. (Bourns Incorporated)
• Inventors: James, Brian, Lund, John, Beran, Stephen R., Quinn, James P., Lareau, Andre G.
• Primary Examiner(s): Rao, Andy
• Assistant Examiner(s): An, Shawn S.
• Application Number:
  US09/113,392
• Time in Patent Office: 823 Days
• Field of Search: 348/144, 348/145-148, 358/109
• US Class Current: 348/144
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
• CPC Class Codes: G01C 3/08 Use of electric radiation detectors
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/682 Vibration or motion blur correction
• CPC Class Codes: H04N 23/687 by shifting the lens or sensor position

Abstract

An aerial reconnaissance system generates imagery of a scene that meets resolution or field of view objectives automatically and autonomously. In one embodiment, a passive method of automatically calculating range to the target from a sequence of airborne reconnaissance camera images is used. Range information is use for controlling the adjustment of a zoom lens to yield frame-to-frame target imagery that has a desired, e.g., constant, ground resolution or field of view at the center of the image despite rapid and significant aircraft altitude and attitude changes. Image to image digital correlation is used to determine the displacement of the target at the focal plane. Camera frame rate and aircraft INS/GPS information is used to accurately determine the frame to frame distance (baseline). The calculated range to target is then used to drive a zoom lens servo mechanism to the proper focal length to yield the desired resolution or field of view for the next image. The method may be performed based on parameters other than range, such as aircraft height and stand off distance.

Claims

1. An autonomous aerial reconnaissance camera system for imaging a scene of interest suitable for carriage by a reconnaissance vehicle, comprising:

(a) an electro-optical imaging detector array comprising a plurality of pixel elements arranged in rows and columns;

(b) a zoom lens system, comprising a zoom lens for imaging a scene external of said reconnaissance vehicle onto said imaging detector array and a drive system for adjusting the focal length of said zoom lens; and

(c) a camera control system for said imaging array and said zoom lens system, comprising;

i.) a memory storing information as to field of view, resolution, or image interpretability rating objectives for imagery to be generated by said imaging array; and

ii.) a camera control computer for receiving (a) aircraft navigation data generated by a navigation system on-board said reconnaissance vehicle and (b) either range-to-scene information or aircraft height and stand-off distance data for said scene of interest, and responsively generating commands for said imaging array,

iii.) said camera control system further comprising a zoom lens adjustment module generating commands for said drive system for said zoom lens to adjust said focal length of said zoom lens from said range-to-scene information or aircraft height and stand-off distance data and said information stored in said memory;

(d) said camera control computer, zoom lens, and electro-optical imaging detector array cooperating to generate an image meeting said field of view, resolution, or image interpretability rating objectives and enabling said field of view, resolution, or ground-resolvable distance objectives to be obtained without real-time operator involvement.

2. The system of claim 1, wherein camera control system further comprises a memory means for storing first and second successive images of said scene, and wherein said range-to-scene information is obtained by a sequential image processor circuit, said sequential image processor circuit executing a correlation algorithm on said first and second successive images to generate range information between said imaging detector array and said scene.

3. The system of claim 2, wherein said camera control computer, zoom lens and imaging array are operated to generate a third successive image of said scene, said zoom lens adjustment circuit operating said zoom lens to have a focal length such that said third image meets said field of view, resolution, or image interpretability rating objectives and enabling said field of view, resolution, or image interpretability rating objectives to be achieved without real-time operator involvement.

12. The system as claimed in one of claims 1 or 8, wherein said imaging array is organized into a plurality of column groups and wherein said camera control system further comprises a means for determining image motion rates in a column direction of said array for a plurality of locations in said array, and wherein said imaging array further comprises a means for transferring pixel information in said column groups in said column direction independently so as to compensate for different rates of image motion at said imaging sensor when said imaging sensor is oriented in a side-oblique orientation relative to said reconnaissance vehicle.

14. The system as claimed in one of claims 1 or 8, wherein said imaging array is selected from the group of imaging devices consisting of charge-coupled devices, charge-injection devices, C-MOS detectors, and photo-diode arrays.

16. The system as claimed in one of claims 1 or 8, wherein said array is organized into a plurality of cells, and wherein said camera control system further comprises a means, responsive to said navigation information, for transferring pixel information in each of said cells in both a row direction and a column direction in substantial synchronism with an image motion having both row and column components, so as to achieve image motion compensation for image motion having components in both said row direction and said column direction.

19. The system as claimed in one of claims 1 or 8, wherein said camera control system further comprises a target cuer processing images generated by said array to identify a target in the field of view of said array, said target associated with a predetermined field of view, resolution or image interpretability objective, and wherein said commands for said drive system of said zoom lens are determined such that the field of view, resolution or image interpretability objectives of imagery for said target is achieved.

4. A method of autonomously imaging a scene with a reconnaissance camera system in a manner so as to continuously generate successive images of said scene meeting field of view, resolution, or image interpretability rating objectives for said images, said aerial reconnaissance camera system installed on board a reconnaissance vehicle and comprising a electro-optical imaging array, a zoom lens, and a camera control system for said array, comprising the steps of:

a) generating first and second successive images of said scene with said electro-optical imaging array and storing said images in a sequential image processing circuit;

b) supplying vehicle information to said camera control system from a navigation system on board said aerial reconnaissance vehicle and responsively correlating said first and second successive images to generate range information between said imaging array and said scene;

c) retrieving information stored in said camera control system as to field of view, resolution, or image interpretability rating objectives for imagery to be generated by said imaging sensor;

d) processing said information as to field of view, resolution, or image interpretability rating objectives, together with said range information derived from correlation of said first and second images and navigation information for said reconnaissance vehicle, to generate commands for a zoom lens drive system so as to adjust the focal length of said zoom lens such that said field of view, resolution, or image interpretability rating objectives are achieved; and

e) exposing said imaging array to said scene to thereby generate a third successive image,

whereby said third successive image achieves said field of view, resolution, or ground-resolvable distance objectives without real-time operator involvement, and while compensating for changes in attitude and altitude of said reconnaissance vehicle between initial expected values of said attitude and attitude during exposure of said array to said scene and the actual altitude and attitude of said reconnaissance vehicle when said array is exposed to said scene.

13. The method as claims in one of claims 4 or 5, wherein said imaging array is organized into a plurality of column groups and wherein said camera control system further comprises a means for determining image motion rates in a column direction of said array for a plurality of locations in said array, and wherein said imaging array further comprises a means for transferring pixel information in said column groups in said column direction independently so as to compensate for different rates of image motion at said imaging sensor when said imaging sensor is oriented in a side-oblique orientation relative to said reconnaissance vehicle.

15. The method as claimed in one of claims 4 or 5, wherein said imaging array is selected from the group of imaging devices consisting of charge-coupled devices, charge-injection devices, C-MOS detectors, and photo-diode arrays.

17. The method as claimed in one of claims 4 or 5, wherein said array is organized into a plurality of cells, and wherein said camera control system further comprises a means, responsive to said navigation information, for transferring pixel information in each of said cells in both a row direction and a column direction in substantial synchronism with an image motion having both row and column components, so as to achieve image motion compensation for image motion having components in both said row direction and said column direction.

18. The method as claimed in one of claims 4 or 5, wherein said camera control system further comprises a target cuer processing images generated by said array to identify a target in the field of view of said array, said target associated with a predetermined field of view, resolution or image interpretability objective, and wherein said commands for said drive system of said zoom lens are determined such that the field of view, resolution or image interpretability objectives of imagery for said target is achieved.

5. A method of autonomously imaging a scene with a reconnaissance camera system in a manner so as to continuously generate successive images of said scene meeting field of view, resolution, or ground-resolvable distance objectives for said images, said aerial reconnaissance camera system installed on board a reconnaissance vehicle and comprising a electro-optical imaging array, a zoom lens, and a camera control system for said array, the method comprising the steps of:

(a) supplying navigation and range-to-target information to said camera control system;

(b) retrieving from a memory information as to field of view, resolution, or image interpretability rating objectives for imagery to be generated by said imaging array;

(c) processing said information as to field of view, resolution, or image interpretability rating objectives, together with said range information, to generate settings for said zoom lens so as have a focal length such that an image generated by said array will achieve said field of view, resolution, or image interpretability rating objectives;

(d) adjusting the focal length of said zoom in accordance with step (c), and

(e) exposing said imaging sensor to said scene;

whereby said image achieves said field of view, resolution, or ground-resolvable distance objectives without real-time operator involvement and compensates for changes in attitude and altitude of said reconnaissance vehicle between initial expected values of said attitude and attitude during exposure of said array to said scene and the actual altitude and attitude of said reconnaissance vehicle when said array is exposed to said scene.

6. The method of claim 5 wherein steps (a)-(e) are performed in a series of cycles, thereby generating successive images of said scene meeting said field of view, resolution, or image interpretability rating objectives while compensating for changes in vehicle attitude and altitude between the performance of said cycles.

7. The method of claim 5, wherein said range information is determined by a correlation algorithm applied to successive frames of imagery of said scene stored in a memory, whereby said range information is determined in a passive manner.

8. An unmanned aerial reconnaissance system capable of generating successive images of a scene meeting field of view, resolution, or image interpretability objectives without real-time operator involvement, comprising:

A. a reconnaissance vehicle having a navigation system and a means of propulsion capable of flying said vehicle in a reconnaissance mission without on-board human operation; and

B. an aerial reconnaissance camera system installed aboard said vehicle, comprising;

an electro-optical imaging array comprising an array of pixel elements arranged in a plurality of rows and columns;

a zoom lens system, comprising a zoom lens for imaging a scene external of said aircraft onto said imaging array and a drive system for adjusting the focal length of said zoom lens; and

a camera control system for said imaging array and said zoom lens system, comprising;

i.) a memory storing information as to field of view, resolution, or image interpretability rating objectives for imagery to be generated by said imaging array; and

ii.) a camera control computer for receiving (a) aircraft navigation generated by said navigation system and (b) either range-to-scene information from a source of range information on board said aircraft or aircraft stand-off distance to said scene and aircraft height data,

iii.) said camera control system further comprising a zoom lens adjustment circuit generating commands for said drive system for said zoom lens to adjust said focal length of said zoom lens in response to said either said range-to-scene information or said aircraft stand-off distance and aircraft height data, and navigation inputs received by said camera control computer and said information stored in said memory,

iv.) said commands for said drive system for said zoom lens accounting for changes in attitude and altitude of said reconnaissance vehicle between initial expected values of said attitude and altitude during exposure of said array to said scene and the actual altitude and attitude of said reconnaissance vehicle when said array is to be exposed to said scene;

said camera control computer, zoom lens adjustment circuit, zoom lens and electro-optical imaging array cooperating to generate an image from said imaging array meeting said field of view, resolution, or image interpretability rating objectives without regard to said changes in altitude or attitude of said reconnaissance vehicle,

whereby said image having said field of view, resolution, or ground-resolvable distance objectives may be obtained without real-time operator involvement either on-board said reconnaissance vehicle or through an up-link to said reconnaissance vehicle.

9. The system of claim 8, wherein said camera control system further comprises a memory means for storing first and second successive images of said scene generated by said imaging array, and wherein said range-to-scene information is obtained by a sequential image processor circuit, said sequential image processor circuit executing a correlation algorithm on said first and second successive images to generate range information between said imaging array and said scene.

10. The system of claim 9, wherein said camera control computer, zoom lens and imaging array are operated to generate a third successive image of said scene, said zoom lens adjustment circuit operating said zoom lens to have a focal length such that said third image meets said field of view, resolution, or image interpretability rating objectives without regard to said changes in altitude or attitude of said reconnaissance vehicle, and enabling said field of view, resolution, or image interpretability rating objectives to be achieved without real-time operator involvement.

11. The system of claim 8, wherein said reconnaissance vehicle comprises a long range UAV.

20. A method for imaging a scene of interest with an aerial reconnaissance camera system installed in an aircraft, said system comprising a camera control computer having a computer memory, an imaging array and a zoom lens, comprising the steps of:

a) storing in the camera control computer memory GRD (Ground Resolvable Distance), NIIRS (National Image Interpretability Rating Scale) or field of view objectives for imagery of the scene of interest;

b) deriving information as to the aircraft height and position relative to the scene of interest;

c) calculating a focal length setting for a zoom lens for the array to meet the GRD, NIIRS or field of view objectives for imagery of the scene from information as to the aircraft'"'"'s position relative to the scene obtained in step b) and the imagery objectives for said scene;

d) adjusting the focal length of the zoom lens in accordance with the calculations made in step c); and

(e) exposing the array to the scene to generate an image of the scene of interest, whereby the image meets the objectives for the imagery.

21. The method of claim 20, further comprising the steps of exposing the array to the scene of interest in a series of exposures, each successive image meeting the GRD, NIIRS or field of view objectives for the scene of interest.

22. The method of claim 21, further comprising the steps of processing said successive images by a target cuer, determining whether a particular class of target is present in said scene, and responsively adjusting the focal length of said zoom lens to meet GRD, NIIRS or field of view objectives associated with said particular class of target if said particular class of target is present in said scene.