System and method for image motion compensation of a CCD image sensor

• US 6,108,032 A
• Filed: 10/23/1997
• Issued: 08/22/2000
• Est. Priority Date: 11/05/1996
• Status: Expired due to Term

• Current Assignee: BAE Systems Information and Electronic Systems Integration Incorporated (BAE Systems Plc)
• Original Assignee: Lockheed Martin Fairchild Systems (Martin Marietta Corporation)
• Inventors: Hoagland, Kenneth A.
• Primary Examiner(s): Kelley, Chris S.
• Assistant Examiner(s): VO, TUNG T
• Application Number:
  US08/956,538
• Time in Patent Office: 1,034 Days
• Field of Search: 348/143, 348/144, 348/150, 348/151, 348/161, 348/162, 348/169, 348/170, 348/171, 348/208, 348/295, 348/312, 348/317, 382/107, 358/213.11, 358/103, 250/206.1, 250/330-334
• US Class Current: 348/144
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
• CPC Class Codes: H04N 23/6842 by controlling the scanning position, e.g. windowing
• CPC Class Codes: H04N 25/50 Control of the SSIS exposure
• CPC Class Codes: H04N 25/53 Control of the integration time
• CPC Class Codes: H04N 25/711 Time delay and integration [TDI] registers; TDI shift registers
• CPC Class Codes: H04N 25/745 Circuitry for generating timing or clock signals

Abstract

An electronic system and method for minimizing the loss of image sharpness caused by image motion during the exposure interval of a camera using electro-optical area array image sensors is provided. Differential speed clocking is performed using a set of parallel shift registers to achieve improved image motion compensation ("IMC") performance. A large number of rows and columns contained in a CCD array are arranged in a variable number of row segments. In situations where IMC is needed, a variable frequency clocking method can be applied to these row segments in the CCD array. This method of parallel shift clocking provides for IMC in a reconnaissance system in low light conditions where narrow shutter widths are not practical. An aerial reconnaissance system user utilizing this IMC method can perform forward oblique reconnaissance missions in a straightforward manner. Complexity in CCD design and processing is reduced with this approach, which is utilized in either a small or large array format CCD.

Claims

1. A focal plane array for use in an electro-optical reconnaissance system, wherein the focal plane array is configured to detect an image of a scene and to convert the image into an electronic charge representation of the image, and wherein the focal plane array is operated to compensate for a motion of objects contained in the scene, comprising:

a main format area, comprising a plurality of photo-sensitive cells arranged in rows and columns, wherein said rows are further arranged in a number of row segments, each row segment corresponding to a separate clocking section;

a focal plane array control unit that supplies a plurality of clocking signals to said number of row segments, each said clocking signal being applied to a respective row segment to transfer charge between rows in the respective row segment at a transfer rate determined by a frequency of the applied clocking signal wherein each said clocking signal has a predetermined synchronous timing relationship with each of the other clocking signals to prevent a mismatch in charge transfer between at least one pair of adjacent row segments having different transfer rates; and

a serial register coupled to said main format area to receive the electronic charge representation from said row segments.

2. The focal plane array of claim 1, wherein

said focal plane array control unit supplies the plurality of clocking signals to said main format area to perform image motion compensation, wherein said plurality of clocking signals includes a set of parallel shift clocking signals and a serial register clocking signal, and wherein said focal plane array control unit includes

a master timing signal generator to generate a plurality of timing signals, and

a focal plane array driver circuit to receive said plurality of timing signals from said master timing signal generator and to drive the focal plane array with said set of parallel shift clocking signals;

and

a camera control unit coupled to said master timing signal generator, wherein said mast timing signal generator receives data signals from said camera control unit.

3. The focal plane array of claim 2, wherein said focal plane array driver circuit comprises:

a first plurality of signal outputs corresponding to said number of row segments, wherein each of said row segments receives one of said set of parallel clocking signals; and

a second plurality of signal outputs corresponding to said serial register, wherein said serial register receives a serial register clocking signal.

4. The focal plane array of claim 2, wherein the electro-optical reconnaissance system is mounted on a vehicle capable of forward motion, further comprising:

focussing means, coupled to the focal plane array, for focussing a portion of the scene onto said focal plane array, wherein said portion of the scene is defined by a field of view of said focussing means;

a signal processor coupled to the focal plane array to convert the electronic charge representation into a digital image data signal, wherein said digital image data signal is a digital representation of the image; and

a plurality of sensors to provide mission parameters to said camera control unit, wherein said camera control unit processes said mission parameters and determines an image motion compensation clocking signal corresponding to said mission parameters and the motion objects contained in the scene.

5. The focal plane array of claim 4, wherein the electro-optical reconnaissance system further comprises:

a control console to receive mission programming information from an operator of said vehicle and to send said mission programming information to said camera control unit.

6. The focal plane array of claim 4, wherein the electro-optical reconnaissance system further comprises:

a memory unit, coupled to said camera control unit, to store a set of image motion rates for a particular set of mission parameters; and

a programmable logic element coupled to said focal plane array driver circuit to determine an integer number of cycles for each of said set of parallel shift clocking signals performing parallel shift clocking on said row segments, wherein said set of parallel shift clocking signals corresponds to said number of row segments, and wherein said integer number of cycles corresponds to said row segment being clocked.

7. The focal plane array of claim 6, wherein said main format area further comprises:

a first row segment viewing a portion of the scene farthest from the focal plane array, wherein a parallel shift clocking signal with a fewest number of cycles clocks said first row segment; and

a last row segment viewing a portion of the scene closest to the focal plane array, wherein a parallel shift clocking signal with a greatest number of cycles clocks said last row segment, said last row segment being closest to said serial register, wherein said integer number of cycles corresponds to mission parameters processed by said camera control unit, wherein said number of cycles increases corresponding to the motion of objects contained in said portion of the scene focussed on said row segment.

8. The focal plane array of claim 4, wherein said mission parameters comprise altitude, velocity, depression angle, field of view, pitch, and roll.

9. The focal plane array of claim 4, wherein the electro-optical reconnaissance system further comprises:

a shutter, coupled to said camera control unit, that opens to expose the focal plane array to the scene, and closes to allow readout of said main format area;

a camera mount housing the focal plane array; and

a signal recording device, coupled to said signal processor to record said digital image data signal.

10. The focal plane array of claim 9, wherein an orientation of said camera mount is a forward oblique mode.

11. The focal plane array of claim 9, wherein said shutter has a moveable exposure slit proximate to said focal plane array.

12. The focal plane array of claim 11, wherein an orientation of said camera mount is a side oblique mode.

13. The focal plane array of claim 12, wherein a shifting of said row segments corresponds to the motion of objects imaged onto each of said row segments, wherein each of said row segments shifts at a same average rate as a projection of the objects move across said row segment.

14. The focal plane array of claim 2, wherein said predetermined number of clocking signals are received by the focal plane array on either side of the focal plane array.

15. The focal plane array of claim 1, wherein said main format area comprises a charge-coupled device (CCD) of at least 9216 columns and 9216 rows.

16. The focal plane array of claim 1, wherein said main format area is divided into an image area and a storage area, wherein said storage area stores the electronic charge representation transferred from said image area, and wherein said storage area is shielded from exposure to the scene.

17. The focal plane array of claim 16, wherein said storage area comprises a plurality of cells arranged in rows and columns, and wherein said storage area is approximately equal in size to said imaging area.

18. The focal plane array of claim 17 having a frame rate of approximately 30 images per second, wherein the electronic charge representations of said scene are shifted from said row segments of said image area to said storage area in a time of less than approximately 1/300 second, and wherein the electronic charge representations stored in all of said rows of said storage area are read out in less than 1/30 second.

19. The focal plane array of claim 18, wherein said image area comprises CCD array having at least 500 pixel columns and 500 pixel rows.

20. The focal plane array of claim 19, wherein the electro-optical reconnaissance system further comprises:

a camera control unit, coupled to the focal plane array, to update a set of mission control parameters;

a shutter, coupled to said camera control unit, that opens to expose the focal plane array to the scene, and closes to allow readout of said main format area;

a camera mount to house the focal plane array;

a focal plane array control unit to provide a plurality of clocking signals to said image area to perform image motion compensation;

a signal processor to process said charge representations from said storage area into an image data signal; and

display means to provide a real-time display of said image data signal, wherein an orientation of said camera mount is forward oblique, and wherein said display means comprise a video monitor located in a cockpit of an aircraft.

21. The focal plane array of claim 1, wherein a first one of the plurality of clocking signals effects charge transfers between rows in an associated one of the plurality of row segments during a row-shift time interval, and

wherein each said clocking signal includes an integer number of clock cycles that collectively start and end at clock cycle transitions aligned respectively with a start and an end of the row-shift time interval, to thereby prevent the mismatch in charge transfer between row segments clocked at different row transfer rates.

22. The focal plane array of claim 1, wherein each row segment includes opposing ends, each clocking signal being applied concurrently to the opposing ends of the associated row segment to thereby propagate from the opposing ends, along the row segment, and toward a center portion of the row segment.

23. The method of claim 2, wherein said at least one serial register comprises a plurality of serial registers spaced apart along an edge portion of the CCD array and coupled to said CCD main format area for reading image pixel information from the CCD array, each said serial register receiving row charges from at least one of said row segments associated with said serial register, wherein said clocking signals are applied to said row segments such that clocking signals having lowest and highest frequencies among the plurality of clocking signals are respectively applied to row segments spaced furthest from and closest to said plurality of serial registers.

24. A focal plane array for use in an electro-optical reconnaissance system, wherein the focal plane array is configured to detect an image of a scene and to convert the image into electronic charge representation of the image, and wherein the focal plane array is operated compensate for motion of objects contained in the scene, comprising:

a main format area having a plurality of photo-sensitive cells arranged in rows and columns, wherein said rows are arranged in a number of row segments, each row segment corresponding to a separate clocking section, wherein said plurality of photo-sensitive cells are further arranged into a predetermined number of column segments, wherein each of said column segments corresponds to a separate parallel clocking section and each of said column segments is electronically isolated from adjacent column segments;

a focal plane array control unit that supplies a plurality of clocking signals to said number of row segments, each said clocking signal being applied to a respective row segment to transfer charge between rows in the respective row segment at a transfer rate determined by a frequency of the applied clocking signal, wherein each said clocking signal has a predetermined synchronous timing relationship with each of the other clocking signals to prevent a mismatch in charge transfer between at least one pair of adjacent row segments having different transfer rates; and

at least one serial register coupled to said main format area to receive the electronic charge representation of the image from said main format area.

25. The focal plane array of claim 24, wherein the electro-optical reconnaissance system further comprises:

a focal plane array control unit to provide a plurality of clocking signals to said main format area to perform image motion compensation, wherein said focal plane array control unit includes

a master timing signal generator to generate a plurality of timing signals, and

a focal plane array driver circuit to receive said plurality of timing signals from said master timing signal generator and to drive the focal plane array with a set of parallel clocking signals;

focussing means, coupled to the focal plane array, for focussing a portion of the scene onto said focal plane array, wherein said portion of the scene is defined by a field of view of said focussing means;

a signal processor coupled to the focal plane array to convert the electronic charge representation into a digital image data signal, wherein said digital image data signal is a digital representation of the image;

a camera control unit coupled to said focal plane array control unit;

a plurality of sensors to provide mission parameters to said camera control unit, wherein said camera control unit processes said mission parameters and calculates said image motion compensation clocking signals corresponding to said processed mission parameters; and

a camera mount housing the focal plane array, wherein a preferred orientation of the electro-optical reconnaissance system is a side oblique mode.

26. The focal plane array of claim 24, wherein a first one of the plurality of clocking signals effects charge transfers between rows in an associated one of the plurality of row segments during a row-shift time interval, and

wherein each said clocking signal includes an integer number of clock cycles that collectively start and end at clock cycle transitions aligned respectively with a start and an end of the row-shift time interval, to thereby prevent the mismatch in charge transfer between row segments clocked at different row transfer rates.

27. A method of image motion compensation for an aerial electro-optical reconnaissance system, comprising the steps of:

(1) imaging a scene onto a focal plane array, wherein the scene contains a plurality of objects, and wherein the focal plane array includes a main format area comprising a plurality of photo-sensitive cells arranged in rows and columns and at least one serial register, wherein the rows are arranged into a number of row segments, each row segment corresponding to a separate clocking section;

(2) generating a predetermined number of synchronous, separate clocking signals to perform parallel shift clocking of the row segments; and

(3) transferring charges representing a portion of the scene stored in each row of a particular row segment at a charge transfer rate corresponding to a motion rate that the object from the portion of the scene appear to move across that particular row segment, wherein said transferring step includes the steps of;

transferring charges between rows in each row segment at a charge transfer rate determined by a frequency of one of said separate clocking signals respectively associated with said row segment, and

synchronizing a timing relationship between said clocking signals to prevent a mismatch in charge transfer between at least one pair of adjacent row segments having different charge transfer rates.

28. The method of claim 27, wherein step (2) comprises the steps of:

(a) measuring a set of mission parameters corresponding to the scene being imaged by the reconnaissance system;

(b) processing the mission parameter information, wherein said processed mission parameter information corresponds to a rate of motion of the objects contained in the scene;

(c) generating a number of parallel shift clocking signals corresponding to the processed mission parameter information and to the number of row segments, wherein each of the parallel shift clocking signals includes an integer number of cycles; and

(d) increasing the number of cycles comprising the parallel shift clocking signal corresponding to the row segment being clocked, wherein a clocking signal with a fewest number of cycles clocks a first row segment and a clocking signal with a greatest number of cycles clocks a last row segment, wherein the first row segment corresponds to the portion of the scene farthest from the focal plane array, and wherein the last row segment corresponds to the portion of the scene closest to the focal plane array.

29. The method of claim 27, further comprising the step of:

recording a digital version of the electronic charge representation of the scene.

30. The method of claim 27, further comprising the steps of

transferring charge between rows in a first one of said row segments during a row-shift time interval using an associated one of said clocking signals;

generating an integer number of clock cycles representing each of said clocking signals; and

aligning a start and an end of the integer number of clock cycles with a start and an end of said row-shift time interval, to thereby prevent the mismatch in charge transfer between row segments clocked at different row transfer rates.

31. A method of image motion compensation for an aerial electro-optical reconnaissance system, comprising the steps of:

(1) imaging a scene onto a focal plane array, wherein the scene contains a plurality of objects, and wherein the focal plane array includes a main format area comprising a plurality of photo-sensitive cells arranged in rows and columns, wherein the rows are arranged into a number of row segments and the columns are arranged into a number of column segments, wherein a region comprising at least one row segment and at least one column segment is called a mosaic region, each mosaic region corresponding to a separate clocking section;

(2) generating a predetermined number of synchronous separate clocking signals to perform parallel shift clocking of the mosaic regions; and

(3) transferring charges representing a portion of the scene stored in a particular mosaic region at a same rate as an object from the scene appears to move across the mosaic region, wherein the charges are read out by a serial register and wherein said transferring step includes the steps of;

transferring charges between rows in each row segment at a charge transfer rate determined by a frequency of one of said separate clocking signals respectively associated with said row segment, and

synchronizing a timing relationship between said clocking signals to prevent a mismatch in charge transfer between at least one pair of adjacent row segments having different charge transfer rates.

32. The method of claim 31, further comprising the steps of

transferring charge between rows in a first one of said row segments during a row-shift time interval using an associated one of said clocking signals;

generating an integer number of clock cycles representing each of said clocking signals; and

aligning a start and an end of the integer number of clock cycles with a start and an end of said row-shift time interval, to thereby prevent the mismatch in charge transfer between row segments clocked at different row transfer rates.

33. A method of transferring charges stored in a charge coupled device (CCD) mounted in a moving vehicle to perform image motion compensation, comprising the steps of:

(1) imaging a moving scene onto the CCD, wherein the CCD includes a main format area comprising a plurality of photo-sensitive cells arranged in rows and columns and at least one serial register;

(2) arranging the rows into a predetermined number of row segments; and

(3) clocking each row segment with a parallel shift clocking signal such that the rows of a particular row segment shift at a substantially similar rate as objects contained in the moving scene appear to move across a mid point row of the particular row segment, wherein said clocking step includes the steps of;

transferring charges between rows in each row segment at a charge transfer rate determined by a frequency of one of said separate clocking signals respectively associated with said row segment, and

synchronizing a timing relationship between said clocking signals to prevent a mismatch in charge transfer between at least one pair of adjacent row segments having different charge transfer rates.

34. The method of claim 33, wherein step (3) comprises the step of:

clocking all row segments in synchronism with the image motion.

35. The method of claim 34, wherein step (3) further comprises the step of:

clocking each row segment at a constant rate corresponding to an average value of the image motion across each row of that row segment.

36. The method of claim 33, further comprising the steps of:

transferring charge between rows in a first one of said row segments during a row-shift time interval using an associated one of said clocking signals;

generating an integer number of clock cycles representing each of said clocking signals; and

aligning a start and an end of the integer number of clock cycles with a start and an end of said row-shift time interval, to thereby prevent the mismatch in charge transfer between row segments clocked at different row transfer rates.