Method and apparatus for deblurring an image

• US 9,063,344 B1
• Filed: 03/21/2012
• Issued: 06/23/2015
• Est. Priority Date: 03/21/2012
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Griffin, Steven, Gerwe, David R., Calef, Brandoch
• Primary Examiner(s): Mehta, Bhavesh
• Assistant Examiner(s): MOYER, ANDREW M
• Application Number:
  US13/425,747
• Time in Patent Office: 1,189 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G02B 27/40 Optical focusing aids
• CPC Class Codes: G02B 27/646 compensating for small deviations, e.g. due to vibration or shake movement of one or more optical elements for control of motion blur in cameras, projectors or printers G03B2205/0007; image stabilisation in cameras peculiar to the presence or use of an electronic image sensor H04N23/68
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20182 Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory

Abstract

A method and apparatus for reducing blur in an image generated by an imaging system is provided. A blur model for the image is identified with respect to an exposure time for the image using a sequence of reference images of an artificial reference point object generated during the exposure time for the image. A source of the artificial reference point object is inertially stabilized over the exposure time. The image is deconvolved with the blur model identified for the image to form a modified image having a desired reduction in blur relative to the image generated by the imaging system.

Claims

1. A method for reducing blur in an image generated by an imaging system comprising an optical system, an inertially stabilized source, and an array of image sensors, the method comprising:

focusing, using the optical system, light emanating from a scene onto the array of image sensors;

detecting, using the array of image sensors, the light to form the image;

generating an artificial reference point using the inertially stabilized source;

directing the artificial reference point towards the optical system such that the array of image sensors also detects the artificial reference point and not a reflection of the artificial reference point from the scene;

identifying a blur model for the image with respect to an exposure time for the image using a sequence of reference images of the artificial reference point object generated during the exposure time; and

deconvolving the image with the blur model to form a modified image having a reduction in blur relative to the image.

2. The method of claim 1 further comprising:

generating the sequence of reference images using a reference imaging system, wherein a relative movement between the reference imaging system and the imaging system is less than a relative movement between the imaging system and the scene.

3. The method of claim 2, wherein the step of generating the sequence of reference images using the reference imaging system comprises:

generating the sequence of reference images using the reference imaging system, wherein the imaging system comprises the array of image sensors and wherein the reference imaging system is formed by a portion of the array of image sensors.

4. The method of claim 2, wherein the step of generating the sequence of reference images using the reference imaging system comprises:

generating the sequence of reference images using the reference imaging system, wherein the reference imaging system is a separate imaging system associated with the imaging system in which the relative movement between the reference imaging system and the imaging system is less than a relative movement between the imaging system and the scene.

5. The method of claim 1 further comprising:

inertially stabilizing the source of the artificial reference point object over the exposure time for the image using an inertial system, wherein the source is a laser device configured to generate the artificial reference point object in the form of a laser beam.

6. The method of claim 1, wherein the step of generating the artificial reference point object using the source comprises:

directing a laser beam towards an optical system in the imaging system using a laser device inertially stabilized over the exposure time for the image generated by the imaging system.

7. The method of claim 1, wherein the step of identifying the blur model comprises:

identifying a reference point spread function for a reference image in the sequence of reference images, wherein the reference point spread function is used as the blur model for the image.

8. The method of claim 1, wherein the step of identifying the blur model comprises:

identifying a path of movement for the artificial reference point object during the exposure time for the image using a plurality of reference images in the sequence of reference images; and

identifying an average point spread function for the image using the path of movement.

9. The method of claim 1, wherein the step of identifying the blur model comprises:

identifying an average reference point spread function using a plurality of reference images in the sequence of reference images, wherein the average reference point spread function is used as the blur model for the image.

22. The method of claim 1, wherein a relative movement between the optical system and the array of image sensors is less than a relative movement between the imaging system and the scene.

10. An apparatus comprising:

an imaging system configured to generate an image over an exposure time, wherein the imaging system comprises;

an inertially stabilized source configured to generate an artificial reference point object during the exposure time;

an optical system configured to receive both the artificial reference point object from the inertially stabilized source without reflecting the artificial reference point from the scene, and also light from a scene;

an array of image sensors configured to generate the image using the light from the scene, and further configured to receive the artificial reference point;

an image processing system in communication with the imaging system and configured to identify a blur model for the image with respect to the exposure time using a sequence of reference images of the artificial reference point object and further configured to deconvolve the image with the blur model to form a modified image having a reduced blur relative to the image generated by the imaging system.

11. The apparatus of claim 10 further comprising:

an inertial system associated with the source, wherein the inertial system is configured to inertially stabilize the source over the exposure time for the image.

12. The apparatus of claim 11, wherein the inertial system is selected from one of an inertial reference unit and a passive isolation system.

13. The apparatus of claim 10 further comprising:

a reference imaging system.

14. The apparatus of claim 13, wherein the reference imaging system is a portion of the array of image sensors.

15. The apparatus of claim 13, wherein the reference imaging system is a separate imaging system associated with the imaging system in which relative movement between the reference imaging system and the imaging system is less than a relative movement between the imaging system and the scene.

16. The apparatus of claim 10, wherein the image processing system comprises:

a model generator configured to identify the blur model for the image with respect to the exposure time for the image using the sequence of reference images of the artificial reference point object; and

a deconvolver configured to deconvolve the image with the blur model to form the modified image using a deconvolution algorithm.

17. The apparatus of claim 16, wherein the model generator is configured to identify a reference point spread function for a reference image in the sequence of reference images and use the reference point spread function as the blur model for the image.

18. The apparatus of claim 16, wherein the model generator is configured to identify a path of movement for the artificial reference point object during the exposure time for the image using a plurality of reference images in the sequence of reference images, identify an average point spread function for the image using the path of movement, and use the average point spread function for the image as the blur model for the image.

19. The apparatus of claim 16, wherein the model generator is configured to identify an average reference point spread function using a plurality of reference images in the sequence of reference images and use the average reference point spread function as the blur model for the image.

20. The apparatus of claim 10, wherein the artificial reference point object appears to be present in a scene captured by the imaging system from a perspective of the imaging system.

21. The apparatus of claim 10, wherein the array of image sensors is an orthogonal transfer array.

23. The apparatus of claim 10, wherein a relative movement between the optical system and the array of image sensors is less than a relative movement between the imaging system and the scene.

24. The apparatus of claim 10, wherein the inertially stabilized source comprises an inertially stabilized laser.