COMPUTATIONAL IMAGING USING VARIABLE OPTICAL TRANSFER FUNCTION

• US 20130329094A1
• Filed: 12/30/2012
• Published: 12/12/2013
• Est. Priority Date: 12/19/2011
• Status: Active Grant

• Current Assignee: Ziva Corporation
• Original Assignee: Ziva Corporation
• Inventors: RANALLI, ELISEO, SAPERSTEIN, ROBERT ELLIOT
• Granted Patent: US 9,002,138 B2
• Time in Patent Office: Days
• US Class Current: 348/241
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/20024 Filtering details
• CPC Class Codes: G06T 2207/20056 Discrete and fast Fourier transform, [DFT, FFT]
• CPC Class Codes: G06T 5/10 using non-spatial domain filtering
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/75 by influencing optical camera components
• CPC Class Codes: H04N 23/81 for suppressing or minimising disturbance in the image signal generation
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 25/60 Noise processing, e.g. detecting, correcting, reducing or removing noise
• CPC Class Codes: H04N 25/615 involving a transfer function modelling the optical system, e.g. optical transfer function [OTF], phase transfer function [PhTF] or modulation transfer function [MTF]
• CPC Class Codes: H04N 5/265 Mixing
• CPC Class Codes: H04N 9/646 for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters

Abstract

In selected embodiments, improved image restoration is realized using extensions of Wiener filtering combined with multiple image captures acquired after simple, fast reconfigurations of an optical imaging system. These reconfigurations may yield distinct OTF responses for each capture. The optical imaging system may reduce fabrication cost, power consumption, and/or system weight/volume by correcting significant optical aberrations. The system may be configured to perform independent correction of fields within the total field-of-regard. The system may also be configured to perform independent correction of different spectral bands.

Claims

1. An imaging method, comprising:

capturing a plurality of M captured images of an object through an optical system, the optical system comprising a configurable optical component, the configurable optical component being capable of being configured in a plurality of configurations, wherein each captured image of the plurality of images is captured with the configurable optical component being in a different corresponding configuration of the plurality of configurations;

transforming each of the captured images using a selected spatial transform to obtain a corresponding transformed captured image, thereby obtaining a plurality of M transformed captured images;

weighting each of the transformed captured images by a weighting coefficient A_m computed using the formula

2. The imaging method of claim 1, wherein the selected transform is a spatial Fourier Transform, and the inverse transform is an inverse spatial Fourier Transform.

3. The imaging method of claim 2, wherein the configurable optical component comprises a deformable mirror and the method further comprises configuring the deformable mirror in the plurality of different configurations using one or more control parameters of the deformable mirror.

4. The imaging method of claim 2, wherein:

the configurable optical component comprises a deformable mirror;

each of the steps of capturing, transforming, weighting, summing, and inverse transforming is performed at least in part by at least one processor of at least one computer system; and

one or more zeroes of the optical transfer function of the optical system differ for at least two configurations of the plurality of configurations.

5. An imaging method, comprising:

capturing a plurality of M captured images of an object through an optical system, the optical system comprising a configurable optical component, the configurable optical component being capable of being configured in a plurality of configurations, wherein each captured image of the plurality of images is captured with the configurable optical component being in a different corresponding configuration of the plurality of configurations;

transforming each of the captured images using a selected spatial transform to obtain a corresponding transformed captured image, thereby obtaining a plurality of M transformed captured images;

weighting each of the transformed captured images by a weighting coefficient (1−

η

)×

B_m wherein η

is a predetermined constant less than 1 and greater than 0, and B_m is computed using the formula

6. The imaging method of claim 5, wherein the selected transform is a spatial Fourier Transform, and the inverse transform is an inverse spatial Fourier Transform.

7. The imaging method of claim 6, wherein the configurable optical component comprises a deformable mirror and the method further comprises configuring the deformable mirror in the plurality of different configurations using one or more control parameters of the deformable mirror.

8. The imaging method of claim 6, wherein:

the configurable optical component is a deformable mirror;

each of the steps of capturing, transforming, weighting, summing, and inverse transforming is performed at least in part by at least one processor of at least one computer system; and

one or more zeroes of the optical transfer function of the optical system differ for at least two configurations of the plurality of configurations.

9. An apparatus for processing images, the apparatus comprising:

an optical system comprising a configurable component, the configurable optical component being capable of being configured in a plurality of different configurations; and

at least one processor, wherein the at least one processor is coupled to the optical system to enable the at least one processor to control configuration of the configurable component and to to capture images in a focal plane of the optical system, and wherein the at least one processor is configured to execute program code instructions to cause the apparatus to perform steps comprising;

capturing a plurality of M captured images of an object through the optical system, wherein each captured image of the plurality of images is captured with the configurable optical component being in a different corresponding configuration of the plurality of configurations;

transforming each of the captured images using a selected spatial transform to obtain a corresponding transformed captured image, thereby obtaining a plurality of M transformed captured images;

weighting each of the transformed captured images by a weighting coefficient A_m computed using the formula

10. The apparatus of claim 9, wherein the selected transform is a spatial Fourier Transform, and the inverse transform is an inverse spatial Fourier Transform.

11. The apparatus of claim 10, wherein the configurable optical component comprises a deformable mirror configurable in the plurality of different configurations using at least one control parameter of the deformable mirror.

12. The apparatus of claim 10, wherein:

one or more zeroes of optical transfer function of the optical system differ for at least two configurations of the plurality of configurations.

13. An apparatus for processing images, the apparatus comprising:

an optical system comprising a configurable component, the configurable optical component being capable of being configured in a plurality of different configurations; and

at least one processor, wherein the at least one processor is coupled to the optical system to enable the at least one processor to control configuration of the configurable component and to capture images in a focal plane of the optical system, and wherein the at least one processor is configured to execute program code instructions to cause the apparatus to perform steps comprising;

capturing a plurality of M captured images of an object through an optical system, wherein each captured image of the plurality of images is captured with the configurable optical component being in a different corresponding configuration of the plurality of configurations;

transforming each of the captured images using a selected spatial transform to obtain a corresponding transformed captured image, thereby obtaining a plurality of M transformed captured images;

weighting each of the transformed captured images by a weighting coefficient (1−

η

)×

B_m wherein η

is a predetermined constant less than 1 and greater than 0, and B_m is computed using the formula

14. The apparatus of claim 13, wherein the selected transform is a spatial Fourier Transform, and the inverse transform is an inverse spatial Fourier Transform.

15. The apparatus of claim 14, wherein the configurable optical component comprises a deformable mirror configurable in the plurality of different configurations using at least one control parameter of the deformable mirror.

16. The apparatus of claim 14, wherein:

the configurable optical component comprises a deformable mirror; and

one or more zeroes of optical transfer function of the optical system differ for at least two configurations of the plurality of configurations.

17. An imaging method, the method comprising steps of:

capturing a plurality of M captured images of an object in a scene through an optical system, the optical system being configurable in a plurality of configurations, each configuration of the plurality of configurations corresponding to a different optical transfer function between the object and an image sensor of the optical system, wherein each captured image of the plurality of images is captured with a distinct optical transfer function between the object and the image sensor;

applying extended Wiener filtering to each of the captured images to obtain a corresponding filtered image, thereby obtaining a plurality of M filtered images; and

combining computationally the plurality of M filtered images to obtain an enhanced image.

18. An imaging method according to claim 17, further comprising at least one of displaying the enhanced image, storing the enhanced image, and transmitting the enhanced image.

19. An imaging method according to claim 18, further comprising configuring the optical system in the distinct configurations of the plurality of configurations.

20. An imaging method according to claim 19, wherein the step of configuring the optical system comprises causing the optical system to wait for the object to move to a different field of the optical system.

21. An imaging method according to claim 19, wherein the step of configuring the optical system comprises changing optical axis of the optical system.

22. An imaging method according to claim 19, wherein the step of configuring the optical system comprises providing relative motion between at least a portion of the optical system and the object.

23. An imaging method according to claim 19, wherein the step of configuring the optical system comprises changing at least one control parameter of a deformable mirror of the optical system.

24. An imaging method according to claim 19, wherein the step of configuring the optical system comprises moving a focal plane array of the optical system.

25. An imaging method according to claim 19, wherein the step of configuring the optical system comprises altering focus of the optical system.

26. An imaging method according to claim 19, wherein the step of configuring the optical system comprises changing dioptric power of a lens of the optical system.

27. An imaging method according to claim 19, wherein the step of configuring the optical system comprises changing size of an aperture of the optical system.

28. An imaging method according to claim 19, wherein the step of configuring the optical system comprises changing zoom setting of the optical system.

29. An imaging method according to claim 19, wherein the step of configuring the optical system comprises panning the optical system.

30. An imaging method according to claim 19, wherein the step of applying extended Wiener filtering comprises performing field correction of the captured images.

31. An imaging method according to claim 19, wherein the step of applying extended Wiener filtering comprises performing spectral band correction of the captured images.

32. An imaging apparatus, the apparatus comprising:

an optical system comprising an image sensor, the optical system being configurable in a plurality of configurations, each configuration of the plurality of configurations corresponding to a different optical transfer function between an object in a scene and the image sensor of the optical system; and

at least one processor, wherein the at least one processor is coupled to the optical system to enable the at least one processor to control configuration of the optical system and to capture images in a focal plane of the optical system, and wherein the at least one processor is configured to execute program code instructions to cause the apparatus to;

capture a plurality of M captured images of the object, wherein each captured image of the plurality of images is captured with a distinct optical transfer function between the object and the image sensor,

apply extended Wiener filtering to each of the captured images to obtain a corresponding filtered image, thereby obtaining a plurality of M filtered images, and

combine computationally the plurality of M weighted images to obtain an enhanced image.

33. An imaging apparatus according to claim 32, wherein the at least one processor is further configured to execute program code instructions to output the enhanced image by displaying the enhanced image, storing the enhanced image, or transmitting the enhanced image.

34. An imaging apparatus according to claim 33, wherein the at least one processor is further configured to execute program code instructions to capture the plurality of images so that the at least one zero of the optical transfer function between the object and the image sensor varies from image to image.

35. An imaging apparatus according to claim 34, wherein the optical system comprises a variable aperture, and the at least one processor is further configured to execute program code instructions to change size of the variable aperture of the optical system from image to image.

36. An imaging apparatus according to claim 34, wherein the at least one processor is further configured to execute program code instructions to perform field-based correction of the captured images in the course of applying extended Wiener filtering.

37. An imaging apparatus according to claim 34, wherein the at least one processor is further configured to execute program code instructions to perform spectral band correction of the captured images in the course of applying extended Wiener filtering.

38. An imaging apparatus according to claim 34, wherein the optical system further comprises a configurable liquid-crystal based spatial light modulator.

39. An imaging apparatus according to claim 34, wherein the optical system further comprises a configurable liquid lens.

40. An imaging apparatus according to claim 34, wherein the optical system further comprises a movable detector array.

41. An imaging apparatus according to claim 34, wherein the optical system further comprises a configurable liquid-crystal based spatial light modulator.