End-to-end design of superresolution electro-optic imaging systems

• US 20070268374A1
• Filed: 05/12/2006
• Published: 11/22/2007
• Est. Priority Date: 05/12/2006
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Robinson, M.
• Granted Patent: US 7,889,264 B2
• Time in Patent Office: Days
• US Class Current: 348/222.100
• CPC Class Codes: H04N 25/48 Increasing resolution by shifting the sensor relative to the scene
• CPC Class Codes: H04N 25/615 involving a transfer function modelling the optical system, e.g. optical transfer function [OTF], phase transfer function [PhTF] or modulation transfer function [MTF]

Abstract

A superresolution electro-optic imaging system operates in a manner that takes into account the different subsystems. For example, rather than designing the optical subsystem to be diffraction-limited, the optical subsystem can be designed in a manner that is better suited for subsequent superresolution processing and/or that reduces aliasing effects.

Claims

1. A superresolution electro-optic imaging system comprising:

an optical subsystem for imaging a source, wherein the optical subsystem is characterized by an optical MTF which, in comparison to an equivalent diffraction-limited optical MTF, contains a higher fraction of energy in a pass band of frequencies below a superresolved sampling frequency;

a detector subsystem positioned to capture two or more lower resolution images of the source, wherein the detector subsystem is characterized by a detector sampling frequency; and

a superresolution digital image processing subsystem coupled to the detector subsystem for combining the lower resolution images via superresolution processing into a higher resolution image of the source, wherein the superresolution processing determines a superresolved sampling frequency that is higher than the detector sampling frequency.

2. The electro-optic imaging system of claim 1 wherein the optical MTF behaves substantially as a low pass filter with a cutoff frequency at the superresolved sampling frequency.

3. The electro-optic imaging system of claim 1 wherein the optical subsystem and detector subsystem together are characterized by an imaging MTF that has no zeroes below the superresolved sampling frequency.

4. The electro-optic imaging system of claim 1 wherein, in comparison both to the equivalent diffraction-limited optical MTF and to an equivalent low pass filtered single-frame optical MTF, the optical MTF contains a higher fraction of energy in a superresolution band of frequencies between the detector sampling frequency and the superresolved sampling frequency.

5. The electro-optic imaging system of claim 1 wherein the superresolved sampling frequency is an integer multiple of the detector sampling frequency.

6. The electro-optic imaging system of claim 1 wherein the lower resolution images include shifted images of a same resolution.

7. The electro-optic imaging system of claim 6 wherein the detector subsystem includes:

a detector array; and

an actuator for shifting the detector array relative to the optical subsystem, to produce the shifted images.

8. The electro-optic imaging system of claim 6 wherein the optical subsystem includes:

imaging optics; and

an actuator for shifting the imaging optics relative to the detector subsystem, to produce the shifted images.

9. The electro-optic imaging system of claim 6 wherein:

the optical subsystem and the detector subsystem have fixed positions relative to each other; and

the optical subsystem includes an optical element for optically shifting an image formed by the optical subsystem relative to the detector subsystem, to produce the shifted images.

10. The electro-optic imaging system of claim 1 wherein the optical subsystem has been designed accounting for the detector subsystem and the superresolution processing.

11. The electro-optic imaging system of claim 1 wherein an effective imaging MTF for the higher resolution image has no zeroes below the superresolved sampling frequency.

12. A superresolution electro-optic imaging system comprising:

an imaging subsystem comprising an optical subsystem coupled to a detector subsystem, wherein the imaging subsystem is characterized by an adjustable imaging MTF for producing lower resolution images of a source using different imaging MTFs; and

a superresolution digital image processing subsystem coupled to the imaging subsystem for combining the lower resolution images via superresolution processing into a higher resolution image of the source.

13. The electro-optic imaging system of claim 12 wherein the detector subsystem includes detectors with an adjustable geometry, and adjusting the detector geometry adjusts the imaging MTF.

14. The electro-optic imaging system of claim 13 wherein different detector geometries have a same detector pitch but a different detector fill factor.

15. The electro-optic imaging system of claim 13 wherein different detector geometries have a same detector pitch but a different detector shape.

16. The electro-optic imaging system of claim 12 wherein the optical subsystem has moveable optical components, moving the optical components adjusts the imaging MTF.

17. The electro-optic imaging system of claim 12 wherein the imaging MTF is characterized by an imaging cutoff frequency, and different imaging MTFs have different imaging cutoff frequencies.

18. The electro-optic imaging system of claim 12 wherein an effective imaging MTF for the higher resolution image has no zeroes below a superresolved sampling frequency of the higher resolution image.

19. The electro-optic imaging system of claim 12 wherein the optical MTF behaves substantially as a low pass filter with a cutoff frequency at a superresolved sampling frequency of the higher resolution image.

20. The electro-optic imaging system of claim 12 wherein the imaging MTF has no zeroes below a superresolved sampling frequency of the higher resolution image.

21. The electro-optic imaging system of claim 12 wherein, in comparison both to the equivalent diffraction-limited imaging MTF and to the equivalent low pass filtered single-frame imaging MTF, the imaging MTFs contains a higher fraction of energy in a superresolution band of frequencies between a detector sampling frequency and a superresolved sampling frequency of the higher resolution image.

22. A multiple mode superresolution electro-optic imaging system comprising:

an imaging subsystem comprising an optical subsystem coupled to a detector subsystem, and a digital image processing subsystem coupled to the imaging subsystem, both the imaging subsystem and digital image processing subsystem capable of operating either in a single-frame mode or in a superresolution mode;

wherein;

in a single-frame mode, the imaging subsystem produces an image of a source and the digital image processing subsystem processes the image; and

in a superresolution mode, the imaging subsystem produces two or more lower resolution images of a source and the digital image processing subsystem combines the lower resolution images via superresolution processing into a higher resolution image of the source.

23. The electro-optic imaging system of claim 22 wherein the optical subsystem has one or more moveable optical components, and the optical subsystem is switched between single-frame mode and superresolution mode by moving the optical component(s).

24. The electro-optic imaging system of claim 22 wherein the optical subsystem and detector subsystem together determine an effective imaging MTF characterized by an imaging cutoff frequency, and the imaging cutoff frequency for the superresolution mode is higher than the imaging cutoff frequency for the single-frame mode.

25. The electro-optic imaging system of claim 24 wherein the optical subsystem and detector subsystem together determine an imaging MTF characterized by an imaging cutoff frequency, and the imaging cutoff frequency for the superresolution mode is an integer multiple of the imaging cutoff frequency for the single-frame mode.

26. The electro-optic imaging system of claim 22 wherein, in the superresolution mode, the superresolution processing determines a superresolved sampling frequency that is higher than the detector sampling frequency and, in comparison both to an equivalent diffraction-limited optical MTF and to a low pass filtered single-frame optical MTF, an optical MTF for the optical subsystem contains a higher fraction of energy in a pass band for frequencies below the superresolved sampling frequency.

27. A method for designing a superresolution electro-optic imaging system for imaging a source, the electro-optic imaging system including an optical subsystem, a detector subsystem and a superresolution digital image processing subsystem, the method comprising:

modeling propagation of the source through the optical subsystem, the detector subsystem and the superresolution digital image processing subsystem based on a spatial model of the source; and

jointly designing the optical subsystem and the superresolution digital image processing subsystem based directly on a post-processing performance metric that is a function of the modeled propagation.

28. A software product comprising instructions stored on a computer readable medium, wherein the instructions cause a processor to assist in a design of a superresolution electro-optic imaging system by executing the following steps:

(a) accessing a spatial model of a source;

(b) based in part on the spatial model, modeling propagation of the source through an electro-optic imaging system;

the electro-optic imaging system comprising an optical subsystem, a detector subsystem and a superresolution digital image processing subsystem;

(c) adjusting the optical subsystem and the superresolution digital image processing subsystem based directly on a post-processing performance metric that is a function of the modeled propagation; and

(d) repeating steps (b) and (c) to improve the post-processing performance metric.