Image shifting apparatus for enhanced image resolution

• US 20050280714A1
• Filed: 06/17/2004
• Published: 12/22/2005
• Est. Priority Date: 06/17/2004
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Freeman, Philip L.
• Granted Patent: US 7,420,592 B2
• Time in Patent Office: Days
• US Class Current: 348/219.1
• CPC Class Codes: G09G 3/007 Use of pixel shift techniques, e.g. by mechanical shift of the physical pixels or by optical shift of the perceived pixels
• CPC Class Codes: H04N 25/48 Increasing resolution by shifting the sensor relative to the scene

Abstract

An image shifting apparatus may capture multiple frames of an image, with each frame offset by a sub-pixel offset distance. After capture, the various sup-pixel offset images may be registered together and analyzed using known resolution enhancement algorithms. Super resolution algorithms, for example, may take the various frames and perform edge identification and peak extraction routines to improve the resolution of high frequency data within an image. The image shifting apparatus may include an actuation controlled optical compensator within the imaging system that may be adjusted to create a sub-pixel offset image of an original reference image. The optical compensator may be an image-stabilizing element, for example, capable of forming sub-pixel shifts in an image plane.

Claims

1. An apparatus for producing an enhanced resolution image, the apparatus comprising:

an image sensor having a pixel array;

at least one optical element positioned to form an image at a first pixel position on the pixel array;

an optical compensator movable between a first position wherein the image is at the first pixel position and a second position wherein the image is at a second pixel position on the pixel array offset from the first pixel position by a sub-pixel offset; and

an actuator for moving the optical compensator between the first position and second position in response to a sub-pixel offset signal.

2. The apparatus of claim 1, wherein the optical compensator includes an adjustable concave optical element.

3. The apparatus of claim 2, wherein the optical compensator comprises a rotatable prism element.

4. The apparatus of claim 1, further comprising a driver coupled to provide the sub-pixel offset signal to the actuator.

5. The apparatus of claim 4, further comprising a processor for enhancing the resolution of the image, wherein the processor is coupled to receive image data from the image sensor and wherein the processor is coupled to the driver to provide the sub-pixel offset signal to the driver.

6. The apparatus of claim 1, wherein the sub-pixel offset is substantially equal to or less than a distance of 1/10 of a pixel on the pixel array.

7. The apparatus of claim 1, wherein the sub-pixel offset is substantially equal to or less than a distance of 1/100 of a pixel on the pixel array.

8. The apparatus of claim 1, further comprising a vibration sensor coupled to sense movement of the optical element.

9. The apparatus of claim 1, further comprising a vibration sensor coupled to sense movement of the optical compensator.

10. The apparatus of claim 9, wherein the vibration sensor is a rate gyro.

11. The apparatus of claim 9, further comprising:

a processor coupled to the vibration sensor; and

a driver coupled to the processor, wherein the processor provides a position adjustment signal to the driver for compensating the position of the actuator.

12. The apparatus of claim 1, further comprising an image capture apparatus including a processor for controlling image capture.

13. The apparatus of claim 12, wherein the processor provides an image capture signal to the image capture apparatus in response to a position of the optical compensator.

14. An article comprising a machine-accessible medium having stored thereon instructions that, when executed by a machine, cause the machine to:

capture a first image at a first pixel position;

provide a sub-pixel offset signal to an optical compensator;

move the optical compensator to form at least one additional image at a second pixel position offset from the first pixel position by a sub-pixel offset distance;

capture the at least one additional image;

compare the first image to the at least one additional image; and

in response to a comparison of the first image and the at least one additional image, derive an enhanced resolution image.

15. The article of claim 14, having further instructions that when executed by the machine, cause the machine to create the enhanced resolution image having a 10×

resolution enhancement in each orthogonal direction of the enhanced resolution image.

16. The article of claim 14, having further instructions that when executed by the machine, cause the machine to identify an edge in the enhanced resolution image.

17. The article of claim 16, having further instructions that when executed by the machine, cause the machine to:

c) determine an edge strength for the edge of the enhanced resolution image; and

d) in response to the determination of the edge strength, enhance a resolution of the edge.

18. The article of claim 16, having further instructions that when executed by the machine, cause the machine to apply a Sobel, Prewitt, Canny, or Susan filter to identify the edge.

19. The article of claim 14, having further instructions that when executed by the machine, cause the machine to calibrate the optical compensator.

20. The article of claim 19, having further instructions that when executed by the machine, cause the machine to:

move the optical compensator an offset distance to produce a calibration image at a calibrated offset position, having a predetermined pixel separation from the first pixel position;

determining a position adjustment signal corresponding to the offset distance; and

deriving a sub-pixel offset signal mapping.

21. The article of claim 14, having further instructions that when executed by the machine, cause the machine to register the first image and the at least one additional image.

22. The article of claim 14, having further instructions that when executed by the machine, cause the machine to move the optical compensator to form the at least one additional image at the second pixel position offset from the first pixel position by a distance equal to or less than approximately 1/10 of a pixel on a pixel array.

23. The article of claim 14, having further instructions that when executed by the machine, cause the machine to move the optical compensator to form the at least one additional image at the second pixel position offset from the first pixel position by a distance equal to or less than approximately 1/100 of a pixel on a pixel array.

24. A method of enhancing resolution of an image, the method comprising:

providing a photodetector;

capturing a first image at a first pixel position on the photodetector;

providing a sub-pixel offset signal to an optical compensator;

moving the optical compensator to form at least one additional image at a second pixel position offset from the first pixel position by a sub-pixel offset distance;

capturing the at least one additional image;

comparing the first image to the at least one additional image; and

in response to the comparison of the first image and the at least one additional image, deriving an enhanced resolution image.

25. The method of claim 24, further comprising moving the optical compensator to form the at least one additional image at the second pixel position offset from the first pixel position by a distance equal to or less than approximately 1/10 of a pixel on the photodetector.

26. The method of claim 25, wherein the distance is equal to or less than approximately 1/100 of the pixel.

28. The method of claim 24, further comprising:

forming the enhanced resolution image to have a 10×

resolution enhancement in each orthogonal direction of the enhanced resolution image.

27. The method of 24, further comprising:

identifying an edge in the enhanced resolution image;

determining an edge strength for the edge of the enhanced resolution image; and

in response to the determination of the edge strength, enhancing a resolution of the edge.