Method and apparatus for improving image resolution

• US 6,208,765 B1
• Filed: 06/19/1998
• Issued: 03/27/2001
• Est. Priority Date: 06/19/1998
• Status: Expired due to Term

• Current Assignee: Sarnoff Corporation (SRI International, Inc.)
• Original Assignee: Sarnoff Corporation (SRI International, Inc.)
• Inventors: Bergen, James Russell
• Primary Examiner(s): ROGERS, SCOTT A
• Application Number:
  US09/100,853
• Time in Patent Office: 1,012 Days
• Field of Search: 382/284, 382/294, 382/268, 382/287, 382/289, 382/266, 382/269, 382/275, 382/276, 382/293, 382/299, 382/300, 358/447, 358/450, 358/463, 345/433, 345/436
• US Class Current: 382/268
• CPC Class Codes: G06T 3/4007 based on interpolation, e.g. bilinear interpolation image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: H04N 1/3876 Recombination of partial images to recreate the original image

Abstract

A method and apparatus that improves the quality of digital images. When provided a plurality of digital images, the method and apparatus aligns the digital images to a reference coordinate system, and synthesizes an enhanced image such that the regions of image overlap between any two images have substantially improved quality.

Claims

1. A method for producing an enhanced quality digital image from a plurality of source images representing a scene comprising the steps of:

selecting one of the source images as a reference image;

aligning image information within each source image with a reference coordinate system of said reference image;

spatially interpolating pixels in said reference image;

warping each of the other source images with respect to said spatially interpolated reference image to produce a plurality of warped images; and

combining the image information in each of said warped images to produce an enhanced image.

2. The method of claim 1 wherein the enhanced image corrects image impairments that result from image sampling and compression.

3. The method of claim 2 wherein said image impairments include low digital capture resolution, low capture dynamic range, or pixel defects.

4. The method of claim 1 further comprising the step of sourcing said source images from a digital camera.

5. The method of claim 1 wherein said alignment step further comprises the steps of:

registering said source images to said reference coordinate system using an affine transformation; and

producing transformation parameters that define said alignment.

6. The method of claim 5 wherein said transformation parameters are used in said warping step.

7. The method of claim 1 wherein said spatially interpolating step uses a fourth order Langragian interpolation method to double a sample density of said reference image.

8. The method of claim 1 wherein said combining step uses a trimmed mean process, where the pixel values at a location in each warped source image and reference image are sorted and averaged after a largest pixel value and a smallest pixel value are discarded.

9. Apparatus for producing an enhanced quality digital image from a plurality of source images representing a scene comprising:

an image selector for selecting one of the source images as a reference image;

an image alignment module for aligning image information within each source image with a reference coordinate system of said reference image;

a pixel interpolator for spatially interpolating pixels in said reference image;

an image warping module for warping each of the other source images with respect to said spatially interpolated reference image to produce a plurality of warped images; and

an image combining module for combining the image information in each of said warped images to produce an enhanced image.

10. The apparatus of claim 9 wherein the enhanced image corrects image impairments that result from image sampling and compression.

11. The apparatus of claim 10 wherein said image impairments include low digital capture resolution, low capture dynamic range, or pixel defects.

12. The apparatus of claim 9 further comprising a digital camera.

13. The apparatus of claim 9 wherein said image alignment module further comprises:

an image registration module for registering said source images to said reference coordinate system using an affine transformation and producing transformation parameters that define said alignment.

14. The apparatus of claim 13 wherein said transformation parameters are used in said image warping module.

15. The apparatus of claim 9 wherein said pixel interpolator uses a fourth order Langragian interpolation method to double a sample density of said reference image.

16. The apparatus of claim 9 wherein said image combining module uses a trimmed mean process, where the pixel values at a location in each warped source image and reference image are sorted and averaged after a largest pixel value and a smallest pixel value are discarded.

17. A digital storage medium containing a program that, when executed by a general purpose computer, causes the computer to perform a method for producing an enhanced quality digital image from a plurality of source images representing a scene comprising the steps of:

selecting one of the source images as a reference image;

aligning image information within each source image with a reference coordinate system of said reference image;

spatially interpolating pixels in said reference image;

warping each of the other source images with respect to said spatially interpolated reference image to produce a plurality of warped images; and

combining the image information in each of said warped images to produce an enhanced image.

18. The medium of claim 17 wherein the enhanced image corrects image impairments that result from image sampling and compression.

19. The medium of claim 18 wherein said image impairments include low digital capture resolution, low capture dynamic range, or pixel defects.

20. The medium of claim 17 wherein the method further comprises the step of sourcing said source images from a digital camera.

21. The medium of claim 17 wherein said alignment step performed by said method further comprises the steps of:

registering said source images to said reference coordinate system using an affine transformation; and

producing transformation parameters that define said alignment.

22. The medium of claim 21 wherein said transformation parameters are used in said warping step.

23. The medium of claim 17 wherein said spatially interpolating step uses a fourth order Langragian interpolation method to double a sample density of said reference image.

24. The medium of claim 17 wherein said combining step uses a trimmed mean process, where the pixel values at a location in each warped source image and reference image are sorted and averaged after a largest pixel value and a smallest pixel value are discarded.