USING CAPTURED HIGH AND LOW RESOLUTION IMAGES

• US 20110052095A1
• Filed: 08/31/2009
• Published: 03/03/2011
• Est. Priority Date: 08/31/2009
• Status: Active Grant

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Intellectual Ventures Fund 83 LLC (Intellectual Ventures LLC)
• Inventors: Deever, Aaron T.
• Granted Patent: US 8,724,928 B2
• Time in Patent Office: Days
• US Class Current: 382/300
• CPC Class Codes: G06T 3/4076 using the original low-resolution images to iteratively correct the high-resolution images
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/81 for suppressing or minimising disturbance in the image signal generation
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 25/46 by combining or binning pixels
• CPC Class Codes: H04N 25/60 Noise processing, e.g. detecting, correcting, reducing or removing noise
• CPC Class Codes: H04N 5/265 Mixing

Abstract

A method for producing an improved high resolution image is disclosed including capturing low resolution images and a high resolution image; combining the low resolution images to provide an aggregate low resolution image; reducing the resolution of the high resolution image to provide an image representative of a low resolution image that could have been captured by the image capture device; interpolating the representative low resolution image to provide an interpolated high resolution image; calculating a residual image from the high resolution image and the interpolated high resolution image; combining the representative low resolution image and the aggregate low resolution image to provide a final low resolution image; and using the final low resolution image and the residual image to provide the improved high resolution image.

Claims

1. A method for using low resolution images and at least one high resolution image of a scene captured by the same image capture device to provide an improved high resolution image, comprising:

a) using the image capture device to capture a plurality of low resolution images of the scene and at least one high resolution image of the scene having a resolution greater than that of the low resolution images;

b) combining the low resolution images to provide an aggregate low resolution image; and

c) combining the high resolution image with the aggregate low resolution image to provide the improved high resolution image by;

i) reducing the resolution of the high resolution image to provide an image representative of a low resolution image that could have been captured by the image capture device;

ii) interpolating the representative low resolution image to provide an interpolated high resolution image;

iii) calculating a residual image from the high resolution image and the interpolated high resolution image;

iv) combining the representative low resolution image and the aggregate low resolution image to provide a final low resolution image; and

v) using the final low resolution image and the residual image to provide the improved high resolution image.

2. The method of claim 1 wherein step b) further includes using an infinite impulse response filter to combine the low resolution images to provide an aggregate low resolution image.

3. The method of claim 1 wherein step b) further includes correcting for motion present between at least some of the low resolution images.

4. The method of claim 1 wherein step c) i) further includes reducing the resolution of the high resolution image using substantially the same binning and decimation procedure used in the capture of the low resolution images.

5. The method of claim 1 wherein step c) iv) further includes correcting for motion present between the aggregate low resolution image and the representative low resolution image.

6. The method of claim 1 wherein step c) iv) further includes forming a map identifying matching pixels for which the aggregate low resolution image and representative low resolution image have matching scene data.

7. The method of claim 6 wherein the final low resolution image is formed for matching pixels by computing a weighted average of corresponding aggregate low resolution image and representative low resolution image pixel data, and the final low resolution image is formed for non-matching pixels by using only the representative low resolution image pixel data.

8. The method of claim 7 further including applying a spatial low pass filter to non-matching pixels of the final low resolution image.

9. The method of claim 1 wherein step c) v) further includes interpolating the final low resolution image and adding the residual image to the interpolated final low resolution image to provide the improved high resolution image.

10. The method of claim 1 wherein the low resolution images and high resolution image are fill color images.

11. The method of claim 1 wherein the low resolution images and high resolution image are color filter array images and the improved high resolution image is a color filter array image.

12. The method of claim 1 wherein the low resolution images and high resolution image have color pixels and panchromatic pixels.

13. A method for using low resolution images and at least one high resolution image of a scene captured by the same image capture device having an image sensor containing panchromatic pixels and color pixels to provide an improved high resolution image, comprising:

a) using an image capture device having an image sensor containing panchromatic pixels and color pixels to capture a plurality of low resolution images of the scene and at least one high resolution image of the scene having a resolution greater than that of the low resolution images;

b) combining the low resolution images to provide an aggregate low resolution image; and

c) combining the high resolution image with the aggregate low resolution image to provide the improved high resolution image.

14. The method of claim 13 wherein step c) comprises:

i) reducing the resolution of the high resolution image to provide an image representative of a low resolution image that could have been captured by the image capture device;

ii) interpolating the representative low resolution image to provide an interpolated high resolution image;

iii) calculating a residual image from the high resolution image and the interpolated high resolution image;

iv) combining the representative low resolution image and the aggregate low resolution image to provide a final low resolution image; and

v) using the final low resolution image and the residual image to provide the improved high resolution image.