Using captured high and low resolution images

• US 9,955,071 B2
• Filed: 04/25/2017
• Issued: 04/24/2018
• Est. Priority Date: 08/31/2009
• Status: Active Grant

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Inventors: Deever, Aaron T.
• Primary Examiner(s): Niu, Feng
• Assistant Examiner(s): Thomas, Mia M
• Application Number:
  US15/497,074
• Publication Number:
  US 20170230573A1
• Time in Patent Office: 364 Days
• CPC Class Codes: G06T 3/4076 using the original low-resolution images to iteratively correct the high-resolution images
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/81 for suppressing or minimising disturbance in the image signal generation
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 25/133 including elements passing panchromatic light, e.g. filters passing white light
• CPC Class Codes: H04N 25/46 by combining or binning pixels
• CPC Class Codes: H04N 25/60 Noise processing, e.g. detecting, correcting, reducing or removing noise
• CPC Class Codes: H04N 5/265 Mixing

Abstract

A method produces an improved high resolution image. This method includes capturing low resolution images and a high resolution image and combining the low resolution images to provide an aggregate low resolution image. The method further includes reducing the resolution of the high resolution image to provide an image representative of a low resolution image at a low resolution setting of the image capture device; interpolating the representative low resolution image to provide an interpolated high resolution image; and calculating a residual image from the high resolution image and the interpolated high resolution image. The method also includes combining the representative low resolution image and the aggregate low resolution image to provide a final low resolution image and using the final low resolution image and the residual image to provide the improved high resolution image.

Claims

1. An image capture device comprising:

a shutter;

an image sensor; and

a processor;

wherein the image capture device is configured to;

capture at least two low resolution images and at least one high resolution image;

combine the low resolution images to provide an aggregate low resolution image;

reduce the resolution of at least one of the at least one high resolution image to provide an image representative of a low resolution image at a low resolution setting of the image capture device;

interpolate the representative low resolution image to provide an interpolated high resolution image;

calculate a residual image from the high resolution image and the interpolated high resolution image; and

map matching pixels from the aggregate low resolution image and the representative low resolution image.

2. The image capture device of claim 1, further including a response filter, wherein the response filter is configured to combine the low resolution images to provide the aggregate low resolution image.

3. The image capture device of claim 1, wherein the processor is configured to:

reduce the resolution of the high resolution image to provide the representative low resolution image;

interpolate the representative low resolution image to provide the interpolated high resolution image;

calculate the residual image; and

map matching pixels from the aggregate low resolution image and the representative low resolution image.

4. The image capture device of claim 1, wherein the aggregate low resolution image and the representative low resolution image are combined to provide a final low resolution image.

5. The image capture device of claim 4, wherein the processor is configured to provide an improved high resolution image using the final low resolution image and the residual image.

6. The image capture device of claim 1, wherein the processor is further configured to form a final low resolution image for matching pixels.

7. The image capture device of claim 6, wherein the processor is further configured to provide an improved high resolution image using the final low resolution image and the residual image.

8. The image capture device of claim 7, wherein the processor is further configured to form a final low resolution image for non-matching pixels.

9. The image capture device of claim 8, wherein the final low resolution for non-matching pixels includes a value of the representative low resolution image.

10. The image capture device of claim 6, wherein the final low resolution image for matching pixel includes a weighted average of the aggregate low resolution image and the representative low resolution image pixel data.

11. The image capture device of claim 10, wherein the weighted average is calculated as a function of a noise standard deviation of the aggregate low resolution image and the representative low resolution image.

12. The image capture device of claim 1, wherein the processor is further configured to estimate motion between the aggregate low resolution image and the representative low resolution image.

13. The image capture device of claim 12, wherein the processor is further configured to compensate for the estimated motion.

14. An image capture device comprising:

a shutter;

an image sensor;

a processor; and

a non-transitory computer-readable medium having instructions stored thereon comprising;

instructions for capturing low resolution images of a scene and at least one high resolution image of the scene;

instructions for combining the low resolution images to provide an aggregate low resolution image;

instructions for reducing the resolution of the high resolution image to provide an image representative of a low resolution image at a low resolution setting of the image capture device;

instructions for interpolating the representative low resolution image to provide an interpolated high resolution image;

instructions for calculating a residual image from the high resolution image and the interpolated high resolution image;

instructions for mapping matching pixels from the aggregate pixels from the aggregate low resolution image and the representative low resolution image.

15. The image capture device of claim 14, wherein the non-transitory computer readable medium further comprises instructions for estimating motion between the aggregate low resolution image and the interpolated high resolution image.

16. The image capture device of claim 15, wherein the non-transitory computer readable medium further comprises instructions for compensating for the estimated motion.

17. The image capture device of claim 14, wherein the non-transitory computer readable medium further comprises instructions for forming a final low resolution image for matching pixels.

18. The image capture device of claim 17, wherein the non-transitory computer readable medium further comprises instructions for forming an improved high resolution image using the final low resolution image and the residual image.

19. An image capture device comprising:

a shutter;

an image sensor;

a processor; and

a non-transitory computer-readable medium having instructions stored thereon comprising;

instructions for capturing a plurality of images of a scene wherein at least one of the plurality of images is a high resolution image;

instructions for estimating motion between different ones of the plurality of images;

instructions for compensating for the estimated motion to provide an aggregate image;

instructions for reducing the resolution of at least one high resolution image to provide an image representative of a low resolution image at a low resolution setting of the image capture device; and

instructions for mapping matching pixels from the aggregate image and representative low resolution image to form a final low resolution image.

20. The image capture device of claim 19 wherein the non-transitory computer readable medium further comprises instructions for forming an improved high resolution image using the final low resolution image.