Image processing method and apparatus

• US 8,737,766 B2
• Filed: 02/20/2013
• Issued: 05/27/2014
• Est. Priority Date: 03/05/2007
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Albu, Felix, Steinberg, Eran, Drimbarean, Alexandru, Florea, Corneliu, Zamfir, Adrian, Corcoran, Peter, Poenaru, Vlad
• Primary Examiner(s): Park, Chan
• Assistant Examiner(s): Roz, Mark
• Application Number:
  US13/772,324
• Publication Number:
  US 20130229542A1
• Time in Patent Office: 461 Days
• Field of Search: None
• US Class Current: 382/275
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening

Abstract

A method and apparatus for providing image processing. For one embodiment of the invention, an image processing apparatus is arranged to process a first relatively underexposed and sharp image of a scene, and a second relatively well exposed and blurred image, nominally of the same scene, the first and second images being derived from respective image sources. The apparatus provides a portion of the relatively first underexposed image as an input signal to an adaptive filter; and a corresponding portion of the second relatively well exposed image as a desired signal to the adaptive filter. The adaptive filter produces an output signal from the input signal and the desired signal; and an image generator constructs a first filtered image from the output signal, relatively less blurred than the second image.

Claims

1. One or more tangible, non-transitory processor readable media having code embodied therein for programming one or more processors to perform an image processing method, wherein the method comprises:

obtaining a first image and a second image of approximately a same scene, the first image being a relatively underexposed, sharp image of the same scene compared with the second image, the second image comprising a relatively well exposed, blurred image compared with the first image, the first image and second image being derived from respective image sources;

providing a first portion of the first image as an input signal to an adaptive filter;

providing a corresponding second portion of the second image as a desired signal to the adaptive filter;

adaptively filtering the input signal to produce an output signal, including replacing the input signal with values from a look up table, wherein the values from the look up table are identified based on calculating values of the input signal and the desired signal; and

constructing a first filtered image from the output signal, relatively less blurred than the second image, and

wherein the first image and the second image are in YCC format and wherein the first portion and the second portion respectively comprise a first Y plane of the first image and a second Y plane of the second image respectively, and

wherein the adaptively filtering and constructing of the first filtered image comprise applying the second Y plane of the relatively well exposed, blurred second image to a color plane of the relatively underexposed, sharp first image to enhance the first Y plane of the first image to construct the first filtered image.

2. The one or more tangible, non-transitory processor readable media according to claim 1 wherein the constructing the first filtered image comprises using the output signal as a Y plane of the first filtered image and using Cb and Cr planes of the first image as the Cb and Cr planes of the first filtered image.

3. The one or more tangible, non-transitory processor readable media according to claim 1 further comprising:

converting the first filtered image to RGB format to result in a first RGB image;

converting the second image to RGB format to result in a second RGB image;

providing a third image portion of the first RGB image as a further input signal to the adaptive filter;

providing a corresponding fourth image portion of the second RGB image as a further desired signal to the adaptive filter;

further adaptively filtering the further input signal to produce a further output signal; and

constructing a further filtered image from the further output signal relatively less blurred than the first filtered image, and

wherein the third image portion and the fourth image portion for further adaptive filtering are in RGB color format.

4. The one or more tangible, non-transitory processor readable media according to claim 3 wherein the method further comprises applying color correction to the first filtered image.

5. The one or more tangible, non-transitory processor readable media according to claim 1 wherein the first image comprises at least one of an image acquired soon before or after the second image.

6. A digital image acquisition device, comprising:

a lens and image sensor for acquiring digital images;

a processor;

a memory having processor readable code embodied therein for programming one or more processors to perform an image processing method, wherein the method comprises;

obtaining a first image and a second image of approximately a same scene, the first image being a relatively underexposed, sharp image of the same scene compared with the second image, the second image comprising a relatively well exposed, blurred image compared with the first image, the first image and second image being derived from respective image sources;

providing a first portion of the first image as an input signal to an adaptive filter;

providing a corresponding second portion of the second image as a desired signal to the adaptive filter;

adaptively filtering the input signal to produce an output signal, including replacing the input signal with values from a look up table, wherein the values from the look up table are identified based on calculating values of the input signal and the desired signal; and

constructing a first filtered image from the output signal, relatively less blurred than the second image, and

wherein the first image and the second image are in YCC format and wherein the first portion and the second portion respectively comprise a first Y plane of the first image and a second Y plane of the second image respectively, and

wherein the adaptively filtering and constructing of the first filtered image comprise applying the second Y plane of the relatively well exposed, blurred second image to a color plane of the relatively underexposed, sharp first image to enhance the first Y plane of the first image to construct the first filtered image.

7. The device according to claim 6 wherein the constructing the first filtered image comprises using the output signal as a Y plane of the first filtered image and using Cb and Cr planes of the first image as the Cb and Cr planes of the first filtered image.

8. The device according to claim 6 wherein the method comprises:

converting the first filtered image to RGB format to result in a first RGB image;

converting the second image to RGB format to result in a second RGB image;

providing a third image portion of the first RGB image as a further input signal to the adaptive filter;

providing a corresponding fourth image portion of the second RGB image as a further desired signal to the adaptive filter;

further adaptively filtering the further input signal to produce a further output signal; and

constructing a further filtered image from the further output signal relatively less blurred than the first filtered image, and

wherein the third image portion and the fourth image portion for further adaptive filtering are in RGB color format.

9. The device according to claim 6 wherein the second image is of a relatively higher resolution than the first image.

10. The device according to claim 6 wherein the method further comprises aligning and interpolating the first image to match the alignment and resolution of the second image.