Method for generating all-in-focus image

• US 8,412,002 B2
• Filed: 09/21/2011
• Issued: 04/02/2013
• Est. Priority Date: 05/24/2011
• Status: Active Grant

• Current Assignee: Altek Corporation
• Original Assignee: Altek Corporation
• Inventors: Wu, Chung-Ta, Peng, Shih-Yuan
• Primary Examiner(s): Allison, Andrae S
• Application Number:
  US13/237,958
• Publication Number:
  US 20120301025A1
• Time in Patent Office: 559 Days
• Field of Search: 382/154, 382/275, 382/285, 382298-300, 382/305, 345419-427, 345/700
• US Class Current: 382/299
• CPC Class Codes: G06T 2207/10148 Varying focus
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction

Abstract

A method for generating an all-in-focus image is provided. In the method, a plurality of frames including one source frame and several reference frames captured at different focus lengths are obtained. After performing a motion compensation procedure on the frames, for each pixel position belonging to an edge, a characteristic similarity between the source frame and each reference frame is calculated respectively to accordingly select a plurality of qualified reference frames among the reference frames. The method further includes determining a first type color space component of the pixel position within an all-in-focus image by an edge sharpness corresponding to the pixel position within each of the source frame and the qualified reference frames, and determining a second type color space component of the pixel position within the all-in-focus image by an color brightness corresponding to the pixel position within each of the source frame and the qualified reference frames.

Claims

1. A method for generating an all-in-focus image, for an image capturing device, the method comprising:

obtaining a plurality of frames captured at a plurality of focal lengths, wherein the frames comprise a source frame and a plurality of reference frames;

performing a motion compensation procedure on the frames;

performing an edge detection procedure on the frames to determine whether each of pixel positions in the frames belongs to an edge; and

for each of the pixel positions belonging to the edge, performing the steps of;

calculating a characteristic similarity between the pixel position within the source frame and the pixel position within each of the reference frames;

according to the characteristic similarities respectively corresponding to the reference frames, selecting a plurality of qualified reference frames among the reference frames;

according to an edge sharpness corresponding to the pixel position within each of the source frame and the qualified reference frames, determining a first type color space component of the pixel position within an all-in-focus image; and

according to a color brightness corresponding to the pixel position within each of the source frame and the qualified reference frames, determining a second type color space component of the pixel position within the all-in-focus image.

2. The method of claim 1, wherein a number of the frames captured at each of the focal lengths is larger than one.

3. The method of claim 1, wherein the step of calculating the characteristic similarity between the pixel position within the source frame and the pixel position within each of the reference frames comprises:

in the source frame, obtaining an n×

n pixel block with the pixel position as a center, wherein n is a positive integer; and

for each of the reference frames, performing the steps of;

obtaining the n×

n pixel block with the pixel position as a center; and

calculating a sum of absolute difference (SAD) of a specific color space component of each of pixels in the n×

n pixel block of the source frame and the reference frame to be the characteristic similarity between the pixel position in the source frame and the pixel position in the reference frame.

4. The method of claim 3, wherein the specific color space component comprises one of a brightness component, a blue chromaticity component and a red chromaticity component.

5. The method of claim 3, wherein the step of selecting the qualified reference frames among the reference frames according to the characteristic similarities respectively corresponding to the reference frames comprises:

for each of the reference frames, detecting whether the pixel position within the reference frame has impulse noises;

when there is no impulse noise, obtaining a threshold value of the specific color space component and determining whether the characteristic similarity between the pixel position in the source frame and the pixel position in the reference frame is smaller than the threshold value of the specific color space component; and

when the characteristic similarity is smaller than the threshold value of the specific color space component, selecting the reference frame as the qualified reference frame.

6. The method of claim 5, wherein the step of obtaining the threshold value of the specific color space component comprises:

calculating a mean absolute error (MAE) of the specific color space component of each of the pixels within the n×

n pixel block in the source frame;

calculating the MAE of the specific color space component of each of the pixels within the n×

n pixel block in the reference frame;

comparing the MAE corresponding to the source frame with the MAE corresponding to the reference frame;

obtaining an initial value of the specific color space component according to one larger MAE among the MAE corresponding to the source frame and the MAE corresponding to the reference frame;

obtaining a specific color space component gain according to the specific color space component of the pixel position within the source frame; and

regarding a product of the initial value of the specific color space component and the specific color space component gain as the threshold value of the specific color space component.

7. The method of claim 1, wherein the step of determining the first type color space component of the pixel position within the all-in-focus image according to the edge sharpness corresponding to the pixel position within each of the source frame and the qualified reference frames comprises:

applying a Laplacian mask to calculate the edge sharpness corresponding to the pixel position within each of the source frame and the qualified reference frames;

obtaining m frames each corresponding to larger edge sharpness among the source frame and the reference frames; and

calculating an average weight value of a brightness component of the pixel position of each of m frames to be the first type color space component.

8. The method of claim 1, wherein the step of determining the second type color space component of the pixel position within the all-in-focus image according to the color brightness corresponding to the pixel position within each of the source frame and the qualified reference frames comprises:

determining whether a chromaticity component of the pixel position within each of the source frame and the qualified reference frames is all close to a neutral color;

when the chromaticity component of the pixel position within each of the source frame and the qualified reference frames is all close to the neutral color, determining a weight value corresponding to each of the source frame and the qualified reference frames according to a first rule;

when the chromaticity component of the pixel position within each of the source frame and the qualified reference frames is not all close to the neutral color, determining the weight value corresponding to each of the source frame and the qualified reference frames according to a second rule; and

according to the weight value corresponding to each of the source frame and the qualified reference frames, calculating an average weight value of the chromaticity component of the pixel position within each of the source frame and the qualified reference frames to be the second type color space component.

9. The method of claim 8, wherein the first rule is that the frame of which the chromaticity component of the pixel position therein is closer to the neutral color is given a relatively large weight value and the second rule is that the frame of which the chromaticity component of the pixel position therein is more far from the neutral color is given a relatively large weight value.

10. The method of claim 1, after the step of determining whether each of the pixel positions in each of the frames belongs to the edge, further comprising:

for each of the pixel positions not belonging to the edge, performing the steps of;

using a brightness component of the pixel position in the source frame as the first type color space component of the pixel position within the all-in-focus image; and

using the chromaticity component of the pixel position in the source frame as the second type color space component of the pixel position within the all-in-focus image.