APPARATUS AND METHOD FOR GENERATING A FULLY FOCUSED IMAGE BY USING A CAMERA EQUIPPED WITH A MULTI-COLOR FILTER APERTURE

• US 20120113227A1
• Filed: 11/07/2011
• Published: 05/10/2012
• Est. Priority Date: 11/05/2010
• Status: Active Grant

• Current Assignee: Industry-University Cooperation Foundation Hanyang University
• Original Assignee: Industry-University Cooperation Foundation Hanyang University
• Inventors: PAIK, Joon-Ki, KIM, Sang-Jin, LEE, Eun-Sung
• Granted Patent: US 8,836,765 B2
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: H04N 23/67 Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/84 for processing colour signals

Abstract

Provided are an apparatus and method for generating a fully focused image. A depth map generation unit generates a depth map of an input image obtained by a multiple color filter aperture (MCA) camera. A channel shifting & alignment unit extractes subimages which include objects with same focal distance based on the depth map, and performing color channel alignment and removing out-of-focus blurs for each subimages obtained from the depth map. An image fusing unit fuses the subimages to generate a fully focused image.

Claims

1. An apparatus for generating a fully focused image, the apparatus comprising:

a depth map generation unit generating a depth map of an input image obtained by a multiple color filter aperture (MCA) camera;

a channel shifting &

alignment unit extracting subimages which include objects with same focal distance based on the depth map, and performing color channel alignment and removing out-of-focus blurs for each subimages obtained from the depth map; and

an image fusing unit fusing the subimages to generate a fully focused image.

2. The apparatus of claim 1, wherein the depth map generation unit generates the depth map by establishing a set of pixel colors with hypothesized disparity d as Equation 1 shown below, and searches for d that minimizes an abstract disparity map represented by following Equation 2:

P₁(x,y;

d)={(I_(R)(s+d,t), I_(G)(s,t−

d), I_(B)(s−

d,t))|(s,t)∈

w(x,y)} 

 

Equation 1

L(x,y;

d)=λ

₀λ

₁λ

₂/σ

_r²σ

_g²σ

_b² 

 

Equation 2,

here, I_(R), I_(G), and I_(B) represent red, green, and blue color channel images, respectively, (s+d,t), (s,t−

d), and (s−

d,t) are shift vectors for each color channel images, λ

₀λ

₁, and λ

₂ represent the eigenvalues of the covariance matrix Σ

of the color distribution P₁(x,y;

d), respectively, w(x,y) is a local window, and σ

_r², σ

_g², and σ

_b² are the diagonal elements of the covariance matrix Σ

.

3. The apparatus of claim 2, wherein the depth map generation unit generates the depth map by combining the abstract disparity map and a mean-shift segmentation result.

4. The apparatus of claim 1, further comprising:

a smoothing unit smoothing the fully focused image to enhance image quality.

5. A method for generating a fully focused image, the method comprising:

(a) generating a depth map of an input image obtained by a multiple color filter aperture (MCA) camera;

(b) extracting subimages which include objects with same focal distance based on the depth map;

(c) performing color channel alignment and removing out-of-focus blurs for each subimages obtained from the depth map; and

(d) fusing the subimages to generate a fully focused image.

6. The method of claim 5, wherein step (a) comprises:

(a1) establishing a set of pixel colors with hypothesized disparity d as Equation 1 shown below; and

(a2) searching for d that minimizes an abstract disparity map represented by following Equation 2;

P₁(x,y;

d)={(I_(R)(s+d,t), I_(G)(s,t−

d), I_(B)(s−

d,t))|(s,t)∈

w(x,y)} 

 

Equation 1

L(x,y;

d)=λ

₀λ

₁λ

₂/σ

_r²σ

_g²σ

_b² 

 

Equation 2

here, I_(R), I_(G), and I_(B) represent red, green, and blue color channel images, respectively, (s+d,t), (s,t−

d), and (s−

d,t) are shift vectors for each color channel images, λ

₀, λ

₁, and λ

₂ represent the eigenvalues of the covariance matrix Σ

of the color distribution P₁(x,y;

d), respectively, w(x,y) is a local window, and σ

_r², σ

_g², and σ

_b² are the diagonal elements of the covariance matrix Σ

.

7. The method of claim 6, wherein step (a) further comprises (a3) generating the depth map by combining the abstract disparity map and a mean-shift segmentation result.

8. The method of claim 5, further comprising:

(e) smoothing the fully focused image to enhance image quality.

9. A non-transitory computer readable medium storing a program for executing the method for managing digital rights according to claim 5 in a computer.