Image processing method, image processing apparatus, and image pickup apparatus

• US 9,041,833 B2
• Filed: 09/28/2009
• Issued: 05/26/2015
• Est. Priority Date: 09/30/2008
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Hatakeyama, Koshi
• Primary Examiner(s): Haskins, Twyler
• Assistant Examiner(s): Chon, Peter
• Application Number:
  US13/119,859
• Publication Number:
  US 20110193997A1
• Time in Patent Office: 2,066 Days
• Field of Search: 348/335, 348/251, 348/222.1, 348/207.99, 348/223.1, 348/224.1, 348/229.1, 348/230.1, 348/234, 348/241, 348/254, 348/255, 359/16, 382/254, 382/260, 382/273, 382/276, 382/280, 382/255, 382/263, 382/264, 382/266, 382/275
• US Class Current: 348/241
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: H04N 23/81 for suppressing or minimising disturbance in the image signal generation
• CPC Class Codes: H04N 25/61 the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"

Abstract

The image processing method processes an image generated by image pickup using an optical system (100). The method includes a step of acquiring the image, an image restoration step (S2-S7) of performing image restoration processing to reduce a blur component of the image using an image restoration filter, and a distortion correction step (S8) of performing geometric transformation processing to reduce a distortion component of the image on which the image restoration processing has been performed.

Claims

1. An image processing method for processing an image generated by image pickup using an optical system, the method comprising:

a step of acquiring the image;

an image restoration step of performing image restoration processing to reduce a blur component of the image using an image restoration filter created on a basis of an optical transfer function of the optical system; and

a distortion correction step of performing geometric transformation processing to reduce a distortion component of the image on which the image restoration processing has been performed, only after performing the image restoration step,

wherein no other geometric transformation processing to reduce a distortion component of the image is performed before the image restoration step,

wherein the image restoration step performs the image restoration processing using image information of an area to be removed by the geometric transformation processing in the image, and

wherein the following relationship is satisfied;

0<

F/D<

2,

where D represents a number of pixels by which an edge pixel of an effective image area in the image on which the image restoration processing has been performed is shifted in a specific direction by the geometric transformation processing, and F represents a number of cells in the specific direction in the image restoration filter applied to the edge pixel.

2. An image processing method according to claim 1, wherein the image restoration filter is created by performing inverse Fourier transformation on a function created on a basis of an inverse function of an optical transfer function of the optical system.

3. An image processing method according to claim 1, wherein:

the image restoration step acquires information on an image pickup condition, and

the image restoration step performs the image restoration processing using the image restoration filter selected or created based on the information on the image pickup condition.

4. An image processing method according to claim 1, wherein:

the distortion correction step acquires information on an image pickup condition, and

the distortion correction step performs the geometric transformation processing using a geometric transformation condition selected or created based on the information on the image pickup condition.

5. An image processing method according to claim 1, wherein:

the image restoration step uses plural image restoration filters for plural colors as the image restoration filter, the plural image restoration filters having mutually different two-dimensional distributions of the filter values, and

the plural image restoration filters have mutually different shift amounts of a position of a cell having a maximum absolute value of the filter value with respect to a position of a center cell, the shift amounts corresponding to an amount of chromatic aberration of magnification of the optical system, and the plural image restoration filters reduce the blur component and a component of chromatic aberration of magnification.

6. An image processing method according to claim 1, wherein the image restoration step performs convolution processing using the image restoration filter on the image.

7. An image processing apparatus processing an image generated by image pickup using an optical system, the image processing apparatus comprising:

an image restoring part configured to perform image restoration processing to reduce a blur component of the image using an image restoration filter created on a basis of an optical transfer function of the optical system; and

a distortion correcting part configured to perform geometric transformation processing to reduce a distortion component of the image on which the image restoration processing has been performed, only after the image restoration processing has been performed,

wherein no other geometric transformation processing to reduce a distortion component of the image is performed before the image restoration processing has been performed,

wherein the image restoring part performs the image restoration processing using image information of an area to be removed by the geometric transformation processing in the image, and

wherein the following relationship is satisfied;

0<

F/D<

2,

where D represents a number of pixels by which an edge pixel of an effective image area in the image on which the image restoration processing has been performed is shifted in a specific direction by the geometric transformation processing, and F represents a number of cells in the specific direction in the image restoration filter applied to the edge pixel.

8. An image pickup apparatus comprising:

an image pickup system configured to photoelectrically convert an object image formed by an optical system to generate an image; and

an image processing apparatus processing an image generated by image pickup using an optical system,

wherein the image processing apparatus comprises;

an image restoring part configured to perform image restoration processing to reduce a blur component of the image using an image restoration filter created on a basis of an optical transfer function of the optical system; and

a distortion correcting part configured to perform geometric transformation processing to reduce a distortion component of the image on which the image restoration processing has been performed, only after the image restoration processing has been performed,

wherein no other geometric transformation processing to reduce a distortion component of the image is performed before the image restoration processing has been performed,

wherein the image restoring part performs the image restoration processing using image information of an area to be removed by the geometric transformation processing in the image, and

wherein the following relationship is satisfied;

0<

F/D<

2,

where D represents a number of pixels by which an edge pixel of an effective image area in the image on which the image restoration processing has been performed is shifted in a specific direction by the geometric transformation processing, and F represents a number of cells in the specific direction in the image restoration filter applied to the edge pixel.