Image capturing apparatus and image processing method

• US 8,922,664 B2
• Filed: 09/14/2012
• Issued: 12/30/2014
• Est. Priority Date: 11/16/2007
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Oshino, Takahiro, Tsubaki, Hidetoshi, Saito, Mitsuhiro
• Primary Examiner(s): Ye, Lin
• Assistant Examiner(s): YODER III, CHRISS S
• Application Number:
  US13/617,247
• Publication Number:
  US 20130010139A1
• Time in Patent Office: 837 Days
• Field of Search: 348/208.1, 348/208.2, 348/208.3, 348/208.5, 348/208.6, 348/208.7, 348/208.99, 348/208.4, 382/293, 382/294, 382/295, 382/296, 382/298, 396/52, 396/53, 396/54, 396/55, 396/153
• US Class Current: 348/208.99
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/6845 by combination of a plurality of images sequentially taken

Abstract

An image processing apparatus includes a geometric converter configured to perform a geometric conversion process to a first frame image among frame images that are sequentially generated through imaging, based on motion information obtained by using a motion detector sensor, a motion vector detector configured to detect a motion vector between a geometric conversion image obtained through the geometric conversion process and a second frame image that has not undergone the geometric conversion process, an image stabilization parameter calculator configured to calculate an image stabilization parameter for different motions by using the motion information and the motion vector, and a stabilized image generator configured to generate a stabilized image through a deformation process to the second frame image or a frame image generated subsequent to the second frame image by utilizing the image stabilization parameter.

Claims

1. An image capturing apparatus comprising:

an image sensor configured to sequentially generate frame images;

a first shake detector configured to detect rotational motion amount in a pitch direction and a yaw direction of the image capturing apparatus;

a second shake detector configured to detect a motion vector based on the frame images; and

a geometric deformation processing unit configured to sequentially generate output images by calculating a rotational motion around an optical axis and a motion amount in a translation direction based on the motion vector and performing a geometric deformation process for the frame images for each motion component of a motion of the image capturing apparatus based on the rotational motion around the optical axis, the motion amount in the translation direction, and the rotational motion amount in the pitch direction and the yaw direction,

wherein the geometric deformation processing unit sequentially generates the output images by generating, as an initial image among the output images that have been sequentially generated, an image where the geometric deformation process has been performed based on the rotational motion amount in the pitch direction and the yaw direction and generating, as images generated subsequent to the initial image among the output images that have been sequentially generated, images where the geometric deformation process has been performed based on the rotational motion around the optical axis and the motion amount in the translation direction and the rotational motion amount in the pitch direction and the yaw direction.

2. The image capturing apparatus according to claim 1, wherein the geometric deformation processing unit sequentially generates the output images by performing an affine conversion for the frame images as the geometric deformation process.

3. The image capturing apparatus according to claim 1, wherein the geometric deformation processing unit makes a weight of the motion vector larger as a distance from the center of the image increases, to perform the geometric deformation process.

4. The image capturing apparatus according to claim 1, further comprising a distortion reduction unit configured to reduce an image distortion component in the frame image as a distortion reduction process, the image distortion component corresponding to a geometric deformation that occurs due to an optical system of the image capturing apparatus,

wherein the geometric deformation processing unit sequentially generates the output images by performing the geometric deformation process for frame images where the distortion reduction process has been performed.

5. A method for controlling an image capturing apparatus including an image sensor configured to sequentially generate frame images, the method comprising:

a first shake detection step of detecting rotational motion amount in a pitch direction and a yaw direction of the image capturing apparatus;

a second shake detection step of detecting a motion vector based on the frame images; and

a geometric deformation processing step of sequentially generating output images by calculating a rotational motion around an optical axis and a motion amount in a translation direction based on the motion vector and performing a geometric deformation process for the frame images for each motion component of a motion of the image capturing apparatus based on the rotational motion around the optical axis, the motion amount in the translation direction, and the rotational motion amount in the pitch direction and the yaw direction,

wherein the geometric deformation processing step sequentially generates the output images by generating, as an initial image among the output images that have been sequentially generated, an image where the geometric deformation process has been performed based on the rotational motion amount in the pitch direction and the yaw direction and generating, as images generated subsequent to the initial image among the output images that have been sequentially generated, images where the geometric deformation process has been performed based on the rotational motion around the optical axis and the motion amount in the translation direction and the rotational motion amount in the pitch direction and the yaw direction.