Image capturing apparatus and image processing method

• US 8,294,773 B2
• Filed: 11/12/2008
• Issued: 10/23/2012
• Est. Priority Date: 11/16/2007
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Oshino, Takahiro, Tsubaki, Hidetoshi, Saito, Mitsuhiro
• Primary Examiner(s): Moe, Aung S
• Assistant Examiner(s): Yoder, III, Chriss
• Application Number:
  US12/269,375
• Publication Number:
  US 20090128700A1
• Time in Patent Office: 1,441 Days
• Field of Search: 348/208.1, 348/208.2, 348/208.3, 348/208.4, 348/208.6, 348/208.99, 382/293, 382/294, 382/295, 382/296, 382/298, 396/52, 396/53, 396/54, 396/55, 396/153
• US Class Current: 348/208.6
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/6845 by combination of a plurality of images sequentially taken

Abstract

An image processing apparatus includes a geometric converter configured to perform a geometric conversion process to a first frame image among frame images that are sequentially generated through imaging, based on motion information obtained by using a motion detector sensor, a motion vector detector configured to detect a motion vector between a geometric conversion image obtained through the geometric conversion process and a second frame image that has not undergone the geometric conversion process, an image stabilization parameter calculator configured to calculate an image stabilization parameter for different motions by using the motion information and the motion vector, and a stabilized image generator configured to generate a stabilized image through a deformation process to the second frame image or a frame image generated subsequent to the second frame image by utilizing the image stabilization parameter.

Claims

1. An image capturing apparatus comprising:

an image pickup system configured to take frame images including a past frame image and a current frame image;

a shake detection sensor configured to detect a shake of the image capturing apparatus at the time of the current frame image, and thereby generate shake information that represents the shake, wherein the shake information includes a pitch direction component and a yaw direction component;

a geometric converter configured to perform a geometric conversion process on the past frame image based on the shake information to obtain a geometric conversion image;

a motion vector detector configured to detect a motion vector between the geometric conversion image and the current frame image that has not undergone the geometric conversion process, wherein the motion vector includes shake components except the pitch and yaw direction components;

an image stabilization parameter calculator configured to calculate an image stabilization parameter for motion information by using the shake information and the motion vector; and

a stabilized image generator configured to generate a stabilized image through a deformation process to the current frame image or a frame image generated subsequent to the current frame image by utilizing the image stabilization parameter.

2. The image capturing apparatus according to claim 1, wherein the image stabilization parameter calculator calculates the image stabilization parameter to a rotation by using the shake information, and the image stabilization parameter to a translation by using the motion vector.

3. An image processing method of controlling an image capturing apparatus which comprises an image pickup system configured to take frame images including a past frame image and a current frame image, the method comprising:

detecting a shake of the image capturing apparatus at the time of the current frame image by a shake detection sensor, and thereby generating shake information that represents the shake, wherein the shake information includes a pitch direction component and a yaw direction component;

performing a geometric conversion process on the past frame image based on the shake information to obtain a geometric conversion image;

detecting a motion vector between the geometric conversion image and the current frame image that has not undergone the geometric conversion process, wherein the motion vector includes shake components except the pitch and yaw direction components;

calculating an image stabilization parameter for motion information by using the shake information and the motion vector; and

generating a stabilized image through a deformation process to the current frame image or a frame image generated subsequent to the current frame image by utilizing the image stabilization parameter.

4. An image capturing apparatus comprising:

an image pickup system configured to take frame images including a past frame image and a current frame image;

a shake detection sensor configured to detect a shake of the image capturing apparatus at the time of the current frame image and thereby generate shake information that represents the shake, wherein the shake information includes a pitch direction component and a yaw direction component;

a memory for storing an image distortion component corresponding to a geometric distortion that occurs by an optical system used for imaging;

a distortion reducer configured to provide a distortion reduction process that reduces an image distortion component in the past frame image and the current frame image based on the image distortion component stored in the memory;

a geometric converter configured to perform a geometric conversion process on the past frame image that has already undergone the distortion reduction process based on the shake information to obtain a geometric conversion image;

a motion vector detector configured to detect a motion vector between the past frame image that has undergone both the distortion reduction process and the geometric conversion process and the current frame image that has undergone the distortion reduction process and has not undergone the geometric conversion process, wherein the motion vector includes shake components except the pitch and yaw direction components;

an image stabilization parameter calculator configured to calculate an image stabilization parameter for motion information by using the shake information and the motion vector; and

a stabilized image generator configured to generate a stabilized image through a deformation process to the current frame image that has undergone the distortion reduction process and has not undergone the geometric conversion process or to a frame image generated subsequent to the current frame image by utilizing the image stabilization parameter.