Information processing apparatus and method for synthesizing corrected image data

• US 8,379,096 B2
• Filed: 04/21/2010
• Issued: 02/19/2013
• Est. Priority Date: 05/21/2009
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Torii, Satoru, Shindo, Yoshinori
• Primary Examiner(s): Giles, Nicholas
• Assistant Examiner(s): Tissire, Abdelaaziz
• Application Number:
  US12/764,546
• Publication Number:
  US 20100295953A1
• Time in Patent Office: 1,035 Days
• Field of Search: 348/333.12, 348/333.03, 348/208.1, 348/208.14, 348/352, 348/402.1, 348/413.1, 348/416.1, 348/E7.007, 348/E7.008
• US Class Current: 348/208.4
• CPC Class Codes: G06T 2200/21 involving computational photography
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations

Abstract

A plurality of image capturing data time-divisionally exposed by a capturing unit are input. By first and second processes, the plurality of image capturing data are divided into synthesis images or motion detection images, a motion amount of the capturing unit at the time of time-division exposure is detected from the image capturing data of the motion detection images, synthesized image capturing data is generated by synthesizing the synthesis images, and a vibration of the synthesized image capturing data is corrected based on a divide pattern indicating divisions and the motion amount. Corrected image capturing data obtained by the first and second processes are integrated to obtain image capturing data after image stabilization.

Claims

1. An image processing apparatus comprising:

an input section configured to input a plurality of image capturing data time-divisionally exposed by a capturing unit;

first and second image processors each comprising (i) a determiner configured to determine a subset of the plurality of input image capturing data to be synthesis images according to a divide pattern, (ii) a synthesizer configured to generate synthesized image capturing data by synthesizing the synthesis images, and (iii) a corrector configured to correct vibration of the synthesized image capturing data based on the divide pattern and a motion amount of the capturing unit; and

an integrator configured to synthesize corrected image capturing data output from the first and second image processors to obtain stabilized image capturing data.

2. The apparatus according to claim 1, wherein a divide pattern used by the first image processor and a divide pattern used by the second image processor are reversal patterns.

3. The apparatus according to claim 1, wherein the determiner randomly divides the plurality of image capturing data into synthesis images or motion detection images, which are used to detect the motion amount, according to the divide pattern.

4. The apparatus according to claim 1, wherein the corrector comprises (i) a generator configured to generate a point spread function based on the divide pattern and the motion amount and (ii) an arithmetic section configured to execute deconvolution of the synthesized image capturing data using the point spread function.

13. The apparatus according to claim 1, the first and second image processors each further comprises a detector configured to detect the motion amount of the capturing unit at the time of time-division exposure.

14. The apparatus according to claim 13, wherein the determiner divides the plurality of image capturing data into synthesis images or motion detection images according to the divide pattern.

15. The apparatus according to claim 14, wherein the detector detects the motion amount from image capturing data of the motion detection images.

5. An image processing apparatus comprising:

an input section configured to input a plurality of image capturing data time-divisionally exposed by a capturing unit;

a plurality of image processors, each of which comprises (i) a divider configured to divide the plurality of image capturing data into synthesis images or motion detection images according to a divide pattern, (ii) a detector configured to detect a motion amount of the capturing unit at the time of time-division exposure from the image capturing data of the motion detection images, (iii) a synthesizer configured to generate synthesized image capturing data by synthesizing the synthesis images, and (iv) a corrector configured to correct a vibration of the synthesized image capturing data based on the divide pattern and the motion amount; and

an integrator configured to synthesize corrected image capturing data output from the plurality of image processors to obtain stabilized image capturing data.

6. The apparatus according to claim 5, wherein the integrator divides pixel values of the synthesized image capturing data by a sum total of ratios at which the divide pattern of each of the plurality of image processors that indicates a division of the image capturing data into synthesis images.

7. The apparatus according to claim 5, wherein the divider randomly divides the plurality of image capturing data into synthesis images or motion detection images according to the divide pattern.

8. The apparatus according to claim 5, wherein the corrector comprises (i) a generator configured to generate a point spread function based on the divide pattern and the motion amount and (ii) an arithmetic section configured to execute deconvolution of the synthesized image capturing data using the point spread function.

9. An image processing method comprising:

using a processor to perform;

an input step of inputting a plurality of image capturing data time-divisionally exposed by a capturing unit;

first and second processing steps of respectively performing a determination process, a synthetic process, and a correction process, wherein the determination process determines a subset of the plurality of input image capturing data to be synthesis images according to a divide pattern, the synthetic process generates synthesized image capturing data by synthesizing the synthesis images, and the correction process corrects vibration of the synthesized image capturing data based on the divide pattern and a motion amount of the capturing unit; and

a synthesis step of synthesizing corrected image capturing data output in the first and second processing steps to obtain stabilized image capturing data.

10. An image processing method comprising:

using a processor to perform;

an input step of inputting a plurality of image capturing data time-divisionally exposed by a capturing unit;

a plurality of processing steps, each of which performs a divide process, a detective process, a synthetic process, and a correction process, wherein the divide process divides the plurality of image capturing data into synthesis images or motion detection images according to a divide pattern, the detective process detects a motion amount of the capturing unit at the time of time-division exposure from the image capturing data of the motion detection images, the synthetic process generates synthesized image capturing data by synthesizing the synthesis images, and the correction process corrects a vibration of the synthesized image capturing data based on the divide pattern and the motion amount; and

a synthesis step of synthesizing corrected image capturing data output in the plurality of processing steps to obtain stabilized image capturing data.

11. A non-transitory computer-readable medium storing a computer-executable program for causing a computer to perform an image processing method, the method comprising:

an input step of inputting a plurality of image capturing data time-divisionally exposed by a capturing unit;

first and second processing steps of respectively performing a determination process, a synthetic process, and a correction process, wherein the determination process determines a subset of the plurality of input image capturing data to be synthesis images according to a divide pattern, the synthetic process generates synthesized image capturing data by synthesizing the synthesis images, and the correction process corrects vibration of the synthesized image capturing data based on the divide pattern and a motion amount of the capturing unit; and

a synthesis step of synthesizing corrected image capturing data output in the first and second processing steps to obtain stabilized image capturing data.

12. A non-transitory computer-readable medium storing a computer-executable program for causing a computer to perform an image processing method, the method comprising:

an input step of inputting a plurality of image capturing data time-divisionally exposed by a capturing unit;

a plurality of processing steps, each of which performs a divide process, a detective process, a synthetic process, and a correction process, wherein the divide process divides the plurality of image capturing data into synthesis images or motion detection images according to a divide pattern, the detective process detects a motion amount of the capturing unit at the time of time-division exposure from the image capturing data of the motion detection images, the synthetic process generates synthesized image capturing data by synthesizing the synthesis images, and the correction process corrects a vibration of the synthesized image capturing data based on the divide pattern and the motion amount; and

a synthesis step of synthesizing corrected image capturing data output in the plurality of processing steps to obtain stabilized image capturing data.