Image pickup apparatus that carries out noise compensation, control method therefor, and storage medium

• US 9,013,597 B2
• Filed: 07/20/2012
• Issued: 04/21/2015
• Est. Priority Date: 08/26/2011
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Okubo, Toshiyuki
• Primary Examiner(s): Haskins, Twyler
• Assistant Examiner(s): TEJANO, DWIGHT ALEX C
• Application Number:
  US13/554,420
• Publication Number:
  US 20130050538A1
• Time in Patent Office: 1,005 Days
• Field of Search: 3482221-2251, 348/229.1, 348/230.1
• US Class Current: 348/229.1
• CPC Class Codes: G06T 2200/21 involving computational photography
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10044 Radar image
• CPC Class Codes: G06T 2207/10144 Varying exposure
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors

Abstract

An image pickup apparatus which is capable of accurately compensating for noise even when images are continuously taken at varying exposures. First and second image signals corresponding to the first and second exposures are output. The first and second image signals are subjected to a noise compensation process and then synthesized together to obtain a composite image signal. Data obtained in the first and second image signals when the noise compensation process was carried out on the first and second image signal is inherited, and a noise compensation process on the next first and second signals is carried out based on the data.

Claims

1. An image pickup apparatus having an image pickup device that obtains an image signal, comprising:

an exposure control unit configured to make the image pickup device output first and second image signals corresponding to the first and second exposures differing from each other;

a compensation processing unit configured to perform a noise compensation process on the first and second image signals; and

a synthesizing unit configured to synthesize the first and second image signals after the noise compensation process to obtain a composite image signal,

wherein said compensation processing unit inherits first data obtained in the first image signal when the noise compensation process was carried out on the first image signal, and carries out a noise compensation process on next first image signal based on the first data, and

wherein said compensation processing unit inherits second data obtained in the second image signal when the noise compensation process was carried out on the second image signal, and carries out a noise compensation process on next second image signal based on the second data,

wherein each of the first and second image signals has a plurality of successive frames,

wherein the image pickup apparatus further comprises a determination unit configured to, when gains of the first and second image signals differ, determine that the first data obtained in the previous frame of the first image signal is to be used for the noise compensation process for the next frame of the first image signal and the second data obtained in the previous frame of the second image signal is to be used for the noise compensation process for the next frame of the second image signal, and

wherein when gains of the first and second image signals are the same, said determination unit determines that data obtained by the noise compensation process on the previous frame of the first signal is to be used for the noise compensation process on the next frame of the second image signal, and data obtained by the noise compensation process on the previous frame of the second signal is to be used for the noise compensation process on the next frame of the first image signal.

2. An image pickup apparatus according to claim 1, wherein at the time of synthesizing the first and second image signals after the noise compensation process, said synthesizing unit synthesizes the first and second image signals based on a synthesis ratio determined according to a pixel level of the first image signal.

3. An image pickup apparatus according to claim 1, wherein at the time of carrying out the noise compensation process on the first frame of the second image signal, said determination unit determines that data obtained by converting data obtained in the first frame of the first image signal according to a gain difference between the first and second image signals.

4. An image pickup apparatus according to claim 3, wherein said determination unit further comprises a weighting unit that weights the gain difference and the data obtained in the first frame of the first image signal according to frame rates of the first and second image signals.

5. An image pickup apparatus according to claim 1, wherein the noise compensation process is cyclic type.

6. An image pickup apparatus according to claim 1, wherein said exposure control unit makes the image pickup device outputs the first and second image signals at respective predetermined cycles.

7. An image pickup apparatus according to claim 1, wherein the composite image signal is HDR image signal.

8. An image pickup apparatus according to claim 1, wherein the first and second exposures differ from each other in gain.

9. A control method for an image pickup apparatus having an image pickup device that obtains an image signal, comprising:

an exposure control step of making the image pickup device output first and second image signals corresponding to the first and second exposures differing from each other, wherein each of the first and second image signals has a plurality of successive frames;

a compensation processing step of performing a noise compensation process on the first and second image signals; and

a synthesizing step of synthesizing the first and second image signals after the noise compensation process to obtain a composite image signal,

wherein in said compensation processing step, first data obtained in the first image signal when the noise compensation process was carried out on the first image signal is inherited, and a noise compensation process on next first image signal is carried out based on the first data, and

wherein in said compensation processing step, second data obtained in the second image signal when the noise compensation process was carried out on the second image signal is inherited, and a noise compensation process on next second image signal is carried out based on the second data, and

a determination step of, when gains of the first and second image signals differ, determining that the first data obtained in the previous frame of the first image signal is to be used for the noise compensation process for the next frame of the first image signal and the second data obtained in the previous frame of the second image signal is to be used for the noise compensation process for the next frame of the second image signal, and, when gains of the first and second image signals are the same, determining that data obtained by the noise compensation process on the previous frame of the first signal is to be used for the noise compensation process on the next frame of the second image signal, and data obtained by the noise compensation process on the previous frame of the second signal is to be used for the noise compensation process on the next frame of the first image signal.

10. A non-transitory computer-readable storage medium storing a control program for implementing a control method of controlling an image pickup apparatus having an image pickup device that obtains an image signal corresponding to an optical image obtained by shooting a subject, the control method comprising:

an exposure control step of making the image pickup device output first and second image signals corresponding to the first and second exposures differing from each other, wherein each of the first and second image signals has a plurality of successive frames;

a compensation processing step of performing a noise compensation process on the first and second image signals; and

a synthesizing step of synthesizing the first and second image signals after the noise compensation process to obtain a composite image signal,

wherein in said compensation processing step, first data obtained in the first image signal when the noise compensation process was carried out on the first image signal is inherited, and a noise compensation process on next first image signal is carried out based on the first data, and

wherein in said compensation processing step, second data obtained in the second image signal when the noise compensation process was carried out on the second image signal is inherited, and a noise compensation process on next second image signal is carried out based on the second data, and

a determination step of, when gains of the first and second image signals differ, determining that the first data obtained in the previous frame of the first image signal is to be used for the noise compensation process for the next frame of the first image signal and the second data obtained in the previous frame of the second image signal is to be used for the noise compensation process for the next frame of the second image signal, and, when gains of the first and second image signals are the same, determining that data obtained by the noise compensation process on the previous frame of the first signal is to be used for the noise compensation process on the next frame of the second image signal, and data obtained by the noise compensation process on the previous frame of the second signal is to be used for the noise compensation process on the next frame of the first image signal.