Image processing apparatus, image processing method, and storage medium

• US 10,298,889 B2
• Filed: 06/20/2017
• Issued: 05/21/2019
• Est. Priority Date: 06/24/2016
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Sekine, Hisato
• Primary Examiner(s): Pasiewicz, Daniel M
• Application Number:
  US15/627,641
• Publication Number:
  US 20170374293A1
• Time in Patent Office: 700 Days
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10144 Varying exposure
• CPC Class Codes: G06T 2207/20212 Image combination
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: H04N 1/3871 the composed originals being of different kinds, e.g. low- and high-resolution originals
• CPC Class Codes: H04N 1/407 Control or modification of tonal gradation or of extreme levels, e.g. background level
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/743 Bracketing, i.e. taking a series of images with varying exposure conditions
• CPC Class Codes: H04N 23/82 for controlling camera response irrespective of the scene brightness, e.g. gamma correction
• CPC Class Codes: H04N 23/841 to modify gamut
• CPC Class Codes: H04N 25/587 acquired sequentially, e.g. using the combination of odd and even image fields
• CPC Class Codes: H04N 5/202 Gamma control circuits for controlling camera response irrespective of the scene brightness H04N23/82

Abstract

An image processing apparatus that generates a combined image by performing gamma processing and combination processing for a plurality of images obtained by capturing images of the same image capturing-target scene under different exposure conditions and includes: an acquisition unit configured to acquire adjustment parameters that adjust gamma characteristics applied to the gamma processing in accordance with a dynamic range of the image capturing-target scene; a gamma processing unit configured to perform the gamma processing to which adjustment parameters acquired by the acquisition unit have been applied for the plurality of images; and a combination unit configured to generate the combined image by performing the combination processing for the plurality of images for which the gamma processing has been performed by the gamma processing unit.

Claims

1. An image processing apparatus that generates a combined image by performing gamma processing and combination processing for a plurality of images obtained by capturing images of the same image capturing-target scene under different exposure conditions, the image processing apparatus comprising:

an acquisition unit configured to acquire adjustment parameters that adjust gamma characteristics applied to the gamma processing in accordance with a dynamic range of the image capturing-target scene;

a gamma processing unit configured to perform the gamma processing to which adjustment parameters acquired by the acquisition unit have been applied for the plurality of images; and

a combination unit configured to generate the combined image by performing the combination processing for the plurality of images for which the gamma processing has been performed by the gamma processing unit.

2. The image processing apparatus according to claim 1, further comprising:

a preliminary gamma processing unit configured to perform preliminary gamma processing based on predetermined gamma characteristics for the plurality of images; and

a preliminary combination unit configured to combine the plurality of images for which the preliminary gamma processing has been performed, wherein

the acquisition unit calculates, by taking combination results by the preliminary combination unit as a reference image, the adjustment parameters based on the reference image.

3. The image processing apparatus according to claim 1, wherein

the plurality of images is images captured in one frame of successive frames of a moving image, and

the acquisition unit calculates, by taking the combined image generated for the previous frame of the one frame as a reference image, the adjustment parameters based on the reference image.

4. The image processing apparatus according to claim 2, further comprising:

a calculation unit configured to calculate a maximum value and a minimum value of a pixel of the reference image, wherein

the acquisition unit calculates the adjustment parameters based on a maximum value and a minimum value calculated by the calculation unit.

5. The image processing apparatus according to claim 4, wherein

the adjustment parameters adjust the gamma characteristics so that the larger a difference between the maximum value and the minimum value, the smaller a slope of an output value for an input value in the gamma processing is made, and the larger the minimum value, the larger a correction constant that reduces an output value on the whole in the gamma processing is made.

6. The image processing apparatus according to claim 4, wherein

the maximum value and the minimum value are calculated for each of luminance, R, G, and B.

7. The image processing apparatus according to claim 4, wherein

the calculation unit calculates histograms of the reference image and calculates the maximum value and the minimum value by using calculated histograms.

8. The image processing apparatus according to claim 1, wherein

the combination unit changes a composite coefficient of each image in the combination processing based on the adjustment parameters and performs combination processing in accordance with a dynamic range of the image capturing-target scene.

9. The image processing apparatus according to claim 1, wherein

the combination processing is HDR combination.

10. The image processing apparatus according to claim 1, wherein

the plurality of images includes at least a first image captured with a relatively short exposure time and a second image captured with a relatively long exposure time.

11. The image processing apparatus according to claim 1, wherein

the gamma processing is log gamma correction on a YUV color space, and

in a case where a pixel value in a YUV color space of the image is taken to be x, a pixel value in a YUV color space after gamma correction is taken to be y, and a coefficient set in advance is taken to be k, adjustment parameters of the gamma processing are set as A and B in y=A·

k·

log(x)−

B.

12. An image processing method of generating a combined image by performing gamma processing and combination processing for a plurality of images obtained by capturing images of the same image capturing-target scene under different exposure conditions, the method comprising the steps of:

acquiring adjustment parameters that adjust gamma characteristics applied to the gamma processing in accordance with a dynamic range of the image capturing-target scene;

performing the gamma processing to which adjustment parameters acquired at the acquisition step have been applied for the plurality of images; and

generating the combined image by performing the combination processing for the plurality of images for which the gamma processing has been performed at the gamma processing step.

13. A non-transitory computer readable storage medium storing a program for causing a computer to function as an image processing apparatus that generates a combined image by performing gamma processing and combination processing for a plurality of images obtained by capturing images of the same image capturing-target scene under different exposure conditions, the image processing apparatus comprising:

an acquisition unit configured to acquire adjustment parameters that adjust gamma characteristics applied to the gamma processing in accordance with a dynamic range of the image capturing-target scene;

a gamma processing unit configured to perform the gamma processing to which adjustment parameters acquired by the acquisition unit have been applied for the plurality of images; and

a combination unit configured to generate the combined image by performing the combination processing for the plurality of images for which the gamma processing has been performed by the gamma processing unit.