Image processing device, imaging device, image processing method, and program

• US 10,298,899 B2
• Filed: 09/26/2016
• Issued: 05/21/2019
• Est. Priority Date: 03/31/2014
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Fujiwara, Shinya
• Primary Examiner(s): Chen, Chia Wei A
• Application Number:
  US15/276,251
• Publication Number:
  US 20170013242A1
• Time in Patent Office: 967 Days
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/632 for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 23/76 by influencing the image signals
• CPC Class Codes: H04N 23/88 for colour balance, e.g. white-balance circuits or colour temperature control

Abstract

The number of light sources of original image data and the types of light source are determined (S11). A reference white balance gain which is set for each light source type of the original image data is acquired (S12). An influence rate of each light source type is acquired for each pixel of the original image data and a mixture ratio of the reference white balance gains is acquired on the basis of the influence rate (S13). The reference white balance gains and the mixture ratio are stored in a storage medium so as to be associated with the original image data (S14). It is possible to acquire the white balance gain for each pixel required for multi-area white balance processing for the original image data from the reference white balance gains and the mixture ratio stored in the storage medium subsequently.

Claims

1. An image processing device comprising:

a processor configured to;

determine a number of light sources in original image data and the types of each light source;

acquire a reference white balance gain which is set for each light source type of the original image data;

acquire an influence rate of each light source type for each pixel of the original image data and acquire a mixture ratio of the reference white balance gains on the basis of the influence rate; and

store the reference white balance gains and the mixture ratio in a storage medium so as to be associated with the original image data.

2. The image processing device according to claim 1,

wherein the processor is further configured to:

determine that the light sources of the original image data are flash light and environmental light,

acquire a reference white balance gain for flash light which is set in a case in which the light source type is the flash light and a reference white balance gain for environmental light which is set in a case in which the light source type is the environmental light,

acquire the influence rate of each of the flash light and the environmental light for each pixel of the original image data and acquire the mixture ratio of the reference white balance gain for flash light and the reference white balance gain for environmental light on the basis of the influence rate, and

store the reference white balance gain for flash light, the reference white balance gain for environmental light, and the mixture ratio in the storage medium so as to be associated with the original image data.

3. The image processing device according to claim 2, wherein the processor is further configured to:

acquire flash emission image data which is captured while flash light is emitted and flash non-emission image data which is captured when flash light is not emitted, and

acquire the influence rate of each of the flash light and the environmental light from the flash emission image data and the flash non-emission image data and acquire the mixture ratio of the reference white balance gain for flash light and the reference white balance gain for environmental light on the basis of the influence rate.

4. The image processing device according to claim 3,

wherein the processor is further configured to acquire a first brightness value of each pixel of the flash non-emission image data, acquire a second brightness value of each pixel of the flash emission image data, acquire the influence rate of each of the flash light and the environmental light on the basis of the first brightness value and the second brightness value, and acquire the mixture ratio of the reference white balance gain for flash light and the reference white balance gain for environmental light on the basis of the influence rate.

5. The image processing device according to claim 3,

wherein the original image data is the flash emission image data.

6. The image processing device according to claim 1, wherein the processor is further configured to:

acquire first image data indicating an influence of a first light source type and second image data indicating an influence of a second light source type,

wherein, in a case in which the light source types of the original image data determined by the processor include the first light source type and the second light source type,

the processor is further configured to;

acquire the reference white balance gain set for the first light source type and the reference white balance gain set for the second light source type,

acquire the influence rate of the first light source type and the second light source type from the first image data and the second image data and acquire the mixture ratio of the reference white balance gain set for the first light source type and the reference white balance gain set for the second light source type on the basis of the influence rate, and

store the reference white balance gain set for the first light source type, the reference white balance gain set for the second light source type, and the mixture ratio in the storage medium so as to be associated with the original image data.

7. The image processing device according to claim 1,

wherein, regardless of a white balance setting mode when the original image data is acquired,

the processor is further configured to:

determine the number of light sources of the original image data and the types of light source,

acquire the reference white balance gain set for each light source type,

acquire the influence rate of each light source type for each pixel of the original image data and acquire the mixture ratio of the reference white balance gains on the basis of the influence rate, and

store the reference white balance gains and the mixture ratio in the storage medium so as to be associated with the original image data.

8. The image processing device according to claim 7,

wherein the white balance setting mode is any one of a preset white balance mode in which the white balance gain is preset, an auto white balance mode in which the white balance gain applied to the original image data is determined on the basis of color distribution information of the original image data, and a custom white balance mode in which the white balance gain applied to the original image data is determined on the basis of color distribution information of reference image data which is different from the original image data.

9. The image processing device according to claim 1, wherein the processor is further configured to:

determine a white balance setting mode when the original image data is acquired,

wherein, only in a case in which the white balance setting mode determined by the processor is an auto white balance mode in which a white balance gain applied to the original image data is determined on the basis of color distribution information of the original image data,

the processor is further configured to;

determine the number of light sources of the original image data and the types of light source,

acquire the reference white balance gain set for each light source type,

acquire the influence rate of each light source type for each pixel of the original image data and acquire the mixture ratio of the reference white balance gains on the basis of the influence rate, and

store the reference white balance gains and the mixture ratio in the storage medium so as to be associated with the original image data.