IMAGE PROCESSING DEVICE AND PROGRAM

• US 20140204246A1
• Filed: 09/03/2012
• Published: 07/24/2014
• Est. Priority Date: 09/02/2011
• Status: Active Grant

• Current Assignee: Nikon Corporation
• Original Assignee: Nikon Corporation
• Inventors: Nishi, Takeshi
• Granted Patent: US 9,432,641 B2
• Time in Patent Office: Days
• US Class Current: 348/239
• CPC Class Codes: G06T 2207/10144 Varying exposure
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: H04N 23/10 for generating image signals from different wavelengths
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 9/646 for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters
• CPC Class Codes: H04N 9/77 Circuits for processing the brightness signal and the chrominance signal relative to each other, e.g. adjusting the phase of the brightness signal relative to the colour signal, correcting differential gain or differential phase circuits for matrixing H04N9/67

Abstract

An image processing device includes: an identifying unit that identifies an image area where gradations are lost in a first captured image obtained by capturing a subject image; an image texture image generation unit that generates an image texture image by aligning a second captured image, obtained by capturing an image of a subject matching the subject of the first captured image, with the first captured image and extracting an image texture corresponding to the image area having been identified by the identifying unit; and an interpolation unit that executes interpolation for the image area in the first captured image by synthetically incorporating the image texture image having been generated by the image texture image generation unit with the first captured image.

Claims

1. An image processing device, comprising:

an identifying unit that identifies an image area where gradations are lost in a first captured image obtained by capturing a subject image;

an image texture image generation unit that generates an image texture image by aligning a second captured image, obtained by capturing an image of a subject matching the subject of the first captured image, with the first captured image and extracting an image texture corresponding to the image area having been identified by the identifying unit; and

an interpolation unit that executes interpolation for the image area in the first captured image by synthetically incorporating the image texture image having been generated by the image texture image generation unit with the first captured image.

2. An image processing device according to claim 1, wherein:

the image area where gradations are lost includes an image area where at least one of underexposure black-out, overexposure white-out and color saturation has occurred.

3. An image processing device according to claim 2, wherein:

the image area where underexposure black-out has occurred is a pixel cluster made up with at least a predetermined number of contiguous pixels indicating luminance values equal to or less than a first threshold value;

the image area where overexposure white-out has occurred is a pixel cluster made up with at least a predetermined number of contiguous pixels indicating luminance values equal to or greater than a second threshold value; and

the image area where color saturation has occurred is a pixel cluster made up with at least a predetermined number of contiguous pixels indicating chromaticity values equal to or greater than a third threshold value.

4. An image processing device according to claim 3, further comprising:

a threshold value setting unit that alters either a value setting for the first threshold value or a value setting for the second threshold value or both the value setting for the first threshold value and the value setting for the second threshold value, wherein;

the threshold value setting unit;

generates a luminance histogram of the first captured image and divides the luminance histogram into a plurality of groups;

calculates a first difference between an integrated value calculated for a current frame in correspondence to a first group, which includes a lowest grade assuming a frequency value other than 0, among the plurality of groups and an integrated value calculated for a previous frame in correspondence to the group;

calculates a second difference between an integrated value calculated for the current frame in correspondence to a second group, which includes a highest grade assuming a frequency value other than 0, among the plurality of groups and an integrated value calculated for the previous frame in correspondence to the group;

sets a largest value in the first group for the first threshold value if the first difference is equal to or greater than a predetermined value; and

sets a smallest value in the second group for the second threshold value if the second difference is equal to or greater than a predetermined value.

5. An image processing device according claim 1, wherein:

the image texture includes at least image edges.

6. An image processing device according to claim 5, wherein:

the image texture further includes image luminance gradation information.

7. An image processing device according to claim 1, further comprising:

an image capturing unit that captures the first captured image and the second captured image.

8. A computer-readable computer program product containing a control program for image processing, the control program comprising:

an instruction for identifying an image area where gradations are lost, in a first captured image obtained by capturing a subject image;

an instruction for generating an image texture image by aligning a second captured image obtained by capturing an image of a subject matching the subject of the first captured image with the first captured image, and by extracting an image texture corresponding to the image area having been identified; and

an instruction for executing interpolation for the image area in the first captured image by synthetically incorporating the image texture image having been generated and the first captured image.

9. An image processing device, comprising:

an image frame extracting unit that extracts a target image frame requiring exposure correction, among image frames constituting a video image recorded in a recording unit;

an identifying unit that identifies an image area where gradations are lost in the target image frame;

an estimating unit that estimates a texture of the image area where gradations are lost by using an image frame, other than the target image frame, constituting the video image; and

an interpolation unit that executes interpolation for the image area where gradations are lost in the image frame based upon estimation results provided by the estimating unit.

10. An image processing device according to claim 9, wherein:

the identifying unit identifies the image area where gradations are lost based upon luminance distribution within the image frame.

11. An image processing device according to claim 9, wherein:

the estimating unit generates a texture image for the image area where gradations are lost by using an image frame other than the target image frame; and

the interpolation unit executes interpolation by incorporating the texture image in the image area where gradations are lost.

12. An image processing device according to claim 9, further comprising:

an image sensor that captures an image formed with subject light, wherein;

a video image generated based upon image capturing signals provided from the image sensor is recorded in the recording unit.