Image processing apparatus and method

• US 8,189,069 B2
• Filed: 04/08/2008
• Issued: 05/29/2012
• Est. Priority Date: 04/13/2007
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Ogawa, Katsuhisa
• Primary Examiner(s): Nguyen, Luong T
• Assistant Examiner(s): Trehan, Akshay
• Application Number:
  US12/099,563
• Publication Number:
  US 20080252750A1
• Time in Patent Office: 1,512 Days
• Field of Search: 348/218.1, 348/229.1, 348234-237, 348/396.1, 348/668, 382/168, 382/172
• US Class Current: 348/229.1
• CPC Class Codes: H04N 23/667 Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 23/76 by influencing the image signals
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

An image processing apparatus is provided that determines a correction gain factor at a high rate of speed to synthesize an image. The apparatus includes a long second middle luminance image detecting unit for detecting a pixel having a pixel value in a first middle luminance pixel value region derived from a long second exposure image, a short second middle luminance image detecting unit for detecting a pixel having a pixel value in a second middle luminance pixel value region derived from a shorter second exposure image; a correction gain factor calculating unit for designating, as a common pixel, the pixel detected by the long second middle luminance image detecting unit and the pixel detected by the short second middle luminance image detecting unit, each of the pixels at a common position, and for calculating a correction gain factor based on the pixel values of the common pixels.

Claims

1. An image processing apparatus for synthesizing a long second exposure image derived by a long exposure time and a short second exposure image derived by a short exposure time, shorter than the long exposure time, for imaging of the same scene, the apparatus comprising:

a long second middle luminance pixel selecting unit configured to select out a pixel having a pixel luminance value in a first middle luminance region from pixels included in the long second exposure image, based on comparing the pixel luminance value of the pixel in the long second exposure image with a first lower threshold value and with a first upper threshold value;

a short second middle luminance pixel selecting unit configured to select out a pixel having a pixel luminance value in a second middle luminance region from pixels included in the short second exposure image, based on comparing the pixel luminance value of the pixel in the short second exposure image with a second lower threshold value and with a second upper threshold value;

a correction gain factor calculating unit configured to designate, as a common pixel, the pixel selected by the long second middle luminance pixel selecting unit and the pixel selected by the short second middle luminance pixel selecting unit, each of the pixels being at a common position, and to calculate a correction gain factor based on the pixel luminance values of both of the pixels; and

an image synthesizing unit configured to multiply a value of data representing the short second exposure image by the correction gain factor, and to synthesize the short second exposure image, the value of the data of which is subjected to the multiplication, and the long second exposure image.

2. The apparatus according to claim 1, wherein the correction gain factor calculating unit calculates, as the correction gain factor, an average value of the ratio between the pixel luminance value of the long second exposure image and the pixel luminance value of the short second exposure image at the pixel position of the common pixel.

3. The apparatus according to claim 1, wherein the first middle luminance region is within a range of 30 to 70% of a maximum luminance pixel value, and

wherein the second middle luminance region is within a range of 30 to 70% of a maximum luminance pixel value.

4. The apparatus according to claim 1, wherein the first middle luminance region is within a range of 10 to 80% of a maximum luminance pixel value, and

wherein the second middle luminance region is within a range of 20 to 90% of a maximum luminance pixel value.

5. An image processing method for synthesizing a long second exposure image derived by a long exposure time and a short second exposure image derived by a short exposure time, shorter than the long exposure time, for imaging of the same scene, the apparatus comprising:

a long second middle luminance pixel selecting step for selecting out a pixel having a pixel value in a first middle luminance region from pixels included in the long second exposure image, based on comparing the pixel luminance value of the pixel in the long second exposure image with a first lower threshold value and with a first upper threshold value;

a short second middle luminance pixel selecting step for selecting out a pixel having a pixel luminance value in a second middle luminance pixel value region from pixels included in the short second exposure image, based on comparing the pixel luminance value of the pixel in the short second exposure image with a second lower threshold value and with a second upper threshold value;

a correction gain factor calculating step for designating, as a common pixel, the pixel selected by the long second middle luminance pixel selecting step and the pixel selected by the short second middle luminance pixel selecting step, each of the pixels being at a common position, and for calculating a correction gain factor based on the pixel luminance values of both of the pixels; and

an image synthesizing step for multiplying a value of data representing the short second exposure image by the correction gain factor, and for synthesizing the short second exposure image, the value of the data of which is subjected to the multiplication, and the long second exposure image.

6. An image processing apparatus for synthesizing a long second exposure image derived by a long exposure time and a short second exposure image derived by a short exposure time, shorter than the long exposure time, for imaging of the same scene, the apparatus comprising:

a long second middle luminance image detecting unit configured to detect a pixel having a pixel value in a first middle luminance pixel value region from the long second exposure image;

a short second middle luminance image detecting unit configured to detect a pixel having a pixel value in a second middle luminance pixel value region from the short second exposure image;

a correction gain factor calculating unit configured to designate, as a common pixel, the pixel detected by the long second middle luminance image detecting unit and the pixel detected by the short second middle luminance image detecting unit, and to calculate a correction gain factor based on the pixel value of the long second exposure image and the pixel value of the short second exposure image at a pixel position of the common pixel; and

an image synthesizing unit configured to multiply a value of data representing the short second exposure image by the correction gain factor, and to synthesize the short second exposure image, the value of the data of which is subjected to the multiplication, and the long second exposure image,

wherein the numbers of the pixels included in the first and second middle luminance pixel value regions are converted by the apparatus operating in a moving image mode and a still image mode, such that the number of the pixels in the moving image mode is larger than that in the still image mode.

7. An image processing apparatus for synthesizing a long second exposure image derived by a long exposure time and a short second exposure image derived by a short exposure time, shorter than the long exposure time, for imaging of the same scene, the apparatus comprising:

a long second middle luminance pixel selecting unit configured to select out a pixel having a pixel luminance value in a first middle luminance region from pixels included in the long second exposure image;

a short second middle luminance pixel selecting unit configured to select out a pixel having a pixel luminance value in a second middle luminance region from pixels included in the short second exposure image;

a correction gain factor calculating unit configured to designate, as a common pixel, a pixel selected by the long second middle luminance pixel selecting unit and the pixel selected by the short second middle luminance pixel selecting unit, and to calculate a correction gain factor based on the pixel luminance value of the long second exposure image and the pixel luminance value of the short second exposure image at a pixel position of the common pixel; and

an image synthesizing unit configured to multiply a value of data representing the short second exposure image by the correction gain factor, and to synthesize the short second exposure image, the value of the data of which is subjected to the multiplication, and the long second exposure image,

wherein the numbers of the pixels included in the first and second middle luminance pixel value regions are converted by the apparatus operating in a first mode and a second mode, such that the number of the pixels in the first mode is different from that in the second mode.

8. The apparatus according to claim 7, wherein the first mode is a still image mode, and the second mode is a moving image mode, and the number of the pixels included in the first and second middle luminance pixel value regions in the first mode is larger than that in the second mode.

9. The apparatus according to claim 7, wherein the first mode is a moving image mode, and the second mode is a still image mode, and the number of the pixels included in the first and second middle luminance pixel value regions in the first mode is larger than that in the second mode.

10. An image processing apparatus for synthesizing a long-exposure image and a short-exposure image into an image having a wider dynamic range than each of the long-exposure image and the short-exposure image, the long-exposure image being derived by a longer exposure time than the short-exposure image, the apparatus comprising:

a first selecting unit configured to select a pixel having a luminance value within a first middle luminance range from pixels included in the long-exposure image, based on comparing luminance values of the pixels included in the long-exposure image with a first lower threshold value and with a first upper threshold value;

a second selecting unit configured to select a pixel having a luminance value within a second middle luminance range from pixels included in the short-exposure image, based on comparing luminance values of the pixels included in the short-exposure image with a second lower threshold value and with a second upper threshold value;

a calculating unit configured to calculate a gain factor based on the luminance value of the pixel selected by the first selecting unit and the luminance value of the pixel selected by the second selecting unit; and

a synthesizing unit configured to synthesize the long-exposure image and the short-exposure image based on the gain factor calculated by the calculating unit,

wherein the calculating unit omits at least one pixel included in the long-exposure image or the short-exposure image from the calculation of the gain factor.

11. The apparatus according to claim 10, wherein the calculating unit uses only the pixel selected by the first selecting unit or by the second selecting unit.

12. The apparatus according to claim 10, wherein the calculating unit omits a pixel which is not selected by the first selecting unit.

13. The apparatus according to claim 10, wherein the calculating unit omits a pixel which is not selected by the second selecting unit.