Image pickup device and image pickup method

• US 8,040,411 B2
• Filed: 05/19/2009
• Issued: 10/18/2011
• Est. Priority Date: 06/03/2008
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Nakajima, Keisuke, Yamamoto, Hiroshi
• Primary Examiner(s): Henn; Timothy J
• Application Number:
  US12/453,683
• Publication Number:
  US 20090295941A1
• Time in Patent Office: 882 Days
• Field of Search: 348/222.1, 348254-256, 348/362, 382/274
• US Class Current: 348/254
• CPC Class Codes: G06T 2207/10144 Varying exposure
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: H04N 25/589 with different integration times, e.g. short and long exposures

Abstract

An image pickup device includes an image pickup section that picks up a long-time exposure image and short-time exposure image; a combining section that combines the long-time exposure image and short-time exposure image with each other, to generate a combined image, the exposure images being combined on the basis of a ratio between the exposure time of the long-time exposure image and the exposure time of the short-time exposure image; a dynamic range calculating section that calculates an object dynamic range from the long-time exposure image, the exposure time of the long-time exposure image, the short-time exposure image, and the exposure time of the short-time exposure image; and a compressing section that, using base compression curve data, calculates a compression curve suitable for the object dynamic range, and generates a compressed image in accordance with the calculated compression curve.

Claims

1. An image pickup device comprising at least:

an image pickup section that picks up a long-time exposure image having a relatively long exposure time in unit period and a short-time exposure image having a relatively short exposure time in the unit period;

a combining section that combines the long-time exposure image and the short-time exposure image, obtained by the image pickup section, with each other, to generate a combined image, the exposure images being combined with each other on the basis of a ratio between the exposure time of the long-time exposure image and the exposure time of the short-time exposure image;

a dynamic range calculating section that calculates a dynamic range of an object for the combined image generated at the combining section from the long-time exposure image used in the combining operation, the exposure time of the long-time exposure image, the short-time exposure image used in the combining operation, and the exposure time of the short-time exposure image;

a storage section that holds base compression curve data corresponding to a few types of dynamic ranges; and

a compressing section that, using the base compression curve data held in the storage section, calculates a compression curve suitable for the dynamic range of the object calculated by the dynamic range calculating section, and that generates a compressed image in accordance with the calculated compression curve, the compressed image being generated by compressing the combined image generated by the combining section,

wherein an image subjected to a gradation compression operation in accordance with the dynamic range of the object is generated from the long-time exposure image and the short-time exposure image obtained by the image pickup section.

2. The image pickup device according to claim 1, wherein the dynamic range calculating section previously determines a maximum value of the dynamic range of the object that is capable of being represented by the long-time exposure image, and multiplies the ratio between the exposure time of the long-time exposure image and the exposure time of the short-time exposure image, to calculate the dynamic range of the object picked up by the image pickup section.

3. The image pickup device according to claim 1, wherein the dynamic range calculating section calculates a proportion with respect to a saturation level of the long-time exposure image from a maximum brightness value of the long-time exposure image, and multiplies the proportion with the dynamic range of the object that is capable of being represented by the long-time exposure image, to calculate the dynamic range of the object picked up by the image pickup section.

4. The image pickup device according to claim 3, wherein, using a histogram, formed from the obtained long-time exposure image, and brightness integration values corresponding thereto, the dynamic range calculating section detects the brightness integration value and the number of pixels in the histogram including a specified nth pixel when counting the integration values in order of a larger brightness value to a smaller brightness value, and divides the brightness integration value in an area thereof by the number of pixels, to calculate an average brightness value that is set as the maximum brightness value of the long-time exposure image.

5. The image pickup device according to claim 1, wherein the compressing section includes compression curve generating means for selecting and reading out two types of the base compression curve data that are closest to the dynamic range of the object, and performing interpolation in terms of the two types of base compression curve data on the basis of the dynamic range of the object, to generate a suitable compression curve, and

wherein, in accordance with the compression curve generated by the compression curve generating means, the compressing section generates the compressed image formed by compressing the combined image generated by the combining section.

6. The image pickup device according to claim 1, further comprising:

a contour correcting section including combined image high-frequency component detecting means for detecting a high-frequency component of the combined image generated by the combining section;

gain controlling means for performing gain control of the high-frequency component of the combined image by applying gain to the high-frequency component of the combined image detected by the combined image high-frequency component detecting means, the gain being in accordance with a brightness level of the combined image or a brightness level of the compressed image generated by the compressing section; and

high-frequency component adding means for adding the high-frequency component of the combined image subjected to the gain control by the gain controlling means to the compressed image generated by the compressing means.

7. The image pickup device according to claim 1, further comprising:

a contour correcting section including long-time exposure high-frequency component detecting means for detecting a high-frequency component of the long-time exposure image used in the combining operation at the combining section;

short-time exposure high-frequency component detecting means for detecting a high-frequency component of the short-time exposure image used in the combining operation at the combining section; and

high-frequency component adding means for adding to the compressed image the long-time exposure high-frequency component detected by the long-time exposure high-frequency component detecting means or the short-time exposure component detected by the short-time exposure high-frequency component detecting means so as to correspond to a pixel used from the long-time exposure image or the short-time exposure image, combined by the combining section.

8. The image pickup device according to claim 7, wherein the contour correcting section includes gain controlling means for performing gain control on the high-frequency component, which is added to the compressed image by the high-frequency component adding means, by applying gain to the high-frequency component, the gain corresponding to a brightness level of the combined image generated by the combining section or a brightness level of the compressed image generated by the compressing section.

9. An image pickup method comprising at least the steps of:

picking up a long-time exposure image having a relatively long exposure time and a short-time exposure image having a relatively short exposure time with an image pickup section;

generating a combined image by combining the obtained long-time exposure image and short-time exposure image with each other on the basis of a ratio between the exposure time of the long-time exposure image and the exposure time of the short-time exposure image;

calculating a dynamic range of an object from the long-time exposure image used in the combining operation, the exposure time of the long-time exposure image, the short-time exposure image used in the combining operation, and the exposure time of the short-time exposure image; and

generating a compressed image by calculating a compression curve suitable for the calculated dynamic range of the object using base compression curve data held in a storage section, and by compressing the combined image, generated in the step of generating the combined image, in accordance with the calculated compression curve,

wherein an image subjected to a gradation compression operation in accordance with the dynamic range of the object is generated from the long-time exposure image and the short-time exposure image obtained by the image pickup step.