Image pickup apparatus, image pickup method, exposure control method, and program

• US 8,885,093 B2
• Filed: 01/17/2008
• Issued: 11/11/2014
• Est. Priority Date: 03/14/2007
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Asoma, Akira
• Primary Examiner(s): Berhan, Ahmed A
• Application Number:
  US12/015,815
• Publication Number:
  US 20080284872A1
• Time in Patent Office: 2,490 Days
• Field of Search: 348/362, 348/364, 348/365, 348/367, 348/345, 348/220.1, 348/221.1, 348/223.1, 348/229.1, 348/36, 348/217.1, 348/235
• US Class Current: 348/364
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors

Abstract

An image pickup apparatus includes: an image pickup unit configured to generate a long-exposure image signal and a short-exposure image signal from incident light and output the generated signals as image pickup signals, the long-exposure image signal being generated by exposure for a relatively long period and the short-exposure image signal being generated by exposure for a relatively short period; a signal processing unit configured to generate a combined image signal by combining the long-exposure image signal and the short-exposure image signal, the combined image signal having a dynamic range that is relatively wider than that of at least any one of the long-exposure image signal and the short-exposure image signal; a detection unit configured to obtain a luminance integrated value and a luminance histogram of the combined image signal; and a control unit configured to perform exposure correction control using the luminance integrated value and the luminance histogram.

Claims

1. An image pickup apparatus comprising:

an image pickup unit configured to generate a long-exposure image signal and a short-exposure image signal from incident light and output the long-exposure image signal and the short-exposure image signal as image pickup signals, the long-exposure image signal being generated by a long exposure period and the short-exposure image signal being generated by a short exposure period, the long exposure period being longer than the short exposure period;

a signal processing unit configured to generate a combined image signal by combining the long-exposure image signal and the short-exposure image signal based on a predetermined pixel value switching point, the combined image signal having pixel values less than the predetermined pixel value switching point and greater than the predetermined pixel value switching point, and having a dynamic range that is wider than that of at least any one of the long-exposure image signal and the short-exposure image signal;

a detection unit configured to obtain a luminance integrated value of the combined image signal and to generate a luminance histogram of the combined image signal; and

a control unit configured to perform exposure correction control for the image pickup unit based on the luminance integrated value and the luminance histogram.

2. The image pickup apparatus according to claim 1, wherein the control unit detects occurrence of underexposure in the combined image signal using the luminance histogram, sets a target luminance integrated value on the basis of a detection result, and performs the exposure correction control for the image pickup unit using the target luminance integrated value.

3. The image pickup apparatus according to claim 2, wherein the control unit performs the exposure correction control for the long-exposure image signal generated by the image pickup unit using the target luminance integrated value.

4. The image pickup apparatus according to claim 3, wherein the control unit performs iris correction, long-exposure period correction, or gain correction for the long-exposure image signal as the exposure correction control for the long-exposure image signal.

5. The image pickup apparatus according to claim 2, wherein the control unit sets a new target luminance integrated value by performing computation using a current target luminance integrated value and a fixed value.

6. The image pickup apparatus according to claim 2, wherein the control unit calculates a difference value required to obtain a convergence target value of the target luminance integrated value, and sets a new target luminance integrated value by performing computation using the current target luminance integrated value and the difference value.

7. The image pickup apparatus according to claim 2,

wherein the detection unit classifies each of luminance levels of the luminance histogram into a black level, an intermediate level, or a white level and generates luminance distribution information about each of these levels, and

wherein the control unit uses the luminance distribution information about the intermediate level for setting of the target luminance integrated value.

8. The image pickup apparatus according to claim 1, wherein the control unit detects occurrence of overexposure in the combined image signal using the luminance histogram, sets a short-exposure period on the basis of a detection result, and performs the exposure correction control for the short-exposure image signal generated by the image pickup unit using the short-exposure period.

9. The image pickup apparatus according to claim 8, wherein the control unit sets a new short-exposure period by performing computation using a current short-exposure period and a fixed value.

10. The image pickup apparatus according to claim 8, wherein the control unit calculates a difference value required to obtain a convergence target value of the short-exposure period, and sets the new short-exposure period by performing computation using the current short-exposure period and the difference value.

11. The image pickup apparatus according to claim 8,

wherein the detection unit classifies each of luminance levels of the luminance histogram into a black level, an intermediate level, or a white level and generates luminance distribution information about each of these levels, and

wherein the control unit uses the luminance distribution information about the intermediate level for setting of the short-exposure period.

12. The image pickup apparatus according to claim 1, wherein the control unit performs both of the exposure correction control for elimination of an underexposed portion and the exposure correction control for elimination of an overexposed portion in each image signal unit period.

13. The image pickup apparatus according to claim 1, wherein the control unit performs any one of the exposure correction control for elimination of an underexposed portion and the exposure correction control for elimination of an overexposed portion in each image signal unit period.

14. The image pickup apparatus according to claim 13, wherein the control unit determines whether an exposure correction operation for elimination of an underexposed portion has been stably performed in each image signal unit period, performs the exposure correction control for elimination of an underexposed portion when it is determined that the exposure correction operation has not been stably performed, and performs the exposure correction control for elimination of an overexposed portion when it is determined that the exposure correction operation has been stably performed.

15. The image pickup apparatus according to claim 13, wherein the control unit counts the number of times the exposure correction control for elimination of an underexposed portion was performed, and performs the exposure correction control for elimination of an overexposed portion when the counted number exceeds a predetermined number.

16. The image pickup apparatus according to claim 1, wherein the combined image signal includes pixels from the long-exposure image signal below the predetermined pixel value switching point and pixels from the short-exposure image signal above the predetermined pixel value switching point.

17. The image pickup apparatus according to claim 1, wherein the predetermined pixel value switching point is a luminance value.

18. An image pickup method, implemented as an image processing apparatus, comprising:

generating a long-exposure image signal by exposure for a long period and a short-exposure image signal by exposure for a short period, the long period being longer than the short period, from incident light transmitted from a subject so as to output the long-exposure image signal and the short-exposure image signal as image pickup signals;

generating a combined image signal by combining the long-exposure image signal and the short-exposure image signal based on a predetermined pixel value switching point, the combined image signal having pixel values less than the predetermined pixel value switching point and greater than the predetermined pixel value switching point, and having a dynamic range that is wider than that of at least any one of the long-exposure image signal and the short-exposure image signal;

obtaining a luminance integrated value of the combined image signal and generating a luminance histogram of the combined image signal; and

performing exposure correction control based on the luminance integrated value and the luminance histogram.

19. An exposure correction method for an image pickup apparatus for obtaining a long-exposure image signal by exposure for a long period and a short-exposure image signal by exposure for a short period, the long period being longer than the short period, generating a combined image signal with a dynamic range that is wider than that of at least any one of the long-exposure image signal and the short-exposure image signal by combining the long-exposure image signal and the short-exposure image signal based on a predetermined pixel value switching point, and outputting the generated combined image signal, the exposure correction method comprising:

obtaining a luminance integrated value of the combined image signal and generating a luminance histogram of the combined image signal, the combined image signal including pixel values less than the predetermined pixel value switching point and greater than the predetermined pixel value switching point;

detecting occurrence of underexposure in the combined image signal from the luminance histogram;

setting a target luminance integrated value on the basis of a detection result of the occurrence of the underexposure;

detecting occurrence of overexposure in the combined image signal from the luminance histogram;

setting a short-exposure period on the basis of a detection result of the occurrence of the overexposure; and

performing exposure correction control using the target luminance integrated value and the short-exposure period.

20. A non-transitory computer readable storage medium having stored thereon a computer program that includes instructions for causing an image pickup apparatus to execute exposure correction, the image pickup apparatus being configured to obtain a long-exposure image signal by exposure for a long period and a short-exposure image signal by exposure for a short period, generate a combined image signal with a dynamic range that is wider than that of at least any one of the long-exposure image signal and the short-exposure image signal by combining the long-exposure image signal and the short-exposure image signal based on a predetermined pixel value switching point, and output the generated combined image signal, which when executed by the image pickup apparatus causes a processor to perform an exposure correction method comprising:

obtaining a luminance integrated value of the combined image signal and generating a luminance histogram of the combined image signal, the combined image signal including pixel values less than the predetermined pixel value switching point and greater than the predetermined pixel value switching point;

detecting occurrence of underexposure in the combined image signal from the luminance histogram;

setting a target luminance integrated value on the basis of a detection result of the occurrence of the underexposure;

detecting occurrence of overexposure in the combined image signal from the luminance histogram;

setting a short-exposure period on the basis of a detection result of the occurrence of the overexposure; and

performing exposure correction control using the target luminance integrated value and the short-exposure period.