IMAGING APPARATUS, IMAGING METHOD, INTEGRATED CIRCUIT, AND STORAGE MEDIUM

• US 20080259181A1
• Filed: 04/16/2008
• Published: 10/23/2008
• Est. Priority Date: 04/18/2007
• Status: Active Grant

• Current Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Ito, Takeshi, Sato, Daisuke, YAMASHITA, Haruo
• Granted Patent: US 8,144,214 B2
• Time in Patent Office: Days
• US Class Current: 348/229.1
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 23/72 Combination of two or more compensation controls
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors

Abstract

An imaging apparatus appropriately captures a large dynamic range image of even a scene including a backlit person with a blue sky background in a manner that the person'"'"'s face has an appropriate luminance level without saturating the background sky. In the imaging apparatus, an imaging unit obtains analogue image signals through exposure control that prevents a highlight from being saturated, an A/D converter converts the analogue image signals to digital image signals, and a signal processing unit linearly increases the dynamic range of the digital image signals. The image signals with the increased dynamic range are nonlinearly compressed to have a dynamic range of 100% or less through nonlinear dynamic range compression that intensively compresses a highlight portion. The imaging apparatus with this structure first increases the dynamic range of an image and efficiently compresses the increased large dynamic range of the image.

Claims

1. An imaging apparatus comprising:

an optical system operable to adjust an exposure light amount;

an A/D converter operable to convert an analogue signal that is obtained by the optical system to a digital signal;

a dynamic-range increasing unit operable to magnify an output value of the digital signal with a dynamic range magnification ratio; and

a control unit operable to generate control signals, one for decreasing the exposure light amount and the other for increasing the dynamic range magnification ratio.

2. The imaging apparatus according to claim 1, further comprising:

a dynamic-range compression unit operable to compress the increased value of the digital signal so as to output a value that is within a predetermined range.

3. The imaging apparatus according to claim 2, further comprising:

a peak detection unit operable to detect a peak value of image data corresponding to one image that is formed using the digital signal,

wherein the dynamic-range compression unit compresses the increased value of the digital signal in a manner that the peak value is converted to be a maximum value of the predetermined range.

4. The imaging apparatus according to claim 1, wherein

the control unit generates a signal for adjusting the exposure light amount according to the peak value.

5. The imaging apparatus according to claim 1, further comprising:

a representative value detection unit operable to detect a representative value of a predetermined portion of image data corresponding to one image that is formed using the digital signal,

wherein the control unit generates a signal for adjusting the exposure light amount according to the representative value.

6. The imaging apparatus according to claim 1, further comprising:

a representative value detection unit operable to detect a representative value of a predetermined portion of image data corresponding to one image that is formed using the digital signal,

wherein the control unit generates a signal for adjusting the dynamic range magnification ratio according to the representative value.

7. The imaging apparatus according to claim 1, further comprising:

a face detection processing unit operable to extract a representative value of a digital signal corresponding to a face portion of image data that is formed using the digital signal,

wherein the dynamic-range increasing unit adjusts the dynamic range magnification ratio according to the representative value.

8. The imaging apparatus according to claim 2, wherein

the dynamic-range compression unit includes

a vicinity luminance detection unit operable to extract a vicinity luminance value of a digital signal corresponding to a predetermined position, and

a dynamic tone correction unit operable to convert an input value of a digital signal for a position to an output value of a digital signal for the position by using a conversion characteristic, wherein the conversion characteristic changes an output value according to the vicinity luminance value for the digital signal.

9. The imaging apparatus according to claim 8, wherein

the conversion characteristic changes the value of the digital signal for the position more in the case a value of the digital signal is close to a value of the vicinity luminance value than in the case the value of the digital signal is not close to the value of the vicinity luminance value.

10. The imaging apparatus according to claim 8, wherein

when m=x, where x is the value of the digital signal corresponding to the 10 predetermined position and m is the vicinity luminance value, the conversion characteristic expressed using a quadratic function f(x, m) satisfies

∂

f(x,m)/∂

x≧

(f(x,m)/x).

11. The imaging apparatus according to claim 8, wherein

the dynamic tone correction unit is a two-dimensional lookup table storing, as data, an output value that is determined by the value of the digital signal and the vicinity luminance value.

12. The imaging apparatus according to claim 11, wherein

the control unit generates signals,

one for decreasing the exposure light amount, and

one for increasing the dynamic range increase ratio, and

one for storing conversion characteristic data that is determined by the dynamic range increase ratio into the lookup table.

13. The imaging apparatus according to claim 8, further comprising:

a range adjustment unit operable to receive the digital signal whose dynamic range has been magnified by the dynamic-range magnification unit, and converts the value of the received digital signal to a value that is within an input range of the dynamic range compression unit when the value of the received digital signal is greater than a predetermined value.

14. The imaging apparatus according to claim 8, further comprising:

a peak adjustment unit operable to adjust the value of the digital signal in a manner that a peak value of image data that is formed using the digital signal whose dynamic range has been increased by the dynamic-range increasing unit, is adjusted to a maximum value of an input range of the dynamic-range compression unit.

15. An imaging apparatus comprising:

an optical system operable to focus light from a subject;

a light amount adjustment unit operable to adjust an amount of light that is incident on the optical system;

an imaging unit operable to read an optical image of the subject that is formed with the light focused by the optical system, and output the read optical image as an image signal that is used to form an image including pixels;

an A/D converter operable to convert an analogue image that is output from the imaging unit to a digital image;

a drive unit operable to drive the imaging unit in a manner that the imaging unit obtains a plurality of analogue images through a plurality of sequential divisional exposures each of which is performed for an identical exposure time, a total exposure time of the divisional exposures being equal to a predetermined exposure time; and

a dynamic-range increasing unit operable to increase a dynamic range of corresponding pixels of a plurality of digital images resulting from conversion of the plurality of analogue images performed by the A/D converter and add up the corresponding pixels of the digital images to form an image with an increased dynamic range, and output the image with the increased dynamic range.

16. The imaging apparatus according to claim 15, further comprising:

a dynamic-range compression unit operable to perform dynamic range compression of nonlinearly compressing the increased dynamic range of the image output from the dynamic-range increasing unit to a predetermined dynamic range.

17. The imaging apparatus according to claim 15, wherein

the dynamic-range increasing unit has an output dynamic range that is at least greater than an input dynamic range.

18. The imaging apparatus according to claim 16, wherein

the dynamic-range compression unit compresses the increased dynamic range of the image using a conversion characteristic that changes according to a vicinity luminance value of the image signal.

19. The imaging apparatus according to claim 15, wherein

before the imaging unit is subjected to the plurality of divisional exposures, the light amount adjustment unit adjusts the amount of light that is incident on the optical system in a manner that a main portion of the subject will have a predetermined luminance level through an exposure performed for the predetermined exposure time.

20. The imaging apparatus according to claim 15, further comprising:

an apparatus-movement detection unit operable to detect a degree and a direction of an image shift caused by movement of the imaging apparatus between two different digital images among the plurality of digital images,

wherein the dynamic-range increasing unit shifts the digital images in a direction to eliminate the image shift according to the degree and the direction of the image shift detected by the apparatus-movement detection unit, and add up the corresponding pixels of the digital images.

21. The imaging apparatus according to claim 20, further comprising:

a subject-movement detection unit operable to detect movement of the subject in a local area on the digital images by comparing corresponding pixels of the two different digital images in which the image shift caused by the movement of the imaging apparatus has been eliminated,

wherein the apparatus-movement detection unit generates the plurality of digital images by shifting each digital image in a direction to eliminate the image shift according to an output from a sensor,

wherein when the movement of the subject is detected by the subject-movement detection unit, the dynamic-range increasing unit excludes the corresponding pixels of the different digital images having the movement of the subject from the pixels of the digital images that are added up.

22. The imaging apparatus according to claim 15, further comprising:

a subject-movement detection unit operable to detect movement of the subject in a local area on two different digital images among the plurality of digital images by comparing corresponding pixels of the two different digital images,

wherein when the movement of the subject is detected by the subject-movement detection unit, the dynamic-range increasing unit excludes the corresponding pixels of the different digital images having the movement of the subject from the pixels of the digital images that are added up.

23. The imaging apparatus according to claim 15, further comprising:

a subject-movement detection unit operable to obtain a plurality of digital images in which movement of the imaging apparatus has been corrected, and detect movement of the subject in a local area on the digital images by comparing corresponding pixels of two different digital images among the plurality of digital images,

wherein when the movement of the subject is detected by the subject-movement detection unit, the dynamic-range increasing unit excludes the corresponding pixels of the different digital images having the movement of the subject from the pixels of the digital images that are added up.

24. The imaging apparatus according to claim 23, wherein

the subject-movement detection unit detects the movement of the subject using the plurality of digital images in which movement of the imaging apparatus has been corrected by means of optical correction.

25. The imaging apparatus according to claim 20, further comprising:

a subject-movement detection unit operable to detect movement of the subject in a local area on the digital images by comparing corresponding pixels of the two different digital images in which the image shift caused by the movement of the imaging apparatus has been eliminated; and

a coordinate conversion unit operable to convert coordinates of corresponding pixels of different digital images having the movement of the subject detected by the subject-movement detection unit to eliminate the movement of the subject,

wherein the dynamic-range increasing unit uses, as the pixels of the digital images that are added up, pixels that are at coordinates obtained by the coordinate conversion unit through conversion of the coordinates of the pixels.

26. The imaging apparatus according to claim 25, further comprising:

a subject-movement determination unit operable to determine whether the movement of the subject will be eliminated by converting the coordinates of the pixels,

wherein when the subject-movement determination unit determines that the movement of the subject will not be eliminated by converting the coordinates of the pixels, the dynamic-range increasing unit excludes the pixels having the movement of the subject from the pixels of the digital images that are added up.

27. An imaging apparatus comprising:

an optical system operable to focus light from a subject;

a light amount adjustment unit operable to adjust an amount of light that is incident on the optical system;

an imaging unit operable to read an optical image of the subject that is formed with the light focused by the optical system, and output the read optical image as an image signal that is used to form an image including pixels;

an A/D converter operable to convert an analogue image that is output from the imaging unit to a digital image;

a drive unit operable to drive the imaging unit in a manner that the imaging unit obtains a plurality of analogue images through a plurality of sequential divisional exposures that are performed to prevent an output of the A/D converter from being saturated; and

a dynamic-range increasing unit operable to increase a dynamic range of corresponding pixels of a plurality of digital images resulting from conversion of the plurality of analogue images performed by the A/D converter and add up the corresponding pixels of the digital images to form an image with an increased dynamic range, and output the image with the increased dynamic range.

28. An imaging method used in an imaging apparatus that includes an optical system operable to focus light from a subject, a light amount adjustment unit operable to adjust an amount of light that is incident on the optical system, an imaging unit operable to read an optical image of the subject that is formed with the light focused by the optical system and output the read optical image as an image signal that is used to form an image including pixels, an A/D converter operable to convert an output of the imaging unit through analogue-to-digital conversion, and a digital signal processing unit operable to adjust an output value of the A/D converter,

the imaging method comprising:

decreasing the amount of light using the light amount adjustment unit; and

linearly increasing a dynamic range of an output of the A/D converter with an increase that is set according to the decreased amount of light using the digital signal processing unit.

29. An imaging method used in an imaging apparatus that includes an optical system operable to focus light from a subject, a light amount adjustment unit operable to adjust an amount of light that is incident on the optical system, an imaging unit operable to read an optical image of the subject that is formed with the light focused by the optical system and output the read optical image as an image signal that is used to form an image including pixels, and an A/D converter operable to convert an analogue image that is output from the imaging unit to a digital image, the method comprising:

driving the imaging unit in a manner that the imaging unit obtains a plurality of analogue images through a plurality of sequential divisional exposures each of which is performed for an identical exposure time, a total exposure time of the divisional exposures being equal to a predetermined exposure time; and

increasing a dynamic range of corresponding pixels of a plurality of digital images resulting from conversion of the plurality of analogue images performed by the A/D converter and adding up the corresponding pixels of the digital images to form an image with an increased dynamic range, and outputting the image with the increased dynamic range.

30. An imaging method used in an imaging apparatus that includes an optical system operable to focus light from a subject, a light amount adjustment unit operable to adjust an amount of light that is incident on the optical system, an imaging unit operable to read an optical image of the subject that is formed with the light focused by the optical system and output the read optical image as an image signal that is used to form an image including pixels, and an A/D converter operable to convert an analogue image that is output from the imaging unit to a digital image, the method comprising:

driving the imaging unit in a manner that the imaging unit obtains a plurality of analogue images through a plurality of sequential divisional exposures that are performed to prevent an output of the A/D converter from being saturated; and

increasing a dynamic range of corresponding pixels of a plurality of digital images resulting from conversion of the plurality of analogue images performed by the A/D converter and adding up the corresponding pixels of the digital images to form an image with an increased dynamic range, and outputting the image with the increased dynamic range.

31. A computer-readable storage medium that stores a program executed by a control unit of an imaging apparatus that includes an optical system operable to focus light from a subject, a light amount adjustment unit operable to adjust an amount of light that is incident on the optical system, an imaging unit operable to read an optical image of the subject that is formed with the light focused by the optical system and output the read optical image as an image signal that is used to form an image including pixels, an A/D converter operable to convert an output of the imaging unit through analogue-to-digital conversion, a digital signal processing unit operable to adjust an output value of the A/D converter, and the control unit operable to control the light amount adjustment unit and the digital signal processing unit,

the program enabling the control unit to:

generate a signal for decreasing the light amount using the light amount adjustment unit; and

linearly increase a dynamic range of an output of the A/D converter with an increase that is set according to the decreased amount of light using the digital signal processing unit.

32. A computer-readable storage medium that stores a program executed by a control unit of an imaging apparatus that includes an optical system operable to focus light from a subject, a light amount adjustment unit operable to adjust an amount of light that is incident on the optical system, an imaging unit operable to read an optical image of the subject that is formed with the light focused by the optical system and output the read optical image as an image signal that is used to form an image including pixels, an A/D converter operable to convert an analogue image output from the imaging unit to a digital image, a digital signal processing unit operable to adjust an output value of the A/D converter, and the control unit operable to control the light amount adjustment unit, the imaging unit, and the digital signal processing unit,

the program enabling the control unit to:

drive the imaging unit in a manner that the imaging unit obtains a plurality of analogue images through a plurality of sequential divisional exposures each of which is performed for an identical exposure time, a total exposure time of the divisional exposures being equal to a predetermined exposure time; and

increase a dynamic range of corresponding pixels of a plurality of digital images resulting from conversion of the plurality of analogue images performed by the A/D converter and add up the corresponding pixels of the digital images to form an image with an increased dynamic range, and output the image with the increased dynamic range.

33. An integrated circuit used in an imaging apparatus that includes an optical system operable to focus light from a subject, a light amount adjustment unit operable to adjust an amount of light that is incident on the optical system, an imaging unit operable to read an optical image of the subject that is formed with the light focused by the optical system and output the read optical image as an image signal that is used to form an image including pixels, and an A/D converter operable to convert an output of the imaging unit through analogue-to-digital conversion, the integrated circuit comprising:

a dynamic-range increasing unit operable to linearly increase an output of the A/D converter; and

a control unit operable to adjust an amount of light of the light amount adjustment unit by an amount contrary to an amount by which the output of the A/D converter is increased.

34. An integrated circuit used in an imaging apparatus that includes an optical system operable to focus light from a subject, a light amount adjustment unit operable to adjust an amount of light that is incident on the optical system, an imaging unit operable to read an optical image of the subject that is formed with the light focused by the optical system and output the read optical image as an image signal that is used to form an image including pixels, an A/D converter operable to convert an analogue image output from the imaging unit to a digital image, and a drive unit operable to drive the imaging unit in a manner that the imaging unit obtains a plurality of analogue images through a plurality of sequential divisional exposures each of which is performed for an identical exposure time, a total exposure time of the divisional exposures being equal to a predetermined exposure time, the integrated circuit comprising:

a dynamic-range increasing unit operable to increase a dynamic range of corresponding pixels of a plurality of digital images resulting from conversion of the plurality of analogue images performed by the A/D converter and add up the corresponding pixels of the digital images to form an image with an increased dynamic range, and output the image with the increased dynamic range.

35. An integrated circuit used together with an optical system that focuses light from a subject, the integrated circuit comprising:

a light amount adjustment unit operable to adjust an amount of light that is incident on the optical system;

an imaging unit operable to read an optical image of the subject that is formed with the light focused by the optical system and output the read optical image as an image signal that is used to form an image including pixels;

an A/D converter operable to convert an analogue image output from the imaging unit to a digital image;

a drive unit operable to drive the imaging unit in a manner that the imaging unit obtains a plurality of analogue images through a plurality of sequential divisional exposures each of which is performed for an identical exposure time, a total exposure time of the divisional exposures being equal to a predetermined exposure time; and

a dynamic-range increasing unit operable to increase a dynamic range of corresponding pixels of a plurality of digital images resulting from conversion of the plurality of analogue images performed by the A/D converter and add up the corresponding pixels of the digital images to form an image with an increased dynamic range, and output the image with the increased dynamic range.

36. For an imaging apparatus equipped with an optical system, a computer logic device comprising:

a two-dimensional lookup table;

a computer logic storage device readable by a digital processing system; and

instructions embodied in the logic storage device, the instructions being executable by the digital processing system for performing method steps for undertaking a soft decision, the method steps comprising;

increasing a dynamic range of an output value of a signal resulting from analogue-to-digital conversion with a dynamic range increase ratio,

adjusting the output value according to a peak value of image data that is formed using the signal whose dynamic range has been magnified, and

obtaining an output value using a conversion characteristic stored in the two-dimensional lookup table according to the adjusted image signal and a vicinity luminance value of the image signal.