Image Pickup Apparatus and Image Pickup Method

• US 20110157413A1
• Filed: 12/23/2010
• Published: 06/30/2011
• Est. Priority Date: 12/24/2009
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: YOSHIDA, Takahiko
• Granted Patent: US 8,717,454 B2
• Time in Patent Office: Days
• US Class Current: 348/223.1
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/88 for colour balance, e.g. white-balance circuits or colour temperature control
• CPC Class Codes: H04N 9/73 Colour balance circuits, e.g. white balance circuits or colour temperature control camera processing pipelines for colour balance H04N23/88

Abstract

An image pickup apparatus includes an image pickup unit that acquires a first image without a flash and a second image with the flash; a down-sampling unit that generates a third image by down-sampling the second image; a brightness difference calculator that calculates a brightness difference between the first image and the third image; a flash reflection intensity calculator that calculates a first flash reflection intensity from the brightness difference; and a white balance gain calculator that calculates a white balance gain based on the first flash reflection intensity. An image pickup method includes acquiring a first image without a flash and a second image with the flash, generating a third image by down-sampling the second image, calculating a brightness difference between the first image and the third image, calculating a first flash reflection intensity from the brightness difference, and calculating a white balance gain.

Claims

1. An image pickup apparatus comprising:

an image pickup unit that acquires a first image without a flash by a down-sampling process and acquires a second image with the flash, wherein the second image has a larger number of pixels than the first image;

a down-sampling unit that acquires a third image by performing a down-sampling process on the second image in such a way that a size of the second image is the same as a size of the first image;

a subject brightness calculator that calculates a first subject brightness for all pixels of the first image and calculates a second subject brightness for all pixels of the third image;

a subject brightness difference calculator that calculates a subject brightness difference, wherein the subject brightness difference corresponds to a difference between the first subject brightness for all the pixels of the first image and the second subject brightness for all the pixels of the third image;

a flash reflection intensity calculator that calculates a first flash reflection intensity in a unit of a pixel from the subject brightness difference;

a first flash reflection intensity interpolator that calculates a second flash reflection intensity through interpolation to reduce an influence of a position mismatch between the first image and the third image for the first flash reflection intensity in a unit of a pixel by using pixel-based brightness values of the third image;

a second flash reflection intensity interpolator that calculates a third flash reflection intensity through interpolation to magnify a size of the third image to the size of the second image for the second flash reflection intensity in a unit of a pixel by using pixel-based brightness values of the second image; and

a white balance gain calculator that calculates a white balance gain in a unit of a pixel from the third flash reflection intensity.

2. The image pickup apparatus of claim 1, wherein the image pickup apparatus further comprises a white balance controller that controls a white balance in a unit of a pixel by applying the white balance gain to all the pixels of the second image in a unit of a pixel.

3. The image pickup apparatus of claim 1, further comprising:

a light-exposure control value calculator that calculates a first light-exposure control value for the first image and a second light-exposure control value for the second image; and

a light-exposure control value difference calculator that calculates a light-exposure control value difference, wherein the light-exposure control value difference corresponds to a difference between the first light-exposure control value and the second light-exposure control value, for all pixels,

wherein the image pickup unit acquires the first image by performing photographing based on the first light-exposure control value and acquires the second image by performing photographing based on the second light-exposure control value.

4. The image pickup apparatus of claim 3, wherein the first light-exposure control value and the second light-exposure control value are independently calculated by the light-exposure control value calculator.

5. The image pickup apparatus of claim 3, wherein the image pickup unit acquires the second image by performing photographing with the second light-exposure control value and wherein an iris value of the second light-exposure control value is updated to correspond to an iris value of the first light-exposure control value calculated by the light-exposure control value calculator.

6. The image pickup apparatus of claim 1, further comprising a color balance calculator that calculates a color balance of a corresponding pixel by linear-interpolating a color balance with respect to surrounding light and a color balance with respect to flash light based on the first flash reflection intensity calculated by the flash reflection intensity calculator,

wherein the white balance gain calculator calculates the white balance gain based on the color balance of the corresponding pixel, the color balance with respect to the surrounding light, and the color balance with respect to the flash light.

7. The image pickup apparatus of claim 1, wherein the first flash reflection intensity interpolator calculates the second flash reflection intensity based on brightness values of interpolation source pixels located at upper, lower, left, and right sides of and centered around a pixel to be interpolated and the first flash reflection intensity.

8. The image pickup apparatus of claim 7, wherein, for the first flash reflection intensity interpolator, a distance between the pixel to be interpolated and each of the interpolation source pixels located at the upper, lower, left, and right sides of the pixel to be interpolated is constant.

9. An image pickup method comprising:

acquiring a first image without a flash by a down-sampling process and acquiring a second image with the flash, wherein the second image has a larger number of pixels than the first image;

acquiring a third image by performing a down-sampling process on the second image in such a way that a size of the second image is the same as a size of the first image;

calculating a first subject brightness for all pixels of the first image and calculating a second subject brightness for all pixels of the third image;

calculating a subject brightness difference, wherein the subject brightness difference corresponds to a difference between the first subject brightness for all the pixels of the first image and the second subject brightness for all the pixels of the third image;

calculating a first flash reflection intensity in a unit of a pixel from the subject brightness difference;

calculating a second flash reflection intensity through interpolation to reduce an influence of a position mismatch between the first image and the third image for the first flash reflection intensity in a unit of a pixel by using pixel-based brightness values of the third image;

calculating a third flash reflection intensity through interpolation to magnify a size of the third image to the size of the second image for the second flash reflection intensity in a unit of a pixel by using pixel-based brightness values of the second image; and

calculating a white balance gain in a unit of a pixel from the third flash reflection intensity.

10. An image pickup apparatus comprising:

an image pickup unit that acquires a first image without a flash and a second image with the flash, wherein the second image has a larger number of pixels than the first image;

a down-sampling unit that generates a third image by down-sampling the second image to a size of the first image;

a brightness difference calculator that calculates a brightness difference between the first image and the third image;

a flash reflection intensity calculator that calculates a first flash reflection intensity in a unit of a pixel from the brightness difference; and

a white balance gain calculator that calculates a white balance gain based on the first flash reflection intensity.

11. The image pickup apparatus of claim 10, further comprising a first flash reflection intensity interpolator that calculates a second flash reflection intensity through interpolation to reduce an influence of a position mismatch between the first image and the third image for the first flash reflection intensity,

wherein the white balance gain calculator calculates the white balance gain based on the second flash reflection intensity.

12. The image pickup apparatus of claim 11, further comprising a second flash reflection intensity interpolator that calculates a third flash reflection intensity through interpolation to magnify a size of the third image to a size of the second image for the second flash reflection intensity by using pixel-based brightness values of the second image,

wherein the white balance gain calculator calculates the white balance gain based on the third flash reflection intensity.

13. The image pickup apparatus of claim 12, further comprising a white balance controller that controls a white balance in a unit of a pixel by applying the white balance gain to all pixels of the second image in a unit of a pixel.

14. An image pickup method comprising:

acquiring a first image without a flash and a second image with the flash, wherein the second image has a larger number of pixels than the first image;

generating a third image by down-sampling the second image to a size of the first image;

calculating a brightness difference between the first image and the third image;

calculating a first flash reflection intensity in a unit of a pixel from the brightness difference; and

calculating a white balance gain based on the first flash reflection intensity.

15. The image pickup method of claim 14, further comprising calculating a second flash reflection intensity through interpolation to reduce an influence of a position mismatch between the first image and the third image for the first flash reflection intensity,

wherein the calculating of the white balance gain includes calculating the white balance gain based on the second flash reflection intensity.

16. The image pickup method of claim 15, further comprising calculating a third flash reflection intensity through interpolation to magnify a size of the third image to a size of the second image for the second flash reflection intensity by using pixel-based brightness values of the second image,

wherein the calculating of the white balance gain includes calculating the white balance gain based on the third flash reflection intensity.

17. The image pickup method of claim 16, further comprising controlling a white balance in a unit of a pixel by applying the white balance gain to all pixels of the second image in a unit of a pixel.