Apparatus and method of obtaining image and apparatus and method of processing image

• US 8,169,491 B2
• Filed: 06/16/2008
• Issued: 05/01/2012
• Est. Priority Date: 01/22/2008
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd., Samsung Electro-Mechanics Co., Ltd. (Samsung Group)
• Original Assignee: Samsung Electronics Co. Ltd., Samsung Electro-Mechanics Co., Ltd. (Samsung Group)
• Inventors: Kang, Dong-goo, Wey, Ho-cheon, Oh, Hyun-hwa, Lee, Seong-deok
• Primary Examiner(s): Tran, Nhan T
• Assistant Examiner(s): LE, TUAN H
• Application Number:
  US12/213,199
• Publication Number:
  US 20090185041A1
• Time in Patent Office: 1,415 Days
• Field of Search: 348/216.1, 348/221.1, 348270-276, 348/362, 348/208.1, 348/222.1, 348/208.99, 348/208.4, 382/300
• US Class Current: 348/208.99
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations

Abstract

Provided are an apparatus and method of obtaining a high-quality image by efficiently processing an image obtained at low-light levels. The image obtaining apparatus comprises a sensor which detects a pixel array value by using a color filter including color pixels for obtaining a color image and a reference pixel for obtaining a reference image used for hand-trembling function estimation; an exposure controller which controls exposure times of the color pixels and the reference pixel; and an image generator which generates a long-exposure color image signal and a short-exposure reference image signal from the detected pixel array value, the long-exposure color image signal and the short-exposure reference image signal being aligned with each other. Accordingly, accurate estimation of the hand-trembling function is possible, thereby enabling the high-quality color image restoration.

Claims

1. An image obtaining apparatus comprising:

a sensor to detect a plurality of pixel values of a plurality of pixels, including detecting color pixel values of multi-color color pixels of the plurality of pixels and monochromatic reference pixel values of monochromatic reference pixels of the plurality of pixels, by using a color filter including the multi-color color pixels configured to obtain a color image and the monochromatic reference pixels configured to obtain a monochromatic reference image, which are used for hand-trembling function estimation;

an exposure controller configured to control exposure times of the multi-color color pixels and the monochromatic reference pixels; and

an image generator configured to generate a long-exposure color image signal corresponding to the obtaining of the color image from the detected color pixel values and a short-exposure reference image signal from the detected monochromatic reference pixel values, the long-exposure color image signal and the short-exposure reference image signal being aligned with each other.

2. The image obtaining apparatus of claim 1, wherein the exposure controller synchronizes exposure starting times of the multi-color color pixels and the monochromatic reference pixels to start with each other and controls the exposure time of the multi-color color pixels to be longer for the obtaining of the color image than the exposure time of the monochromatic reference pixels.

3. The image obtaining apparatus of claim 2, wherein the exposure controller controls the sensor such that a short-exposure of the monochromatic reference pixels takes place repeatedly while the multi-color color pixels are being exposed for the obtaining of the color image in order to obtain initial information of the hand-trembling function.

4. The image obtaining apparatus of claim 1, wherein the image generator performs interpolation on the plurality of pixel values detected through the multi-color color pixels and the monochromatic reference pixels so as to generate long-exposure image signals of respective colors of the multi-color color pixels and a short-exposure image signal of the monochromatic reference pixels.

5. The image obtaining apparatus of claim 1, wherein the image generator is configured to obtain a single-channel reference image signal and a multi-channel color image signal.

6. The image obtaining apparatus of claim 1, wherein the monochromatic reference pixels are white (W) pixels or green (G) pixels.

7. The image obtaining apparatus of claim 1, wherein all of the monochromatic reference pixels of the image obtaining apparatus are pixels of the same color.

8. A method of obtaining an image, the method comprising:

obtaining a plurality of pixel values of a plurality of pixels, including color pixel values of multi-color color pixels of the plurality of pixels and monochromatic reference pixel values of monochromatic reference pixels of the plurality of pixels, by controlling exposure times of the multi-color color pixels for obtaining a color image and the monochromatic reference pixels for obtaining a monochromatic reference image used for hand-trembling function estimation, the multi-color color pixels and the monochromatic reference pixels being included in a color filter; and

generating a long-exposure color image signal corresponding to the obtaining of the color image from the detected color pixel values and a short-exposure reference image signal based on the monochromatic reference pixel values, the long-exposure color image signal and the short-exposure reference image signal being aligned with each other.

9. The method of claim 8, wherein, in the controlling of the exposure times, exposure starting times of the multi-color color pixels and the monochromatic reference pixels are synchronized to start with each other, and the exposure time of the multi-color color pixels for the obtaining of the color image is set to be longer than the exposure time of the monochromatic reference pixels.

10. The method of claim 9, wherein, in the controlling of the exposure times, the exposure time of the monochromatic reference pixels is controlled such that a short-exposure of the monochromatic reference pixels takes place repeatedly while the multi-color color pixels are being exposed for the obtaining of the color image in order to obtain initial information of the hand-trembling function.

11. The method of claim 8, wherein, in the generating of the image signals, interpolation is performed on the plurality of pixel values detected through the multi-color color pixels and the monochromatic reference pixels so as to generate long-exposure image signals of respective colors of the multi-color color pixels for the obtaining of the color image from the multi-color color pixels and a short-exposure image signal from the monochromatic reference pixels.

12. An image processing apparatus comprising:

an image obtaining unit which obtains a long-exposure color image signal and a short-exposure reference image signal, which are aligned with each other, by controlling exposure times of a sensor, the sensor being configured to detect a plurality of pixels values of a plurality of pixels, including color pixel values of multi-color color pixels of the plurality of pixels and monochromatic reference pixel values of monochromatic reference pixels of the plurality of pixels, by using a color filter including the multi-color color pixels for obtaining a color image and the monochromatic reference pixels for obtaining a monochromatic reference image;

a function estimating unit which estimates a hand-trembling function based on the long-exposure color image signal and the short-exposure reference image signal; and

an image restoring unit which generates a restored image signal by performing deconvolution on the long-exposure color image signal, corresponding to the obtaining of the color image using the multi-color color pixels, based on the estimated hand-trembling function.

13. The image processing apparatus of claim 12, wherein the image obtaining unit obtains the long-exposure color image signal and the short-exposure reference image signal by synchronizing exposure starting times of the multi-color color pixels and the monochromatic reference pixels to start with each other, respectively, and controlling the exposure time of the multi-color color pixels for the obtaining of the color image to be longer than the exposure time of the monochromatic reference pixels.

14. The image processing apparatus of claim 13, wherein the image obtaining unit collects a plurality of short-exposure reference image signals by controlling short-exposure of the monochromatic reference pixels to take place repeatedly for the monochromatic reference image while the multi-color color pixels are being exposed for the obtaining of the color image, and the function estimating unit obtains information by tracing the locus of hand-trembling based on the collected reference image signals and uses the information as initial information of the hand-trembling function.

15. The image processing apparatus of claim 12, wherein the function estimating unit reduces noise of the long-exposure color image signal and the short-exposure reference image signal, converts the noise-reduced long-exposure color image signal into a long-exposure single-channel image signal, and estimates the hand-trembling function based on the converted long-exposure single-channel image signal and the noise-reduced short-exposure reference image signal.

16. The image processing apparatus of claim 12, wherein the function estimating unit is configured to convert a multi-channel color image signal into a single-channel color image signal.

17. The image processing apparatus of claim 16, wherein the conversion involves using one of the multiple channels of the multi-channel color image signal as the single-channel color image signal or wherein the conversion involves averaging the multiple channels of the multi-channel color image signal into a single-channel color image signal or wherein the conversion involves inverting the multiple channels of the multi-channel image signal into a single-channel grey image signal.

18. A method of processing image, the method comprising:

obtaining a long-exposure color image signal and a short-exposure reference image signal, which are aligned with each other, by controlling respective exposure times of a plurality of pixels, including multi-color color pixels of the plurality of pixels for obtaining the long-exposure color image signal and monochromatic reference pixels of the plurality of pixels for obtaining the short-exposure reference image signal;

estimating a hand-trembling function based on the long-exposure color image signal and the short-exposure reference image signal; and

generating a restored image signal by performing deconvolution on the long-exposure color image signal based on the estimated hand-trembling function.

19. The method of claim 18, wherein the obtaining of the image signals, exposure starting times of the multi-color color pixels for the long-exposure color image signal and the monochromatic reference pixel for the short exposure color image signal are synchronized to start with each other, and the exposure time of the multi-color color pixels for the obtaining of the long-exposure color image signal is set to be longer than the exposure time of the monochromatic reference pixels.

20. The method of claim 19, wherein, in the obtaining of the image signals, a plurality of the short-exposure image signals are collected by controlling short-exposures of the monochromatic reference pixels to take place repeatedly while the multi-color color pixels for the long-exposure color image signal are being exposed, and in the estimating of the hand-trembling function, information is obtained by tracing the locus of hand-trembling based on the collected plurality of short exposure image signals and the obtained information is used as initial information of the hand-trembling function.

21. The method of claim 18, wherein the estimating of the hand-trembling function comprises:

reducing noise of the long-exposure color image signal and the short-exposure image signal;

converting the noise-reduced long-exposure color image signal into a long-exposure single-channel image signal; and

estimating the hand-trembling function based on the converted long-exposure single-channel image signal and the noise-reduced short-exposure reference image signal.