Method and apparatus for outputting images

• US 10,027,933 B2
• Filed: 07/07/2016
• Issued: 07/17/2018
• Est. Priority Date: 12/22/2015
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd., SungKyunKwan University's Research & Business Foundation (SungKyunKwan University)
• Original Assignee: Samsung Electronics Co. Ltd., SungKyunKwan University's Research & Business Foundation (SungKyunKwan University)
• Inventors: Won, Kwang-hyun, Kim, Chan-yul, Jeon, Byeung-Woo, Dinh, Khanh Quoc, Park, Young-hyeon, Shim, Hiuk-jae, Lee, Sun-il
• Primary Examiner(s): AGGARWAL, YOGESH K
• Application Number:
  US15/204,355
• Publication Number:
  US 20170180683A1
• Time in Patent Office: 740 Days
• Field of Search: 348267-273
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/13 with multiple sensors
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/136 using complementary colours
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20
• CPC Class Codes: H04N 9/67 for matrixing camera processing pipelines for matrixing of colour signals H04N23/85
• CPC Class Codes: H04N 9/76 for mixing of colour signals H04N9/75 takes precedence
• CPC Class Codes: H04N 9/77 Circuits for processing the brightness signal and the chrominance signal relative to each other, e.g. adjusting the phase of the brightness signal relative to the colour signal, correcting differential gain or differential phase circuits for matrixing H04N9/67

Abstract

A method and an apparatus for outputting an image by using a color filter array (CFA) sensor are provided. The method includes receiving a 4-channel combination image signal through the CFA sensor, and separating the 4-channel combination image signal into a visible image signal and a near-infrared image signal, based on a signal separation coefficient including a color coefficient that is determined to convert 4-channel color data into 3-channel color data and a weight coefficient that is determined based on a 4-channel signal configured of near-infrared image signals. The method further includes outputting the visible image signal and the near-infrared image signal.

Claims

1. A method of outputting an image, using a color filter array (CFA) sensor, the method comprising:

receiving a 4-channel combination image signal through the CFA sensor;

separating the 4-channel combination image signal into a visible image signal and a near-infrared image signal, based on a signal separation coefficient comprising a color coefficient that is determined to convert 4-channel color data into 3-channel color data and a weight coefficient that is determined based on a 4-channel signal configured of near-infrared image signals; and

outputting the visible image signal and the near-infrared image signal,

wherein the color coefficient is determined based on a parameter estimation algorithm to convert 4-channel color data into 3-channel color data.

2. The method of claim 1, wherein the 4-channel combination image signal comprises a combination of a 4-channel visible image signal and a 1-channel near-infrared image signal.

3. The method of claim 1, wherein the parameter estimation algorithm is determined based on least squares estimation at least once.

4. The method of claim 1, wherein the weight coefficient is determined based on calculating respective ratios of channels based on a first channel signal that is selected from 4-channel signals configured of near-infrared image signals.

5. The method of claim 1, further comprising interpolating the 4-channel combination image signal into four color separated combination image signals that comprise independent color signals.

6. The method of claim 1, further comprising simultaneously outputting a visible image from the visible image signal and a near-infrared image from the near-infrared image signal.

7. The method of claim 1, further comprising performing white balance processing on the visible image signal.

8. The method of claim 1, further comprising:

calculating a luminance image signal comprising brightness information of the visible image signal;

calculating a luminance average value of the calculated luminance image signal;

correcting brightness of the visible image signal based on comparing the luminance average value and a reference luminance average value; and

outputting a visible image from the visible image signal of which the brightness is corrected.

9. The method of claim 1, further comprising fusing the near-infrared image signal and the visible image signal to correct the visible image signal.

10. A non-transitory computer-readable storage medium storing a program comprising instructions to cause a computer to perform the method of claim 1.

11. An apparatus for outputting an image, using a color filter array (CFA) sensor, the apparatus comprising:

a signal receiver configured to receive a 4-channel combination image signal through the CFA sensor;

a signal separator configured to separate the 4-channel combination image signal into a visible image signal and a near-infrared image signal, based on a signal separation coefficient comprising a color coefficient that is determined to convert 4-channel color data into 3-channel color data and a weight coefficient that is determined based on a 4-channel signal configured of near-infrared image signals; and

an image outputter configured to output the visible image signal and the near-infrared image signal,

wherein the color coefficient is determined based on a parameter estimation algorithm to convert 4-channel color data into 3-channel color data.

12. The apparatus of claim 11, wherein the 4-channel combination image signal comprises a combination of a 4-channel visible image signal and a 1-channel near-infrared image signal.

13. The apparatus of claim 11, wherein the weight coefficient is determined based on a calculation of respective ratios of channels based on a first channel signal that is selected from 4-channel signals configured of near-infrared image signals.

14. The apparatus of claim 11, further comprising an image interpolator configured to interpolate the 4-channel combination image signal into four color separated combination image signals that comprise independent color signals.

15. The apparatus of claim 11, wherein the image outputter is further configured to simultaneously output a visible image from the visible image signal and output a near-infrared image from the near-infrared image signal.

16. The apparatus of claim 11, further comprising a white balance processor configured to perform white balance processing on the visible image signal.

17. The apparatus of claim 11, further comprising a brightness corrector configured to:

calculate a luminance image signal comprising brightness information of the visible image signal;

calculate a luminance average value of the calculated luminance image signal; and

correct brightness of the visible image signal based on a comparison of the luminance average value and a reference luminance average value, and output a visible image from the visible image signal of which the brightness is corrected.

18. The apparatus of claim 11, further comprising an image corrector configured to fuse the near-infrared image signal and the visible image signal to correct the visible image signal.