Apparatus and method for combining images

• US 10,368,015 B2
• Filed: 08/14/2017
• Issued: 07/30/2019
• Est. Priority Date: 05/19/2008
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Choe, Won-hee, Kwon, Jae-hyun, Lee, Seong-deok
• Primary Examiner(s): Tsai, Tsung Yin
• Application Number:
  US15/676,235
• Publication Number:
  US 20180013963A1
• Time in Patent Office: 715 Days
• CPC Class Codes: H04N 1/486 with separate detectors, each detector being used for one specific colour component
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 25/133 including elements passing panchromatic light, e.g. filters passing white light
• CPC Class Codes: H04N 25/135 based on four or more different wavelength filter elements
• CPC Class Codes: H04N 9/67 for matrixing camera processing pipelines for matrixing of colour signals H04N23/85
• CPC Class Codes: H04N 9/76 for mixing of colour signals H04N9/75 takes precedence
• CPC Class Codes: H04N 9/78 for separating the brightness signal or the chrominance signal from the colour television signal, e.g. using comb filter

Abstract

Provided are an image composition apparatus for composing color images with black-and-white images including infrared components, and an image composition method thereof. The image composition method includes generating a first image signal with color information and a second image signal including infrared components without color information, dividing the first image signal into a brightness signal and a color signal, composing the brightness signal of the first image signal with a brightness signal of the second image signal to generate a composed brightness signal, and composing the composed brightness signal with the color signal of the first image signal to generate a color image.

Claims

1. An image composition apparatus, the apparatus comprising:

at least one image sensor configured to acquire incident light and generate a first image signal with color information of a visible band of an optical spectrum and a second image signal comprising a wider band than the first image signal and including black-and-white components of the visible band of the optical spectrum; and

at least one processor configured to control to;

divide the first image signal into a color signal and a brightness signal,

combine the divided brightness signal of the first image signal with the second image signal, from the at least one image sensor, to generate a combined brightness signal including the wider band of the second image signal, and

compose the combined brightness signal including the wider band with the color signal of the first image signal to generate a color image,

wherein the second image signal, which is combined with the divided brightness signal of the first image signal, is a same image signal as the second image signal generated by the at least one image sensor.

2. The apparatus of claim 1,

wherein the first image signal includes a signal corresponding to a specific region of the visible band of the optical spectrum;

and

wherein the second image signal includes a combination of signals corresponding to specific regions of the visible band, which generate the black-and-white components.

3. The apparatus of claim 1, wherein the at least one processor is further configured to control to equalize resolutions of the first image signal and the second image signal.

4. The apparatus of claim 1, wherein the at least one processor is further configured to control to transform spatial domains of the first image signal and the second image signal into frequency domains.

5. The apparatus of claim 1, wherein the at least one processor is further configured to control to generate the combined brightness signal by adding a coefficient of the brightness signal of the first image signal and a coefficient of the brightness signal of the second image signal using Discrete Wavelet Transform (DWT).

6. The apparatus of claim 1, wherein the at least one processor is further configured to control to combine the brightness signal of the first image signal with the brightness signal of the second image signal based on a conversion function representing a relationship between brightness information of the first image signal and brightness information of the second image signal.

7. The apparatus of claim 1, wherein the at least one processor is further configured to control to combine the brightness signal of the first image signal with the brightness signal of the second image signal based on a lookup table.

8. An image composition method in an image composition apparatus, the method comprising:

generating, by at least one image sensor, a first image signal with color information of a visible band of an optical spectrum and a second image signal comprising a wider band than the first image signal and including black-and-white components of the visible band of the optical spectrum;

dividing, by at least one processor, the first image signal into a color signal and a brightness signal;

combining, by the at least one processor, the divided brightness signal of the first image signal with the second image signal generated by the at least one image sensor, to generate a combined brightness signal including the wider band; and

composing, by the at least one processor, the combined brightness signal with the color signal of the first image signal to generate a color image,

wherein the second image signal, which is combined with the divided brightness signal of the first image signal, is a same image signal as the second image signal generated by the at least one image sensor.

9. The method of claim 8, wherein the first image signal includes a signal corresponding to a specific region of the visible band of the optical spectrum, and the second image signal includes a combination of signals corresponding to specific regions of the visible band, which generate the black-and-white components.

10. The method of claim 8, further comprising matching, by the at least one processor, a resolution of the first image signal with a resolution of the second image signal.

11. The method of claim 10, wherein the matching of the resolution comprises interpolating an image signal with the lower resolution among the first and second image signals with respect to an image signal with the higher resolution among the first and second image signals.

12. The method of claim 8, wherein the combining comprises combining the brightness signal of the first image signal with the brightness signal of the second image signal by adding a coefficient of the brightness signal of the color image signal and a coefficient of the brightness signal of the first image signal and the second image signal using Discrete Wavelet Transform (DWT).

13. The method of claim 8, wherein the combining comprises combining the brightness signal of the first image signal with the brightness signal of the second image signal based on a conversion function representing a relationship between brightness information of the first image signal and brightness information of the second image signal.

14. The method of claim 8, wherein the combining comprises combining the brightness signal of the first image signal with the brightness signal of the second image signal based on a lookup table.

15. A portable communication device comprising:

a first image sensor to generate a first image signal corresponding to a color image having a first resolution;

a second image sensor to generate a second image signal corresponding to a monochrome image having a second resolution being different from the first resolution; and

a processor adapted to;

obtain a color signal and a first brightness signal from the first image signal,

obtain a second brightness signal from the second image signal,

generate a third brightness signal based at least in part on the first brightness signal and the second brightness signal, and

generate an image based at least in part on the color signal and the third brightness signal.

16. The portable communication device of claim 15, wherein the first image sensor comprises a filter array to filter specified light having a specified wavelength from incident light.

17. The portable communication device of claim 15, wherein the processor is further adapted to:

as at least part of the obtaining of the color signal and the first brightness signal, convert the first image signal into a third image signal including at least one color signal component and at least one brightness signal component.

18. The portable communication device of claim 17, wherein the processor is further adapted to:

as at least part of the obtaining of the color signal and the first brightness signal, divide the third image signal into the at least one color signal component and the at least one brightness signal component, and

identify the at least one color signal component as the color signal and the at least one brightness signal component as the brightness signal.

19. The portable communication device of claim 15, wherein the processor is further adapted to:

compose the first brightness signal and the second brightness signal into the third brightness signal, as at least part of the generating of the third brightness signal.

20. The portable communication device of claim 15,

further comprising a memory to store reference information associating brightness information related to the color image with brightness information related to the monochrome image,

wherein the processor is further adapted to perform the generating of the third brightness signal using the reference information.

21. The portable communication device of claim 15,

wherein a bandwidth of the second brightness signal is wider than a bandwidth of the first brightness signal, and

wherein the processor is further adapted to:

as at least part of the generating of the third brightness signal, adjust the bandwidth of the first brightness signal into the bandwidth of the second brightness signal.

22. The portable communication device of claim 15, wherein the processor is further adapted to:

generate the third brightness signal without any further processing of the first brightness signal and the second brightness signal.

23. The portable communication device of claim 15, wherein the processor is further adapted to:

generate the image by combining the color signal and the third brightness signal.

24. The portable communication device of claim 15, wherein the processor is further adapted to:

as at least part of the generating of the image, perform an inverse processing of the obtaining of the color signal and the first brightness signal.

25. The portable communication device of claim 15, wherein the first resolution is lower than the second resolution.

26. The portable communication device of claim 15, wherein the processor is further adapted to:

prior to the generating of the third brightness signal, process the first brightness signal to adjust a resolution of the first brightness signal into the second resolution.

27. The portable communication device of claim 15, wherein the processor is further adapted to:

prior to the generating of the image, process the color signal to adjust a resolution of the color signal into the second resolution.

28. The portable communication device of claim 15, wherein the first image sensor and the second image sensor are disposed on a same side of the portable communication device.

29. A method of generating an image using a portable communication device, the method comprising:

generating, using a first image sensor operatively coupled with the portable communication device, a first image signal corresponding to a color image having a first resolution;

generating, using a second image sensor operatively coupled with the portable communication device, a second image signal corresponding to a monochrome image having a second resolution being different from the first resolution;

obtaining a color image signal and a first brightness signal from the first image signal;

obtaining a second brightness signal from the second image signal;

generating a third brightness signal based at least in part on the first brightness signal and the second brightness signal; and

generating the image based at least in part on the color image signal and the third brightness signal.

30. The method of claim 29, wherein a bandwidth of the second brightness signal is wider than a bandwidth of the first brightness signal, and wherein the generating of the third brightness signal comprises:

adjusting the bandwidth of the second brightness signal into the bandwidth of the first brightness signal.

31. The method of claim 29, wherein the first resolution is lower than the second resolution.

32. The method of claim 29, wherein the generating of the third brightness signal comprises:

prior to the generating of the third brightness signal, processing the second brightness signal to adjust a resolution of the second brightness signal into the first resolution.

33. The method of claim 29, wherein the generating of the second image signal is performed based at least in part on a determination that illumination falls into a low illumination range.

34. A portable communication device comprising:

a first image sensor;

a second image sensor; and

a processor adapted to;

generate a first color image signal using the first image sensor,

generate a monochrome image signal using the second image sensor,

divide the first color image signal into a color signal and a first brightness signal,

obtain a second brightness signal from the monochrome image signal, and

generate a second color image signal based at least in part on the color signal, the first brightness signal, and the second brightness signal.