Digital cameras with direct luminance and chrominance detection

• US 7,964,835 B2
• Filed: 06/06/2007
• Issued: 06/21/2011
• Est. Priority Date: 08/25/2005
• Status: Active Grant

• Current Assignee: Intellectual Ventures II LLC (Intellectual Ventures LLC)
• Original Assignee: Protarius Filo AG LLC (Intellectual Ventures LLC)
• Inventors: Sato, Darryl L., Olsen, Richard Ian, Gates, James
• Primary Examiner(s): Sohn; Seung C
• Application Number:
  US11/810,623
• Publication Number:
  US 20070296835A1
• Time in Patent Office: 1,476 Days
• Field of Search: 250/208.1, 250/226, 250/559.08, 348/223.1, 348/280, 348/297, 348/308, 348/262
• US Class Current: 250/208.1
• CPC Class Codes: G02B 3/0031 Replication or moulding, e.g. hot embossing, UV-casting, injection moulding
• CPC Class Codes: G02B 3/0043 Inhomogeneous or irregular arrays, e.g. varying shape, size, height
• CPC Class Codes: G02B 3/0062 Stacked lens arrays, i.e. refractive surfaces arranged in at least two planes, without structurally separate optical elements in-between
• CPC Class Codes: G02B 3/0075 characterized by non-optical structures, e.g. having integrated holding or alignment means
• CPC Class Codes: G02B 9/12 having three components only
• CPC Class Codes: H01L 27/14618 Containers
• CPC Class Codes: H01L 27/14621 Colour filter arrangements
• CPC Class Codes: H01L 27/14625 Optical elements or arrangements associated with the device
• CPC Class Codes: H01L 27/14634 Assemblies, i.e. Hybrid structures
• CPC Class Codes: H01L 27/14645 Colour imagers
• CPC Class Codes: H01L 27/14652 Multispectral infrared imagers, having a stacked pixel-element structure, e.g. npn, npnpn or MQW structures
• CPC Class Codes: H01L 2924/00 Indexing scheme for arrangements or methods for connecting or disconnecting semiconductor or solid-state bodies as covered by H01L24/00
• CPC Class Codes: H01L 2924/0002 Not covered by any one of groups H01L24/00, H01L24/00 and H01L2224/00
• CPC Class Codes: H01L 31/0232 Optical elements or arrangements associated with the device H01L31/0236 takes precedence; for photovoltaic cells H01L31/054; for photovoltaic modules H02S40/20
• CPC Class Codes: H01L 31/02325 the optical elements not being integrated nor being directly associated with the device
• CPC Class Codes: H04N 2209/049 having three pick-up sensors
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/13 with multiple sensors
• CPC Class Codes: H04N 23/16 Optical arrangements associated therewith, e.g. for beam-splitting or for colour correction
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 23/54 : Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/71 : Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 23/73 : by influencing the exposure time
• CPC Class Codes: H04N 23/843 : Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 23/88 : for colour balance, e.g. white-balance circuits or colour temperature control
• CPC Class Codes: H04N 25/00 : Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 25/134 : based on three different wavelength filter elements
• CPC Class Codes: H04N 25/41 : Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors
• CPC Class Codes: H04N 25/53 : Control of the integration time
• CPC Class Codes: H04N 25/531 : by controlling rolling shutters in CMOS SSIS
• CPC Class Codes: H04N 5/33 : Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Digital camera systems and methods are described that provide a color digital camera with direct luminance detection. The luminance signals are obtained directly from a broadband image sensor channel without interpolation of RGB data. The chrominance signals are obtained from one or more additional image sensor channels comprising red and/or blue color band detection capability. The red and blue signals are directly combined with the luminance image sensor channel signals. The digital camera generates and outputs an image in YCrCb color space by directly combining outputs of the broadband, red and blue sensors.

Claims

1. A digital camera comprising:

a broadband channel including a first optics component and a broadband image sensor configured to directly detect luminance;

at least one color channel including a second optics component and at least one color image sensor integrated in a substrate and configured to directly detect chrominance data, wherein the at least one color channel includes a plurality of color channels comprising a first color channel with a first image sensor configured to detect data in a red color band, and a second color channel with a second image sensor configured to detect data in a blue color band, and wherein detecting chrominance data comprises generating red chroma data through direct combination of the red color band data with the luminance data; and

generating blue chroma data through direct combination of the blue color band data with the luminance data; and

a processing component coupled to the broadband channel and the at least one color channel and configured to combine outputs of the broadband channel and the at least one color channel to provide an image in YCrCb color space.

2. The digital camera of claim 1, wherein the broadband channel is configured to detect data approximately in a range of 400 nanometers (nm) to 700 nm.

3. The digital camera of claim 1, wherein the broadband channel is configured to detect data approximately in a range of 400 nanometers (nm) up to 1060 nm.

4. The digital camera of claim 1, wherein the broadband channel is configured to detect data approximately in a range of 250 nanometers (nm) up to 1060 nm.

5. The digital camera of claim 1, wherein detecting chrominance data excludes interpolation of the data from the at least one color channel.

6. The digital camera of claim 1, wherein detecting chrominance data excludes interpolation of the red color band and blue color band data.

7. The digital camera of claim 1, wherein the processing component is configured to separately and simultaneously control an integration time of the first image sensor and an integration time of the second image sensor during a frame, wherein the integration times of the first and second image sensors are different.

8. The digital camera of claim 7, wherein the processing component is configured to use an image signal level to determine integration times for subsequent frames.

9. The digital camera of claim 7, wherein the integration time of the first image sensor and the integration time of the second image sensor are selected such that the integration times correspond to a range of optical signal levels incident on the broadband and color channels.

10. The digital camera of claim 7, wherein the processing component is configured to control the integration time of the first image sensor and the integration time of the second image sensor using a control technique selected from a group consisting of automatic control and manual control.

11. The digital camera of claim 1, wherein the first optics component includes a different configuration than the second optics component.

12. The digital camera of claim 1, wherein one or more of the first and second optics components includes a plurality of lenses.

13. The digital camera of claim 1, wherein the second optics component includes at least one filter.

14. The digital camera of claim 1, wherein the first optics component has a first field of view (FOV) and the second optics component has a second FOV that is different than the first FOV.

15. The digital camera of claim 1, wherein the first optics component has a first focal length and the second optics component has a second focal length that is different than the first focal length.

16. The digital camera of claim 1, wherein the broadband sensor has a first set of dimensions and the at least one color image sensor has a second set of dimensions at least one of which is different than a dimension of the first set of dimensions.

17. A digital camera comprising:

a plurality of channels, wherein each channel includes an optics component and an array of photo-detectors integrated on a semiconductor substrate, the plurality of channels including a first channel configured to detect luminance data and at least one other channel configured to directly detect chrominance data, wherein the at least one other channel includes a second channel configured to detect data in a red and a blue color band, and wherein detecting chrominance data includes generating red chroma and blue chroma data through direct combination of the red and blue color band data with the luminance data; and

a processing component coupled to the plurality of channels, the processing component configured to independently control simultaneous data acquisition with each of the plurality of channels, the processing component configured to combine data from the plurality of channels to provide an image in YCrCb color space, wherein the combining excludes interpolation of the data from the plurality of channels.

18. The digital camera of claim 17, wherein the first channel is configured to detect data approximately in a range of 400 nanometers (nm) to 700 nm.

19. The digital camera of claim 17, wherein detecting chrominance data excludes interpolation of the data from the at least one color channel.

20. The digital camera of claim 17, wherein the processing component is configured to separately and simultaneously control an integration time of each array of photo-detectors during a frame.

21. The digital camera of claim 20, wherein the processing component is configured to use the image to determine the integration time for each array of photo-detectors for a subsequent frame.

22. The digital camera of claim 17, wherein the optics components of at least two of the plurality of channels have different configurations.

23. A digital camera comprising:

a first channel including a first optics component and a first image sensor, the first channel configured to image a white color band and output luminance data;

a second channel including a second optics component and a second image sensor, the second channel configured to image at least one of a red and a blue color band and output chrominance data, wherein outputting the chrominance data includes generating red chroma data through direct combination of the red color band data with the luminance data without interpolation of RGB data, and further includes generating blue chroma data through combination of the blue color band data with the luminance data without interpolation of RGB data; and

a processing component coupled to the first and second channels and configured to combine outputs of the first and second channels to provide an image in YCrCb color space without interpolation in RGB color space.

24. The digital camera of claim 23, wherein the first channel is configured to detect data approximately in a range of 400 nanometers (nm) to 700 nm.

25. A method comprising:

detecting luminance data using a broadband sensor of a digital camera by sensing and outputting a broadband wavelength response at the broadband sensor and excluding use of RGB color data;

generating chrominance data using at least one color sensor of the digital camera by sensing and outputting at least one color band wavelength response, wherein generating the chrominance data comprises sensing data in a red color band and generating red chroma data through direct combination of the red color band data with the luminance data, and further comprises sensing data in a blue color band and generating blue chroma data through direct combination of the blue color band data with the luminance data; and

generating an image in YCrCb color space by combining outputs of the broadband sensor and the at least one color sensor and outputting the image from the digital camera.

26. The method of claim 25, wherein generating the chrominance data excludes interpolation of the at least one color band wavelength response.

27. The method of claim 25, further comprising sensing data in a green color band.

28. The method of claim 27, further comprising generating an image in RGB color space by combining data of the red, blue and green color bands and outputting the image from the digital camera.

29. The method of claim 25, wherein detecting the luminance data comprises detecting data approximately in a range of 400 nanometers (nm) to 700 nm.

30. A method comprising:

detecting luminance data using a broadband wavelength response and excluding use of RGB color data;

generating chrominance data by detecting, via one or more sensors, red color band data and blue color band data, generating a red chroma component by combining the red color band data with the luminance data, and generating a blue chroma component by combining the blue color band data with the luminance data; and

outputting an image in YCrCb color space using the luminance and chrominance data.

31. The method of claim 30, further comprising detecting green color band data and outputting an image in RGB color space.

32. An article of manufacture including a non-transitory computer-readable medium having instructions stored thereon that, if executed by a computing device, cause the computing device to perform operations comprising:

detecting luminance data using a broadband sensor of a digital camera by sensing and outputting a broadband wavelength response at the broadband sensor and excluding use of RGB color data;

generating chrominance data using at least one color sensor of the digital camera by sensing and outputting at least one color band wavelength response, wherein generating the chrominance data comprises sensing data in a red color band and generating red chroma data through direct combination of the red color band data with the luminance data, and further comprises sensing data in a blue color band and generating blue chroma data through direct combination of the blue color band data with the luminance data; and

generating an image in YCrCb color space by combining outputs of the broadband sensor and the at least one color sensor and outputting the image from the digital camera.