Digital cameras with direct luminance and chrominance detection

• US 20070296835A1
• Filed: 06/06/2007
• Published: 12/27/2007
• Est. Priority Date: 08/25/2005
• Status: Active Grant

• Current Assignee: Intellectual Ventures II LLC (Intellectual Ventures LLC)
• Original Assignee: Protarius Filo AG LLC (Intellectual Ventures LLC)
• Inventors: Gates, James, Sato, Darryl, Olsen, Richard
• Granted Patent: US 7,964,835 B2
• Time in Patent Office: Days
• US Class Current: 348/234
• CPC Class Codes: G02B 3/0031 Replication or moulding, e.g. hot embossing, UV-casting, injection moulding
• CPC Class Codes: G02B 3/0043 Inhomogeneous or irregular arrays, e.g. varying shape, size, height
• CPC Class Codes: G02B 3/0062 Stacked lens arrays, i.e. refractive surfaces arranged in at least two planes, without structurally separate optical elements in-between
• CPC Class Codes: G02B 3/0075 characterized by non-optical structures, e.g. having integrated holding or alignment means
• CPC Class Codes: G02B 9/12 having three components only
• CPC Class Codes: H01L 27/14618 Containers
• CPC Class Codes: H01L 27/14621 Colour filter arrangements
• CPC Class Codes: H01L 27/14625 Optical elements or arrangements associated with the device
• CPC Class Codes: H01L 27/14634 Assemblies, i.e. Hybrid structures
• CPC Class Codes: H01L 27/14645 Colour imagers
• CPC Class Codes: H01L 27/14652 Multispectral infrared imagers, having a stacked pixel-element structure, e.g. npn, npnpn or MQW structures
• CPC Class Codes: H01L 2924/00 Indexing scheme for arrangements or methods for connecting or disconnecting semiconductor or solid-state bodies as covered by H01L24/00
• CPC Class Codes: H01L 2924/0002 Not covered by any one of groups H01L24/00, H01L24/00 and H01L2224/00
• CPC Class Codes: H01L 31/0232 Optical elements or arrangements associated with the device H01L31/0236 takes precedence; for photovoltaic cells H01L31/054; for photovoltaic modules H02S40/20
• CPC Class Codes: H01L 31/02325 the optical elements not being integrated nor being directly associated with the device
• CPC Class Codes: H04N 2209/049 having three pick-up sensors
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/13 with multiple sensors
• CPC Class Codes: H04N 23/16 Optical arrangements associated therewith, e.g. for beam-splitting or for colour correction
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 23/54 : Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/71 : Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 23/73 : by influencing the exposure time
• CPC Class Codes: H04N 23/843 : Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 23/88 : for colour balance, e.g. white-balance circuits or colour temperature control
• CPC Class Codes: H04N 25/00 : Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 25/134 : based on three different wavelength filter elements
• CPC Class Codes: H04N 25/41 : Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors
• CPC Class Codes: H04N 25/53 : Control of the integration time
• CPC Class Codes: H04N 25/531 : by controlling rolling shutters in CMOS SSIS
• CPC Class Codes: H04N 5/33 : Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Digital camera systems and methods are described that provide a color digital camera with direct luminance detection. The luminance signals are obtained directly from a broadband image sensor channel without interpolation of RGB data. The chrominance signals are obtained from one or more additional image sensor channels comprising red and/or blue color band detection capability. The red and blue signals are directly combined with the luminance image sensor channel signals. The digital camera generates and outputs an image in YCrCb color space by directly combining outputs of the broadband, red and blue sensors.

Claims

1. A digital camera comprising:

a broadband channel including a first optics component and a first image sensor integrated in a substrate and configured to directly detect luminance data;

at least one color channel each of which includes an optics component and an image sensor integrated in a substrate and configured to directly detect chrominance data; and

a processing component coupled to the first and second channels and configured to directly combine outputs of the first and second channels to provide an image in YCrCb color space.

2. The digital camera of claim 1, wherein the broadband channel is configured to detect data approximately in a range of 400 nanometers (nm) to 700 nm.

3. The digital camera of claim 1, wherein the broadband channel is configured to detect data approximately in a range of 400 nm up to 1060 nm.

4. The digital camera of claim 1, wherein the broadband channel is configured to detect data approximately in a range of 250 nm up to 1060 nm.

5. The digital camera of claim 1, wherein directly detecting chrominance data excludes interpolation of the data from the at least one color channel.

6. The digital camera of claim 1, wherein the at least one color channel includes one color channel with one image sensor configured to detect data in a red and a blue color band.

7. The digital camera of claim 6, wherein directly detecting chrominance data includes generating red chroma and blue chroma data through direct combination of the red and blue color band data with the luminance data.

8. The digital camera of claim 1, wherein the at least one color channel includes a plurality of color channels comprising:

a first color channel with a first image sensor configured to detect data in a red color band; and

a second color channel with a second image sensor configured to detect data in a blue color band.

9. The digital camera of claim 8, wherein directly detecting chrominance data comprises:

generating red chroma data through direct combination of the read color band data with the luminance data; and

generating blue chroma data through direct combination of the blue color band data with the luminance data.

10. The digital camera of claim 9, wherein directly detecting chrominance data excludes interpolation of the red color band and blue color band data.

11. The digital camera of claim 8, wherein the plurality of color channels includes a third color channel with a third image sensor configured to detect data in a green color band.

12. The digital camera of claim 11, wherein directly detecting the chrominance data excludes use of the green color band data.

13. The digital camera of claim 11, wherein the processing component directly outputs RGB data from the plurality of color channels.

14. The digital camera of claim 13, wherein the processing component simultaneously outputs the RGB data and YCrCb data.

15. The digital camera of claim 1, wherein the processing component is configured to separately and simultaneously control an integration time of each image sensor during a frame, wherein the integration time of at least one sensor is relatively shorter than the integration time of at least one other sensor.

16. The digital camera of claim 15, wherein the processing component is configured to use the image signal level to determine the integration time for each image sensor for a subsequent frame.

17. The digital camera of claim 15, wherein the processing component being configured to separately and simultaneously control the integration time of each image sensor comprises controlling the integration time of each image sensor so collectively the integration times of the image sensors correspond to a range of optical signal levels incident on the broadband and color channels.

18. The digital camera of claim 15, wherein the processing component is configured to control the integration time of each image sensor using a control technique selected from a group consisting of automatic control and control in response to an input by an operator.

19. The digital camera of claim 1, wherein the first optics component includes a different configuration than the second optics component.

20. The digital camera of claim 1, wherein one or more of the first and second optics components includes a plurality of lenses.

21. The digital camera of claim 1, wherein the second optics component includes at least one filter.

22. The digital camera of claim 1, wherein the first optics component has a first field of view (FOV) and the second optics component has a second FOV that is different than the first FOV.

23. The digital camera of claim 1, wherein the first optics component has a first focal length and the second optics component has a second focal length that is different than the first focal length.

24. The digital camera of claim 1, wherein the first image sensor has a first set of dimensions and the second image sensor has a second set of dimensions at least one of which is different than a dimension of the first set of dimensions.

25. A digital camera comprising:

a plurality of channels, wherein each channel includes an optics component and an array of photo-detectors integrated on a semiconductor substrate, the plurality of channels including a first channel configured to directly detect luminance data and at least one other channel configured to directly detect chrominance data; and

a processing component coupled to the plurality of channels, the processing component configured to independently control simultaneous data acquisition with each of the plurality of channels, the processing component configured to directly combine data from the plurality of channels to provide an image in YCrCb color space, wherein the direct combining excludes interpolation of the data from the plurality of channels.

26. The digital camera of claim 25, wherein the first channel is configured to detect data approximately in a range of 400 nanometers (nm) to 700 nm.

27. The digital camera of claim 25, wherein directly detecting chrominance data excludes interpolation of the data from the at least one color channel.

28. The digital camera of claim 25, wherein the at least one other channel includes a second channel configured to detect data in a red and a blue color band.

29. The digital camera of claim 28, wherein directly detecting chrominance data includes generating red chroma and blue chroma data through direct combination of the red and blue color band data with the luminance data.

30. The digital camera of claim 25, wherein the at least one other channel comprises:

a second channel configured to detect data in a red color band; and

a third channel configured to detect data in a blue color band.

31. The digital camera of claim 30, wherein directly detecting chrominance data comprises:

generating red chroma data through direct combination of the read color band data with the luminance data; and

generating blue chroma data through direct combination of the blue color band data with the luminance data, wherein directly detecting chrominance data excludes interpolation of the red color band and blue color band data.

32. The digital camera of claim 30, wherein the at least one other channel comprises a fourth color channel configured to detect data in a green color band, wherein directly detecting the chrominance data excludes use of the green color band data, wherein the processing component directly outputs RGB data from the at least one other channel.

33. The digital camera of claim 25, wherein the processing component is configured to separately and simultaneously control an integration time of each array of photo-detectors during a frame.

34. The digital camera of claim 33, wherein the processing component is configured to use the image to determine the integration time for each array of photo-detectors for a subsequent frame.

35. The digital camera of claim 25, wherein the optics components of at least two of the plurality of channels have different configurations.

36. A digital camera comprising:

a first channel including a first optics component and a first image sensor, the first channel configured to image a white color band and directly output luminance data;

a second channel including a second optics component and a second image sensor, the second channel configured to image at least one of a red and a blue color band and output chrominance data; and

a processing component coupled to the first and second channels and configured to directly combine outputs of the first and second channels to provide an image in YCrCb color space without interpolation in RGB color space.

37. The digital camera of claim 36, wherein outputting the chrominance data includes generating red chroma data through direct combination of the red color band data with the luminance data without interpolation of RGB data.

38. The digital camera of claim 36, wherein outputting the chrominance data includes generating blue chroma data through direct combination of the blue color band data with the luminance data without interpolation of RGB data.

39. The digital camera of claim 36, wherein the first channel is configured to detect data approximately in a range of 400 nanometers (nm) to 700 nm.

40. The digital camera of claim 36, wherein the second channel is configured to detect data in the red color band, the digital camera comprising a third channel with a third optics component and a third image sensor configured to detect data in the blue color band.

41. The digital camera of claim 40, comprising a fourth channel with a fourth optics component and a fourth image sensor configured to detect data in a green color band, wherein output of the chrominance data excludes use of the green color band data.

42. The digital camera of claim 41, wherein the processing component directly outputs RGB data from the second, third and fourth channels.

43. A method comprising:

detecting luminance data using a broadband sensor of a digital camera by directly sensing and outputting a broadband wavelength response at the broadband sensor and excluding use of RGB color data;

generating chrominance data using at least one color sensor of the digital camera by sensing and outputting at least one color band wavelength response; and

generating an image in YCrCb color space by directly combining outputs of the first and second sensors and outputting the image from the digital camera.

44. The method of claim 43, wherein generating the chrominance data excludes interpolation of the at least one color band wavelength response.

45. The method of claim 43, wherein generating the chrominance data comprises sensing data in a red color band and generating red chroma data through direct combination of the read color band data with the luminance data.

46. The method of claim 45, wherein generating the chrominance data comprises sensing data in a blue color band and generating blue chroma data through direct combination of the blue color band data with the luminance data.

47. The method of claim 45, comprising sensing data in a green color band.

48. The method of claim 47, comprising generating an image in RGB color space by directly combining data of the red, blue and green color bands and outputting the image from the digital camera.

49. The method of claim 43, wherein detecting the luminance data comprises detecting data approximately in a range of 400 nanometers (nm) to 700 nm.

50. A method comprising:

directly detecting luminance data using a broadband wavelength response and excluding use of RGB color data;

directly generating chrominance data by detecting red color band data and blue color band data, generating a red chroma component by directly combining the red color band data with the luminance data, and generating a blue chroma component by directly combining the blue color band data with the luminance data; and

outputting an image in YCrCb color space using the luminance and chrominance data.

51. The method of claim 50, comprising detecting green color band data and outputting an image in RGB color space.

52. A computer readable medium containing executable instructions which when executed in a processing system provide direct luminance and chrominance detection by:

detecting luminance data using a broadband sensor of a digital camera by directly sensing and outputting a broadband wavelength response at the broadband sensor and excluding use of RGB color data;

generating chrominance data using at least one color sensor of the digital camera by sensing and outputting at least one color band wavelength response; and

generating an image in YCrCb color space by directly combining outputs of the first and second sensors and outputting the image from the digital camera.