Digital camera with integrated infrared (IR) response

• US 7,566,855 B2
• Filed: 08/01/2007
• Issued: 07/28/2009
• Est. Priority Date: 08/25/2005
• Status: Active Grant

• Current Assignee: Intellectual Ventures II LLC (Intellectual Ventures LLC)
• Original Assignee: Protarius Filo AG LLC (Intellectual Ventures LLC)
• Inventors: Sun, Feng-Qing, Sato, Darryl L., Olsen, Richard Ian, Gates, James, Oten, Remzi, Moller, Borden, Vitomirov, Olivera, Brady, Jeffrey A., Gunawan, Ferry
• Primary Examiner(s): Sohn; Seung C
• Application Number:
  US11/888,570
• Publication Number:
  US 20080029714A1
• Time in Patent Office: 727 Days
• Field of Search: 250/208.1
• US Class Current: 250/208.1
• CPC Class Codes: G02B 13/0035 having three lenses
• CPC Class Codes: G02B 7/04 with mechanism for focusing or varying magnification
• CPC Class Codes: G03B 13/18 Focusing aids
• CPC Class Codes: H01L 27/14618 Containers
• CPC Class Codes: H01L 27/14621 Colour filter arrangements
• CPC Class Codes: H01L 27/14625 Optical elements or arrangements associated with the device
• CPC Class Codes: H01L 27/14634 Assemblies, i.e. Hybrid structures
• CPC Class Codes: H01L 27/1469 Assemblies, i.e. hybrid integration
• CPC Class Codes: H01L 2924/00 Indexing scheme for arrangements or methods for connecting or disconnecting semiconductor or solid-state bodies as covered by H01L24/00
• CPC Class Codes: H01L 2924/0002 Not covered by any one of groups H01L24/00, H01L24/00 and H01L2224/00
• CPC Class Codes: H01L 31/02325 the optical elements not being integrated nor being directly associated with the device
• CPC Class Codes: H04N 23/16 Optical arrangements associated therewith, e.g. for beam-splitting or for colour correction
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 25/79 Arrangements of circuitry being divided between different or multiple substrates, chips or circuit boards, e.g. stacked image sensors

Abstract

There are many, many inventions described herein. In one aspect, what is disclosed is a digital camera including a plurality of arrays of photo detectors, including a first array of photo detectors to sample an intensity of light of a first wavelength and a second array of photo detectors to sample an intensity of light of a second wavelength. The digital camera further may also include a first lens disposed in an optical path of the first array of photo detectors, wherein the first lens includes a predetermined optical response to the light of the first wavelength, and a second lens disposed in with an optical path of the second array of photo detectors wherein the second lens includes a predetermined optical response to the light of the second wavelength. In addition, the digital camera may include signal processing circuitry, coupled to the first and second arrays of photo detectors, to generate a composite image using (i) data which is representative of the intensity of light sampled by the first array of photo detectors, and (ii) data which is representative of the intensity of light sampled by the second array of photo detectors; wherein the first array of photo detectors, the second array of photo detectors, and the signal processing circuitry are integrated on or in the same semiconductor substrate.

Claims

1. A digital camera comprising:

a plurality of arrays of photo detectors disposed upon a substrate, wherein the plurality of arrays of photo detectors includes

a first array of photo detectors configured to sample light of a visible spectrum; and

a second array of photo detectors configured to sample infrared (IR) radiation; and

a processor operatively coupled to the plurality of arrays of photo detectors, wherein the processor includes

a first channel processor configured to receive first output data of the first array of photo detectors;

a second channel processor configured to receive second output data of the second array of photo detectors; and

an image pipeline configured to generate a composite image based at least in part on the first output data and the second output data.

2. The digital camera of claim 1, wherein the light of the visible spectrum includes light of a color in a spectrum bounded by violet and red colors of an optical spectrum.

3. The digital camera of claim 1, wherein the light of the visible spectrum includes light of a plurality of colors in a spectrum bounded by violet and red colors of an optical spectrum.

4. The digital camera of claim 1, wherein the second array of photo detectors samples the IR radiation in a region of an IR spectrum.

5. The digital camera of claim 1, wherein the second array samples the IR radiation in a plurality of regions of an IR spectrum.

6. The digital camera of claim 1, wherein the second array of photo detectors samples the IR radiation in a first region of an IR spectrum, wherein the first region of the IR spectrum extends beyond red colors of an optical spectrum.

7. The digital camera of claim 1, further comprising a third array of photo detectors configured to sample second light of the visible spectrum, wherein the processor is operatively coupled to the third array of photo detectors, and further wherein the composite image is based at least in part on third output data of the third array of photo detectors.

8. The digital camera of claim 7, further comprising a fourth array of photo detectors configured to sample third light of the visible spectrum, wherein the processor is operatively coupled to the fourth array of photo detectors, and further wherein the composite image is based at least in part on fourth output data of the fourth array of photo detectors.

9. The digital camera of claim 1, further comprising a third array of photo detectors configured to sample second IR radiation, wherein the processor is operatively coupled to the third array of photo detectors, and further wherein the composite image is based at least in part on third output data of the third array of photo detectors.

10. The digital camera of claim 9, wherein the IR radiation is in a first region of an IR spectrum and the second IR radiation is in a second region of the IR spectrum.

11. The digital camera of claim 1, wherein the second array of photo detectors is configured to sample the IR radiation based at least in part on an infrared photo detection process that results from silicon optical absorption properties of the substrate.

12. The digital camera of claim 11, wherein the second array of photo detectors is configured to sample the IR radiation based at least in part on a material absorption process that extends a photo detector photo response up to a cutoff wavelength of silicon.

13. The digital camera of claim 12, wherein the cutoff wavelength of the photo detector photo response is approximately 1.1 microns.

14. The digital camera of claim 12, wherein the second array includes at least one thermal detector operatively coupled to the processor.

15. The digital camera of claim 14, wherein a photo response of the thermal detector is in a range of 1.1 microns to 14 microns.

16. The digital camera of claim 14, wherein the thermal detector is thermally isolated on a top of the substrate.

17. The digital camera of claim 1, further comprising a frame including a spacer and a support, wherein the spacer is mounted on the substrate and includes a lower set of apertures, wherein the support is detachably coupled to the spacer and includes an upper set of apertures, and wherein the lower set of apertures and the upper set of apertures form a discontinuous optical path to at least one of the plurality of arrays of photo detectors.

18. The digital camera of claim 17, further comprising a first optics component configured to pass the light of the visible spectrum and a second optics component configured to pass the infrared radiation, wherein the first optics component and the second optics component each include a plurality of optical lenses and a lens barrel configured to position and hold the plurality of optical lenses, and wherein the first optics component and the second optics component are coupled to the frame.

19. The digital camera of claim 1, wherein the first channel processor is configured to generate a first image based at least in part on the first output data, and further wherein the first channel processor and the first array of photo detectors form a first camera channel.

20. The digital camera of claim 19, wherein the second channel processor is configured to generate a second image based at least in part on the second output data, wherein the second channel processor and the second array of photo detectors form a second camera channel.

21. The digital camera of claim 20, wherein the second camera channel operates in a passive mode such that available light from a scene is detected.

22. The digital camera of claim 20, further comprising an illumination source, wherein the second camera channel operates in an active mode using light directed by the illumination source to enhance a scene or derive additional information from the scene.

23. The digital camera of claim 22, wherein the illumination source is a pulsed infrared source.

24. The digital camera of claim 20, wherein the second camera channel is configured to collect infrared photons from a scene at a frame rate synchronous with the first camera channel.

25. The digital camera of claim 20, wherein the second camera channel is configured to collect infrared photons from a scene at a frame rate asynchronous with the first camera channel.

26. The digital camera of claim 20, wherein the second camera channel is configured for active covert imaging beyond human vision detection.

27. The digital camera of claim 20, wherein the second camera channel is configured for passive low light level imaging.

28. The digital camera of claim 20, wherein the second camera channel is configured for active low light level imaging.

29. The digital camera of claim 20, wherein the second camera channel is configured for active range detection.

30. The digital camera of claim 20, wherein the first channel processor is configured to generate first control data to control the first camera channel.

31. The digital camera of claim 30, wherein the second channel processor is configured to generate second control data to control the second camera channel.

32. The digital camera of claim 19, wherein the first camera channel is configured to operate as a stand-alone imager.

33. The digital camera of claim 20, wherein the first camera channel is configured to operate in combination with the second camera channel.

34. The digital camera of claim 20, wherein the second camera channel is configured to operate as a stand-alone imager.

35. The digital camera of claim 20, wherein the second channel processor controls the second camera channel to operate in combination with the first camera channel.

36. The digital camera of claim 1, wherein the light of the visible spectrum has a first set of wavelengths.

37. The digital camera of claim 36, wherein the IR radiation has a second set of wavelengths.

38. The digital camera of claim 19, wherein the first channel processor is configured to control at least one parameter of the first camera channel, wherein the at least one parameter comprises gain, noise, dynamic range, linearity, double sampling, black level clamp, or pixel substitution.

39. The digital camera of claim 20, wherein the second channel processor is configured to control at least one parameter of the second camera channel, wherein the at least one parameter comprises gain, noise, dynamic range, linearity, double sampling, black level clamp, or pixel substitution.

40. The digital camera of claim 20, wherein the first channel processor is configured to control exposure of the first camera channel and the second channel processor is configured to control exposure of the second camera channel.

41. The digital camera of claim 20, wherein the first channel processor is configured to control a first camera channel integration time using a first integration time, and wherein the second channel processor is configured to control a second camera channel integration time using a second integration time, wherein the first integration time is different than the second integration time.

42. The digital camera of claim 20, wherein the image pipeline is configured to generate the composite image based at least in part on the first image and the second image.

43. The digital camera of claim 42, wherein the image pipeline is further configured to generate control data to control the first camera channel and the second camera channel.

44. The digital camera of claim 43, wherein the image pipeline is configured to control a first intensity of the light from the first camera channel and a second intensity of the IR radiation from the second camera channel.

45. The digital camera of claim 1, wherein the processor is disposed upon the substrate.

46. The digital camera of claim 1, wherein the image pipeline is configured to perform at least one of analog and digital signal processing, color space conversion, channel mapping, exposure control, integration time control of at least one of the plurality of arrays of photo detectors, pixel correction, double sampling, image/color plane integration, parallax estimation and correction, image plane alignment and stitching, pixel binning and windowing, image interpolation, focus control, gamma control, color correction, enhancement and balance, edge enhancement and image sharpening, chroma and random noise reduction, white balance, image scaling, zoom, or resizing.

47. The digital camera of claim 1, wherein the processor further comprises a postprocessor coupled to the image pipeline.

48. The digital camera of claim 47, wherein the postprocessor is configured to perform at least one of analog and digital signal processing, downsampling, still image encoding, video encoding, frame buffering, or output interfacing.

49. The digital camera of claim 47, wherein the postprocessor is disposed upon the substrate.

50. The digital camera of claim 1, further comprising timing and control logic configured to provide timing and control information to the first channel processor and the second channel processor.

51. The digital camera of claim 50, wherein the timing and control logic is disposed upon the substrate.

52. The digital camera of claim 1, further comprising a memory configured to store the first output data and the second output data.

53. The digital camera of claim 52, wherein the memory is disposed upon the substrate.

54. The digital camera of claim 1, wherein the image pipeline comprises a color plane integrator configured to combine at least one first color plane from the first output data and at least one second color plane from the second output data into the composite image.

55. The digital camera of claim 54, wherein the image pipeline further comprises an image adjustor configured to adjust the composite image.

56. A method for capturing an image, the method comprising:

sampling light of a visible spectrum via a first array of photo detectors;

sampling infrared (IR) radiation via a second array of photo detectors;

receiving, at a first channel processor, first output data of the first array of photo detectors;

receiving, at a second channel processor, second output data of the second array of photo detectors; and

generating, via an image pipeline operatively coupled to the first channel processor and the second channel processor, a composite image based at least in part on the first output data and the second output data.

57. The method of claim 56, further comprising:

generating by the first channel processor of a first image based at least in part on the first output data; and

generating by the second channel processor of a second image based at least in part on the second output data, wherein the composite image comprises a combination of at least the first image and the second image.

58. The method of claim 56, wherein the first output data includes at least one first color plane and the second output data includes at least one second color plane, and further comprising combining the at least one first color plane and the at least one second color plane to form the composite image.

59. The method of claim 58, wherein the at least one first color plane and the at least one second color plane are combined using a color plane integrator of the image pipeline.