Digital camera with multiple pipeline signal processors

• US 20100208100A9
• Filed: 08/01/2007
• Published: 08/19/2010
• Est. Priority Date: 08/25/2004
• Status: Active Grant

• Current Assignee: Intellectual Ventures II LLC (Intellectual Ventures LLC)
• Original Assignee: Newport Imaging Corporation
• Inventors: Brady, Jeffrey, Olsen, Richard, Sato, Darryl, Sun, Feng-Qing, Gates, James, Oten, Remzi, Moller, Borden, Vitomirov, Olivera, Gunawan, Ferry
• Granted Patent: US 7,884,309 B2
• Time in Patent Office: Days
• US Class Current: 348/227.100
• CPC Class Codes: G02B 13/0035 having three lenses
• CPC Class Codes: G02B 3/0031 Replication or moulding, e.g. hot embossing, UV-casting, injection moulding
• CPC Class Codes: G02B 3/0043 Inhomogeneous or irregular arrays, e.g. varying shape, size, height
• CPC Class Codes: G02B 3/0062 Stacked lens arrays, i.e. refractive surfaces arranged in at least two planes, without structurally separate optical elements in-between
• CPC Class Codes: G02B 3/0075 characterized by non-optical structures, e.g. having integrated holding or alignment means
• CPC Class Codes: G02B 7/04 with mechanism for focusing or varying magnification
• CPC Class Codes: G02B 9/12 having three components only
• CPC Class Codes: G03B 13/18 Focusing aids
• CPC Class Codes: H01L 27/14618 Containers
• CPC Class Codes: H01L 27/14621 Colour filter arrangements
• CPC Class Codes: H01L 27/14625 Optical elements or arrangements associated with the device
• CPC Class Codes: H01L 27/14634 Assemblies, i.e. Hybrid structures
• CPC Class Codes: H01L 27/14645 Colour imagers
• CPC Class Codes: H01L 27/1469 Assemblies, i.e. hybrid integration
• CPC Class Codes: H01L 2924/00 Indexing scheme for arrangements or methods for connecting or disconnecting semiconductor or solid-state bodies as covered by H01L24/00
• CPC Class Codes: H01L 2924/0002 Not covered by any one of groups H01L24/00, H01L24/00 and H01L2224/00
• CPC Class Codes: H01L 31/0232 Optical elements or arrangements associated with the device H01L31/0236 takes precedence; for photovoltaic cells H01L31/054; for photovoltaic modules H02S40/20
• CPC Class Codes: H01L 31/02325 the optical elements not being integrated nor being directly associated with the device
• CPC Class Codes: H04N 13/214 using spectral multiplexing
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 2209/049 : having three pick-up sensors
• CPC Class Codes: H04N 23/11 : for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/13 : with multiple sensors
• CPC Class Codes: H04N 23/16 : Optical arrangements associated therewith, e.g. for beam-splitting or for colour correction
• CPC Class Codes: H04N 23/40 : Circuit details for pick-up tubes
• CPC Class Codes: H04N 23/54 : Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/57 : Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/69 : Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/73 : by influencing the exposure time
• CPC Class Codes: H04N 23/84 : for processing colour signals
• CPC Class Codes: H04N 25/00 : Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 25/134 : based on three different wavelength filter elements
• CPC Class Codes: H04N 25/41 : Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors
• CPC Class Codes: H04N 25/531 : by controlling rolling shutters in CMOS SSIS
• CPC Class Codes: H04N 5/265 : Mixing

Abstract

There are many, many inventions described herein. In one aspect, what is disclosed is a digital camera including a plurality of arrays of photo detectors, including a first array of photo detectors to sample an intensity of light of a first wavelength and a second array of photo detectors to sample an intensity of light of a second wavelength. The digital camera further may also include a first lens disposed in an optical path of the first array of photo detectors, wherein the first lens includes a predetermined optical response to the light of the first wavelength, and a second lens disposed in with an optical path of the second array of photo detectors wherein the second lens includes a predetermined optical response to the light of the second wavelength. In addition, the digital camera may include signal processing circuitry, coupled to the first and second arrays of photo detectors, to generate a composite image using (i) data which is representative of the intensity of light sampled by the first array of photo detectors, and (ii) data which is representative of the intensity of light sampled by the second array of photo detectors; wherein the first array of photo detectors, the second array of photo detectors, and the signal processing circuitry are integrated on or in the same semiconductor substrate.

Claims

1-67. -67. (canceled)

68. A digital camera comprising:

a plurality of arrays of photo detectors, including;

a first array of photo detectors to sample an intensity of light; and

a second array of photo detectors to sample an intensity of light;

a first channel processor coupled to the first array and configured to generate a first image using first array data which is representative of the intensity of light sampled by the first array;

a second channel processor coupled to the second array and configured to generate a second image using second array data which is representative of the intensity of light sampled by the second array; and

a semiconductor substrate, wherein the first array, the second array, the first channel processor and the second channel processor are integrated on or in the substrate.

69. The camera of claim 68, wherein the first channel processor is configured to generate first control data to control a first camera channel that includes the first array and the first channel processor.

70. The camera of claim 69, wherein the second channel processor is configured to generate second control data to control a second camera channel that includes the second array and the second channel processor.

71. The camera of claim 70, wherein the first channel processor controls the first camera channel to operate as a stand-alone imager.

72. The camera of claim 70, wherein the first channel processor controls the first camera channel to operate in combination with the second camera channel.

73. The camera of claim 70, wherein the second channel processor controls the second camera channel to operate as a stand-alone imager.

74. The camera of claim 70, wherein the second channel processor controls the second camera channel to operate in combination with the first camera channel.

75. The camera of claim 70, wherein the first channel processor is configured to process the first array data of light of a first wavelength.

76. The camera of claim 75, wherein the second channel processor is configured to process the second array data of light of a second wavelength.

77. The camera of claim 70, wherein the first channel processor is configured to process the first array data of light of a first set of wavelengths.

78. The camera of claim 77, wherein the second channel processor is configured to process the second array data of light of a second set of wavelengths.

79. The camera of claim 78, wherein the first set of wavelengths and the second set of wavelengths each include at least one common wavelength.

80. The camera of claim 70, wherein the first channel processor is configured to control a gain of the first camera channel and the second channel processor is configured to control a gain of the second camera channel.

81. The camera of claim 70, wherein the first channel processor is configured to control noise of the first camera channel and the second channel processor is configured to control noise of the second camera channel.

82. The camera of claim 70, wherein the first channel processor is configured to control dynamic range of the first camera channel and the second channel processor is configured to control dynamic range of the second camera channel.

83. The camera of claim 70, wherein the first channel processor is configured to control linearity of the first camera channel and the second channel processor is configured to control linearity of the second camera channel.

84. The camera of claim 70, wherein the first channel processor is configured to control exposure of the first camera channel and the second channel processor is configured to control exposure of the second camera channel.

85. The camera of claim 70, wherein the first channel processor is configured to control double sampling of the first camera channel and the second channel processor is configured to control double sampling of the second camera channel.

86. The camera of claim 70, wherein the first channel processor is configured to control black level clamp of the first camera channel and the second channel processor is configured to control black level clamp of the second camera channel.

87. The camera of claim 70, wherein the first channel processor is configured to control pixel substitution of the first camera channel and the second channel processor is configured to control pixel substitution sampling of the second camera channel.

88. The camera of claim 70, wherein the first channel processor is configured to control an integration time of the first camera channel and the second channel processor is configured to control an integration time of the second camera channel.

89. The camera of claim 88, wherein the first channel processor is configured to control the integration time using a first integration time and the second channel processor is configured to control the integration time using a second integration time, wherein the first integration time is different than the second integration time.

90. The camera of claim 68, comprising a pipeline processor coupled to the first channel processor and the second channel processor, the pipeline processor configured to generate a composite image using data of the first image and the second image.

91. The camera of claim 90, wherein the pipeline processor is configured to generate control data to control a first camera channel that includes the first array and the first channel processor and a second camera channel that includes the second array and the second channel processor.

92. The camera of claim 91, wherein the pipeline processor is configured to control, the intensity of light from the first camera channel and the second camera channel.

93. The camera of claim 90, wherein the pipeline processor is integrated on or in the substrate.

94. The camera of claim 90, wherein the pipeline processor is configured to perform analog and digital signal processing.

95. The camera of claim 90, wherein the pipeline processor is configured to perform processing including color space conversion.

96. The camera of claim 90, wherein the pipeline processor is configured to perform processing including channel mapping.

97. The camera of claim 90, wherein the pipeline processor is configured to perform processing including exposure control.

98. The camera of claim 90, wherein the pipeline processor is configured to perform processing including integration time control of at least one of the plurality of arrays of photo detectors.

99. The camera of claim 90, wherein the pipeline processor is configured to perform processing including pixel correction.

100. The camera of claim 90, wherein the pipeline processor is configured to perform processing including double sampling.

101. The camera of claim 90, wherein the pipeline processor is configured to perform processing including image/color plane integration.

102. The camera of claim 90, wherein the pipeline processor is configured to perform processing including parallax estimation and correction.

103. The camera of claim 90, wherein the pipeline processor is configured to perform processing including image plane alignment and stitching.

104. The camera of claim 90, wherein the pipeline processor is configured to perform processing including pixel binning and windowing.

105. The camera of claim 90, wherein the pipeline processor is configured to perform processing including image interpolation.

106. The camera of claim 90, wherein the pipeline processor is configured to perform processing including focus control.

107. The camera of claim 90, wherein the pipeline processor is configured to perform processing including gamma control.

108. The camera of claim 90, wherein the pipeline processor is configured to perform processing including color correction.

109. The camera of claim 90, wherein the pipeline processor is configured to perform processing including enhancement and balance.

110. The camera of claim 90, wherein the pipeline processor is configured to perform processing including edge enhancement and image sharpening.

111. The camera of claim 90, wherein the pipeline processor is configured to perform processing including chroma and random noise reduction.

112. The camera of claim 90, wherein the pipeline processor is configured to perform processing including white balance.

113. The camera of claim 90, wherein the pipeline processor is configured to perform processing including image scaling.

114. The camera of claim 90, wherein the pipeline processor is configured to perform processing including zoom and resizing.

115. The camera of claim 68, comprising a postprocessor coupled to the pipeline processor.

116. The camera of claim 115, wherein the postprocessor is integrated on or in the substrate.

117. The camera of claim 115, wherein the postprocessor is configured to perform analog and digital signal processing.

118. The camera of claim 115, wherein the postprocessor is configured to perform processing selected from a group consisting of downsampling, still image encoding, video encoding, frame buffering, and output interfacing.

119. The camera of claim 68, comprising timing and control logic configured to provide timing and control information to the first channel processor and the second channel processor.

120. The camera of claim 119, wherein the timing and control logic is integrated on or in the substrate.

121. The camera of claim 68, comprising a memory configured to store the first array data and the second array data.

122. The camera of claim 121, wherein the memory is integrated on or in the substrate.

123. The camera of claim 68, comprising a frame including a spacer and a support, wherein the spacer is mounted on the substrate and includes a lower set of apertures, wherein the support is detachably coupled to the spacer and includes an upper set of apertures, wherein the lower and upper set of apertures form a discontinuous optical path to the photo detectors.

124. The camera of claim 123, comprising a plurality of optics components, wherein each optics component includes a plurality of optical lenses and a lens barrel configured to position and hold the plurality of optical lenses.

125. The camera of claim 124, wherein a first configuration of a first optics component is different from a second configuration of a second optics component.