Monolithically integrated RGB pixel array and Z pixel array

• US 10,291,870 B2
• Filed: 09/28/2017
• Issued: 05/14/2019
• Est. Priority Date: 12/22/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Wan, Chung Chun
• Primary Examiner(s): Walker, Jared
• Application Number:
  US15/718,478
• Publication Number:
  US 20180020179A1
• Time in Patent Office: 593 Days
• Field of Search: 348135
• CPC Class Codes: G01S 17/08 for measuring distance only indirect measurement G01S17/46; active triangulation systems G01S17/48
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G01S 7/4813 Housing arrangements
• CPC Class Codes: G01S 7/4816 of receivers alone
• CPC Class Codes: G01S 7/4863 Detector arrays, e.g. charge-transfer gates
• CPC Class Codes: G01S 7/4914 of detector arrays, e.g. charge-transfer gates
• CPC Class Codes: H01L 27/14618 Containers
• CPC Class Codes: H01L 27/14621 Colour filter arrangements
• CPC Class Codes: H01L 27/14625 Optical elements or arrangements associated with the device
• CPC Class Codes: H01L 27/14627 Microlenses
• CPC Class Codes: H01L 27/14634 Assemblies, i.e. Hybrid structures
• CPC Class Codes: H01L 27/14645 Colour imagers
• CPC Class Codes: H01L 27/14649 Infrared imagers
• CPC Class Codes: H01L 27/14685 Process for coatings or optical elements
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/667 Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 25/131 including elements passing infrared wavelengths
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 25/41 : Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors
• CPC Class Codes: H04N 25/705 : Pixels for depth measurement, e.g. RGBZ
• CPC Class Codes: H04N 25/745 : Circuitry for generating timing or clock signals

Abstract

An apparatus is described that includes first and second pixels arrays integrated on a same semiconductor chip. The first pixel array contains visible light pixels and no Z pixels. The second pixel array contains Z pixels and no visible light pixels. The first and second pixel arrays do not overlap on said same semiconductor chip.

Claims

1. A device comprising:

a visible light optical system that is configured to pass visible light and to block infrared light, and a separate, infrared light optical system that is configured to pass infrared light and block visible light,

wherein the visible light optical system includes a first set of lenses that receives light reflected off an object and passes through the visible light and the infrared light optical system includes a second, different set of lenses that receives light reflected off the object and passes through the infrared light; and

an image sensor that includes a visible light pixel array for receiving visible light that passes through the visible light optical system, and an infrared light pixel array for receiving infrared light that passes through the separate, infrared light optical system, wherein the visible light pixel array and the infrared light pixel array are both mounted on a same semiconductor chip,

wherein a distance between a center of the visible light pixel array and a center of the infrared light pixel array is less than a distance between an optical axis of the first set of lenses of the visible light optical system and an optical axis associated with the second set of lenses of the infrared light optical system.

2. The device of claim 1, comprising a first system of mirrors that is configured to steer visible light that is output by the visible light optical system after the infrared light is blocked by the visible light optical system towards the visible light pixel array, and a second system of mirrors that is configured to steer infrared light that is output from the infrared light optical system after the visible light is blocked by the infrared light optical system towards the infrared light pixel array.

3. The device of claim 1, comprising, for each of the visible light pixel array and the infrared light pixel array, (i) timing and control circuitry for the pixel array, (ii) analog-to-digital circuitry for the pixel array, and (iii) timing and control circuitry for the pixel array, all mounted on the same semiconductor chip.

4. The device of claim 1, comprising:

for each of the visible light pixel array and the infrared light pixel array, timing and control circuitry for the pixel array, mounted on the same semiconductor chip, and

wherein, for each of the visible light pixel array and the infrared light pixel array, (i) analog-to-digital circuitry for the pixel array, and (ii) timing and control circuitry for the pixel array are not mounted on the same semiconductor chip.

5. The device of claim 1, wherein the infrared light pixel array is for capturing a pixelated depth profile of an object.

6. The device of claim 1, wherein the visible light pixel array and the infrared light pixel array do not overlap on the semiconductor chip.

7. The device of claim 1, wherein the semiconductor chip is mounted on a second semiconductor chip.

8. The device of claim 1, wherein pixels of the infrared light pixel array are smaller than pixels of the visible light pixel array.

9. The device of claim 1, wherein the infrared light pixel array and the visible light pixel array are the same size.

19. The device of claim 1, wherein the first set of lenses includes a plano-concave lens, a plano convex lens, and a convex lens.

20. The device of claim 2, wherein the first system of mirrors that is configured to steer visible light includes two mirrors off which the visible light reflects.

10. A system comprising:

a camera comprising;

a visible light optical system that is configured to pass visible light and to block infrared light, and a separate, infrared light optical system that is configured to pass infrared light and block visible light,

wherein the visible light optical system includes a first set of lenses that receives light reflected off an object and passes through the visible light and the infrared light optical system includes a second, different set of lenses that receives light reflected off the object and passes through the infrared light; and

an image sensor that includes a visible light pixel array for receiving visible light that passes through the visible light optical system, and an infrared light pixel array for receiving infrared light that passes through the separate, infrared light optical system, wherein the visible light pixel array and the infrared light pixel array are both mounted on a same semiconductor chip,

wherein a distance between a center of the visible light pixel array and a center of the infrared light pixel array is less than a distance between an optical axis of the first set of lenses of the visible light optical system and an optical axis associated with the second set of lenses of the infrared light optical system; and

one or more computer processors that are configured to receive signals output by the image sensor, and to generate a pixelated depth profile of an object position in front of the camera.

11. The system of claim 10, wherein the camera comprises a first system of mirrors that is configured to steer visible light that is output by the visible light optical system after the infrared light is blocked by the visible light optical system towards the visible light pixel array, and a second system of mirrors that is configured to steer infrared light that is output from the infrared light optical system after the visible light is blocked by the infrared light optical system towards the infrared light pixel array.

12. The system of claim 10, wherein the camera comprises, for each of the visible light pixel array and the infrared light pixel array, (i) timing and control circuitry for the pixel array, (ii) analog-to-digital circuitry for the pixel array, and (iii) timing and control circuitry for the pixel array, all mounted on the same semiconductor chip.

13. The system of claim 10, wherein the camera comprises:

for each of the visible light pixel array and the infrared light pixel array, timing and control circuitry for the pixel array, mounted on the same semiconductor chip, and

wherein, for each of the visible light pixel array and the infrared light pixel array, (i) analog-to-digital circuitry for the pixel array, and (ii) timing and control circuitry for the pixel array are not mounted on the same semiconductor chip.

14. The system of claim 10, wherein the visible light pixel array and the infrared light pixel array do not overlap on the semiconductor chip.

15. The system of claim 10, wherein the semiconductor chip is mounted on a second semiconductor chip.

16. The system of claim 10, wherein pixels of the infrared light pixel array are smaller than pixels of the visible light pixel array.

17. The system of claim 10, wherein the infrared light pixel array and the visible light pixel array are the same size.

18. A method comprising:

passing visible light through a visible light optical system that is configured block infrared light;

passing infrared light through a separate, infrared light optical system that is configured to block visible light,

wherein the visible light optical system includes a first set of lenses that receives light reflected off an object and passes through the visible light and the infrared light optical system includes a second, different set of lenses that receives light reflected off the object and passes through the infrared light; and

receiving, by an image sensor, the visible light at a visible light pixel array for receiving visible light that passes through the visible light optical system, and an infrared light pixel array for receiving infrared light that passes through the separate, infrared light optical system, wherein the visible light pixel array and the infrared light pixel array are both mounted on a same semiconductor chip,

wherein a distance between a center of the visible light pixel array and a center of the infrared light pixel array is less than a distance between an optical axis of the first set of lenses of the visible light optical system and an optical axis associated with the second set of lenses of the infrared light optical system.