Devices and methods for high-resolution image and video capture

• US 9,369,621 B2
• Filed: 05/03/2011
• Issued: 06/14/2016
• Est. Priority Date: 05/03/2010
• Status: Active Grant

• Current Assignee: InVisage Technologies, Inc. (Apple Inc.)
• Original Assignee: InVisage Technologies, Inc. (Apple Inc.)
• Inventors: Malone, Michael R., Nave, Pierre Henri Rene Della, Brading, Michael Charles, Lee, Jess Jan Young, Tian, Hui, Ivanov, Igor Constantin, Sargent, Edward Hartley
• Primary Examiner(s): NGUYEN, LUONG TRUNG
• Application Number:
  US13/099,903
• Publication Number:
  US 20110267510A1
• Time in Patent Office: 1,869 Days
• US Class Current: 1/1
• CPC Class Codes: H01L 27/14609 Pixel-elements with integrated switching, control, storage or amplification elements scanning details of imagers (circuitry of solid-state image sensors H04N25/00); circuitry of imagers H04N25/70
• CPC Class Codes: H01L 27/1462 Coatings
• CPC Class Codes: H01L 27/14621 Colour filter arrangements
• CPC Class Codes: H01L 27/14625 Optical elements or arrangements associated with the device
• CPC Class Codes: H01L 27/14627 Microlenses
• CPC Class Codes: H01L 27/14636 Interconnect structures
• CPC Class Codes: H01L 27/1464 Back illuminated imager structures
• CPC Class Codes: H01L 27/14645 Colour imagers
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 25/75 Circuitry for providing, modifying or processing image signals from the pixel array
• CPC Class Codes: H04N 25/76 Addressed sensors, e.g. MOS or CMOS sensors
• CPC Class Codes: H04N 25/771 comprising storage means other than floating diffusion
• CPC Class Codes: H04N 25/78 Readout circuits for addressed sensors, e.g. output amplifiers or A/D converters

Abstract

In various example embodiments, an imaging system and method are provided. In an embodiment, the system comprises a first image sensor array, a first optical system to project a first image on the first image sensor array, the first optical system having a first zoom level. A second optical system is to project a second image on a second image sensor array, the second optical system having a second zoom level. The second image sensor array and the second optical system are pointed in the same direction as the first image sensor array and the first optical system. The second zoom level is greater than the first zoom level such that the second image projected onto the second image sensor array is a zoomed in on portion of the first image projected on the first image sensor array. The first image sensor array includes at least four megapixels and the second image sensor array includes one-half or less than the number of pixels in the first image sensor array.

Claims

1. An imaging system comprising:

a first image sensor array;

a first optical system configured to project a first image on the first image sensor array, the first optical system having a first zoom level;

a second image sensor array, the first image sensor array and the second image sensor array being fabricated on a single integrated circuit, the first image sensor array employing pixels of a first linear dimension and the second image sensor array employing pixels of a second linear dimension, a ratio of the first linear dimension and the second linear dimension being greater than 1.3 to 1;

a second optical system configured to project a second image on the second image sensor array, the second optical system having a second zoom level, the second image sensor array and the second optical system are pointed in the same direction as the first image sensor array and the first optical system, the second zoom level being greater than the first zoom level such that the second image projected onto the second image sensor array is a zoomed in portion of the first image projected on the first image sensor array, and the first image sensor array includes at least four megapixels, and the second image sensor array includes about one-half or less than a number of pixels in the first image sensor array;

first pixel circuitry configured to read image data from the first image sensor array;

second pixel circuitry configured to read image data from the second image sensor array;

an electronic global shutter configured to stop charge integration between the first image sensor array and the first pixel circuitry and between the second image sensor array and the second pixel circuitry at substantially the same time; and

a photosensitive layer formed over a substrate with at least one of the first pixel circuitry and the second pixel circuitry being fabricated below at least a portion of the photosensitive layer.

2. The imaging system of claim 1, wherein the first image sensor array includes at least six megapixels.

3. The imaging system of claim 1, wherein the first image sensor array includes at least eight megapixels.

4. The imaging system of claim 1, wherein the second image sensor array includes four megapixels or less.

5. The imaging system of claim 1, wherein the second image sensor array includes two megapixels or less.

6. The imaging system of claim 1, wherein the second image sensor array includes one megapixel or less.

7. The imaging system of claim 1, wherein the first image sensor array includes a first array of first pixel regions and the second image sensor array includes a second array of second pixel regions, wherein each of the first pixel regions is larger than each of the second pixel regions.

8. The imaging system of claim 7, wherein each of the first pixel regions has a lateral distance across the first pixel region of less than about 2.5 microns.

9. The imaging system of claim 7, wherein each of the first pixel regions has an area of less than about 2.5 microns squared.

10. The imaging system of claim 7, wherein each of the first pixel regions has a lateral distance across the first pixel region of less than about 2 microns.

11. The imaging system of claim 7, wherein each of the first pixel regions has an area of less than about 2 microns squared.

12. The imaging system of claim 7, wherein each of the first pixel regions has a lateral distance across the first pixel region of less than about 1.5 microns.

13. The imaging system of claim 7, wherein each of the first pixel regions has an area of less than about 1.5 microns squared.

14. The imaging system of claim 7, wherein each of the second pixel regions has a lateral distance across the second pixel region of less than about 2.1 microns.

15. The imaging system of claim 7, wherein each of the second pixel regions has an area of less than about 2.1 microns squared.

16. The imaging system of claim 7, wherein each of the second pixel regions has a lateral distance across the second pixel region of less than about 1.6 microns.

17. The imaging system of claim 7, wherein each of the second pixel regions has an area of less than about 1.6 microns squared.

18. The imaging system of claim 7, wherein each of the second pixel regions has a lateral distance across the second pixel region of less than about 1.3 microns.

19. The imaging system of claim 7, wherein each of the second pixel regions has an area of less than about 1.3 microns squared.

20. The imaging system of claim 1, further comprising a third image sensor array and a third optical system configured to project a third image on the third image sensor array, the third optical system having a third zoom level;

wherein the third image sensor array and the third optical system are pointed in the same direction as the first image sensor array and the first optical system.

21. The imaging system of claim 20, wherein the third zoom level is greater than the second zoom level.

22. The imaging system of claim 20, wherein the third zoom level is less than the first zoom level.

23. The imaging system of claim 20, wherein the third image sensor array includes the same number of pixels as the second image sensor array.

24. The imaging system of claim 20, wherein the third image sensor array includes four megapixels or less.

25. The imaging system of claim 20, wherein the third image sensor array includes two megapixels or less.

26. The imaging system of claim 20, wherein the third image sensor array includes one megapixel or less.

27. The imaging system of claim 20, wherein the third image sensor array includes a third array of third pixel regions, wherein each of the third pixel regions is smaller than each of the first pixel regions.

28. The imaging system of claim 27, wherein each of the third pixel regions has a lateral distance across the third pixel region of less than about 1.9 microns.

29. The imaging system of claim 27, wherein each of the third pixel regions has an area of less than about 1.9 microns squared.

30. The imaging system of claim 27, wherein each of the third pixel regions has a lateral distance across the third pixel region of less than about 1.4 microns.

31. The imaging system of claim 27, wherein each of the third pixel regions has an area of less than about 1.4 microns squared.

32. The imaging system of claim 27, wherein each of the third pixel regions has a lateral distance across the third pixel region of less than about 1.2 microns.

33. The imaging system of claim 27, wherein each of the third pixel regions has an area of less than about 1.2 microns squared.

34. The imaging system of claim 1, wherein the first image sensor array and the second image sensor array are formed on the same substrate.

35. The imaging system of claim 20, wherein the first image sensor array, the second image sensor array, and the third image sensor array are formed on the same substrate.

36. The imaging system of claim 1, further comprising a user interface control configured to select a zoom level and circuitry to read out images from the first sensor array and the second sensor array to generate an output image based on the selected zoom level.

37. The imaging system of claim 1, wherein the first image is to be selected for output when the first zoom level is selected.

38. The imaging system of claim 1, wherein the second image is used to enhance the first image for output when the first zoom level is selected.

39. The imaging system of claim 1, wherein the second image is to be selected for output when the first zoom level is selected and the first image is used to enhance the second image.

40. The imaging system of claim 1, wherein the imaging system is part of a camera device and wherein a user control is to be selected to output both the first image and the second image from the camera device.

41. The imaging system of claim 20, wherein the imaging system is part of a camera device and wherein a user control is to be selected to output the first image, the second image, and the third image from the camera device.

42. The imaging system of claim 1, wherein the electronic global shutter is configured to stop the integration period for each pixel region in the first pixel sensor array and the second pixel sensor array within about one millisecond of one another.

43. The imaging system of claim 1, further comprising third pixel circuitry configured to read image data from a third image sensor array, wherein the electronic global shutter is configured to stop charge integration between the third image sensor array and the third pixel circuitry at substantially the same time as the first sensor array and the second sensor array.

44. The imaging system of claim 1, wherein the electronic global shutter is configured to stop the integration period for each pixel region in a third pixel sensor array within about one millisecond of each of the pixel regions in the first image sensor array and the second image sensor array.

45. An imaging system comprising:

a primary image sensor array;

a primary optical system configured to project a primary image on the primary image sensor array, the primary optical system having a first zoom level;

a plurality of secondary image sensor arrays, the primary image sensor array and the plurality of secondary image sensor arrays being fabricated on a single integrated circuit, the primary image sensor array employing pixels of a first linear dimension and the plurality of secondary image sensor arrays each employing pixels of a second linear dimension, a ratio of the first linear dimension and the second linear dimension being greater than 1.3 to 1;

a secondary optical system for each of the secondary image sensor arrays, each of the secondary optical systems being configured to project a secondary image on a respective one of the secondary image sensor arrays, each of the secondary optical systems having a respective zoom level different than the first zoom level, each of the secondary image sensor arrays and each of the secondary optical systems are pointed in the same direction as the primary image sensor array and the primary optical system, the primary image sensor array being larger than each of the secondary image sensor arrays;

first pixel circuitry configured to read image data from the primary image sensor array;

second pixel circuitry configured to read image data from the plurality of secondary image sensor arrays;

an electronic global shutter configured to stop charge integration between the primary image sensor array and the first pixel circuitry and between a selected one or more of the plurality of secondary image sensor arrays and the second pixel circuitry at substantially the same time; and

a photosensitive layer formed over a substrate with at least one of the first pixel circuitry and the second pixel circuitry being fabricated below at least a portion of the photosensitive layer.

46. The imaging system of claim 45, further comprising a control circuit to output a primary image output based on the first image projected onto the primary image sensor array during a first mode of operation, wherein the primary image output is not to be generated based on any of the secondary images projected onto the secondary image arrays.

47. The imaging system of claim 45, further comprising a control circuit to output a primary image output based on the first image projected onto the primary image sensor array during a first mode of operation, wherein the primary image output is configured to be enhanced based on at least one of the secondary images.

48. The imaging system of claim 46, wherein the control circuit is configured to output a zoomed image having a zoom level greater than the first zoom level during a second mode of operation, wherein the zoomed image is based on at least one of the secondary images and the primary image.

49. The imaging system of claim 45, wherein the number of secondary image sensor arrays is at least two.

50. The imaging system of claim 45, wherein the number of secondary image sensor arrays is at least four.

51. The imaging system of claim 45, wherein the number of secondary image sensor arrays is at least six.

52. The imaging system of claim 45, wherein each of the secondary optical systems has a different zoom level from one another.

53. The imaging system of claim 45, wherein at least some of the zoom levels of the plurality of secondary optical systems are greater than the first zoom level.

54. The imaging system of claim 45, wherein at least some of the zoom levels of the plurality of secondary optical systems are less than the first zoom level.

55. The imaging system of claim 45, wherein the plurality of secondary optical systems include at least two respective secondary optical systems having a zoom level greater than the first zoom level and at least two respective secondary optical systems having a zoom level less than the first zoom level.

56. The imaging system of claim 45, wherein the imaging system is part of a camera device, further comprising control circuitry configured to output a plurality of images during a mode of operation, wherein the plurality of images includes at least one image corresponding to each of the image sensor arrays.

57. The imaging system of claim 45, wherein the imaging system is part of a camera device, further comprising control circuitry configured to output an image with super resolution generated from the first image and at least one of the secondary images.

58. The imaging system of claim 45, further comprising global electronic shutter circuitry configured to control an imaging period for the primary image sensor array and each of the secondary image sensor arrays to be substantially the same.

59. The imaging system of claim 45, further comprising global electronic shutter circuitry configured to control an integration period for the primary image sensor array and each of the secondary image sensor arrays to be substantially the same.

60. An imaging system comprising:

a semiconductor substrate;

a plurality of image sensor arrays, including a primary image sensor array and a plurality of secondary image sensor arrays, the primary image sensor array and the plurality of secondary image sensor arrays being fabricated on a single integrated circuit, the primary image sensor array employing pixels of a first linear dimension and the plurality of secondary image sensor arrays employing pixels of a second linear dimension, a ratio of the first linear dimension and the second linear dimension being greater than 1.3 to 1;

a plurality of optical systems, including at least one optical system for each image sensor array, each of the optical systems having a different zoom level, each of the image sensor arrays including pixel circuitry formed on the semiconductor substrate, the pixel circuitry being configured to read an image signal from the respective image sensor array, the pixel circuitry for each of the image sensor arrays including switching circuitry;

a control circuit operatively coupled to the switching circuitry of each of the image sensor arrays;

first pixel circuitry configured to read image data from the primary image sensor array;

second pixel circuitry configured to read image data from the plurality of secondary image sensor arrays;

an electronic global shutter configured to stop charge integration between the primary image sensor array and the first pixel circuitry and between a selected one or more of the plurality of secondary image sensor arrays and the second pixel circuitry at substantially the same time; and

a photosensitive layer formed over the semiconductor substrate with at least one of the first pixel circuitry and the second pixel circuitry being fabricated below at least a portion of the photosensitive layer.

61. The image sensor of claim 60, wherein the control circuit is configured to switch the switching circuitry at substantially the same time to provide the electronic global shutter for each of the image sensor arrays.

62. The image sensor of claim 60, wherein the control circuit is configured to switch the switching circuitry to end an integration period for each of the image sensor arrays at substantially the same time.

63. The imaging system of claim 60, wherein a number in the plurality of secondary image sensor arrays is at least four.

64. The imaging system of claim 60, wherein the optical systems for the secondary image sensor arrays include at least two respective optical systems having a zoom level greater than the zoom level of the primary image sensor array and at least two respective optical systems having a zoom level less than the primary image sensor array.

65. The imaging system of claim 60, wherein the primary image sensor array is larger than each of the secondary image sensor arrays.

66. The imaging system of claim 60, wherein the pixel circuitry for each image sensor array includes a plurality of pixel circuits formed on the semiconductor substrate corresponding to pixel regions of the respective image sensor array, each pixel circuit comprising a charge store and a switching element between the charge store and the respective pixel region.

67. The imaging system of claim 66, wherein the switching circuitry of each image sensor array is operatively coupled to each of the switching elements of the pixel circuits in the image sensor array, such that an integration period for each of the pixel circuits is configured to end at substantially the same time.

68. The imaging system of claim 66, wherein each pixel region comprises optically sensitive material over the pixel circuit for the respective pixel region.

69. The imaging system of claim 66, wherein each pixel region comprises an optically sensitive region on a first side of the semiconductor substrate, wherein the pixel circuit includes read out circuitry for the respective pixel region on the second side of the semiconductor substrate.

70. The imaging system of claim 66, wherein the charge store comprises a pinned diode.

71. The imaging system of claim 66, wherein the switching element is a transistor.

72. The imaging system of claim 66, wherein the switching element is a diode.

73. The imaging system of claim 66, wherein the switching element is a parasitic diode.

74. The imaging system of claim 66, wherein the control circuitry is configured to switch the switching element of each of the pixel circuits at substantially the same time.

75. The imaging system of claim 68, wherein each pixel region comprises a respective first electrode and a respective second electrode, wherein the optically sensitive material of the respective pixel region is positioned between the respective first electrode and the respective second electrode of the respective pixel region.

76. The imaging system of claim 66, wherein each pixel circuit is configured to transfer charge between a first electrode to the charge store when the switching element of the respective pixel region is in a first state and to block the transfer of the charge from the first electrode to the charge store when the switching element of the respective pixel region is in a second state.

77. The imaging system of claim 66, wherein the control circuitry is configured to switch the switching element of each of the pixel circuits from the first state to the second state at substantially the same time for each of the pixel circuits after an integration period of time.

78. The imaging system of claim 68, wherein each pixel circuit further comprises reset circuitry configured to reset the voltage difference across the optically sensitive material while the switching element is in the second state.

79. The imaging system of claim 66, wherein each pixel circuit further comprises a read out circuit formed on one side of the semiconductor substrate below the plurality of pixel regions.

80. The imaging system of claim 68, wherein the optically sensitive material is a continuous film of nanocrystal material.

81. The imaging system of claim 60, further comprising analog to digital conversion circuitry to generate digital pixel values from the signal read out of the pixel circuits for each of the image sensor arrays and a processor configured to process the pixel values corresponding to at least two of the image sensor arrays in a first mode of operation to generate an output image.

82. The imaging system of claim 81, wherein the output image has a zoom level between the zoom level of the primary image sensor array and at least one of the secondary image sensor arrays used to generate the output image.

83. The imaging system of claim 60, further comprising a processor configured to generate an output image during a selected mode of operation based on the pixel values corresponding to the primary image sensor array without modification based on the images projected onto the secondary image sensor arrays.

84. The imaging system of claim 60, wherein the primary image sensor array includes a number of pixels corresponding to a full resolution of the imaging system and wherein each of the secondary image sensor arrays includes a number of pixels less than the full resolution of the imaging system.

85. The imaging system of claim 84, wherein an image corresponding to the primary image sensor array is output when a first zoom level is selected and an image generated from the primary image sensor array and at least one of the secondary image sensor arrays is output when a different zoom level is selected.