DEVICES AND METHODS FOR HIGH-RESOLUTION IMAGE AND VIDEO CAPTURE

• US 20130250150A1
• Filed: 05/14/2013
• Published: 09/26/2013
• Est. Priority Date: 05/03/2010
• Status: Abandoned Application

• Current Assignee: InVisage Technologies, Inc. (Apple Inc.)
• Original Assignee: InVisage Technologies, Inc. (Apple Inc.)
• Inventors: Malone, Michael R., Della Nave, Pierre Henri Rene, Brading, Michael Charles, Lee, Jess Jan Young, Tian, Hui, Ivanov, Igor Constantin, Sargent, Edward Hartley
• Application Number:
  US13/894,184
• Publication Number:
  US 20130250150A1
• Time in Patent Office: Days
• US Class Current: 348/262
• CPC Class Codes: G02B 15/00 Optical objectives with means for varying the magnification anamorphotic objectives G02B13/08
• CPC Class Codes: H01L 27/14609 Pixel-elements with integrated switching, control, storage or amplification elements scanning details of imagers (circuitry of solid-state image sensors H04N25/00); circuitry of imagers H04N25/70
• CPC Class Codes: H01L 27/14625 Optical elements or arrangements associated with the device
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 25/771 comprising storage means other than floating diffusion

Abstract

In various example embodiments, an imaging system and method are provided. In an embodiment, the system comprises a first image sensor array, a first optical system to project a first image on the first image sensor array, the first optical system having a first zoom level. A second optical system is to project a second image on a second image sensor array, the second optical system having a second zoom level. The second image sensor array and the second optical system are pointed in the same direction as the first image sensor array and the first optical system. The second zoom level is greater than the first zoom level such that the second image projected onto the second image sensor array is a zoomed in on portion of the first image projected on the first image sensor array. The first image sensor array may include at least four megapixels and the second image sensor array may include one-half or less than the number of pixels in the first image sensor array.

Claims

1. An imaging system comprising:

a first image sensor array;

a first optical system configured to project a first image on the first image sensor array, the first optical system having a first zoom level;

a second image sensor array;

a second optical system configured to project a second image on the second image sensor array, the second optical system having a second zoom level;

wherein the second image sensor array and the second optical system are pointed in the same direction as the first image sensor array and the first optical system;

wherein the second zoom level is greater than the first zoom level such that the second image projected onto the second image sensor array is a zoomed-in portion of the first image projected on the first image sensor array; and

wherein the first image sensor array includes at least four megapixels; and

wherein the second image sensor array includes one-half or less than the number of pixels in the first image sensor array.

2. The imaging system of claim 1, wherein the first image sensor array includes at least six megapixels.

3. The imaging system of claim 1, wherein the first image sensor array includes at least eight megapixels.

4. The imaging system of claim 1, wherein the second image sensor array includes four megapixels or less.

5. The imaging system of claim 1, wherein the second image sensor array includes two megapixels or less.

6. The imaging system of claim 1, wherein the second image sensor array includes one megapixel or less.

7. The imaging system of claim 1, wherein the first image sensor array includes a first array of first pixel regions and the second image sensor array includes a second array of second pixel regions, wherein each of the first pixel regions is larger than each of the second pixel regions.

8. The imaging system of claim 7, wherein each of the first pixel regions has a lateral distance across the first pixel region of less than 2.5 microns.

9. The imaging system of claim 7, wherein each of the first pixel regions has an area of less than about 2.5 microns squared.

10. The imaging system of claim 7, wherein each of the first pixel regions has a lateral distance across the first pixel region of less than 2 microns.

11. The imaging system of claim 7, wherein each of the first pixel regions has an area of less than about 2 microns squared.

12. The imaging system of claim 7, wherein each of the first pixel regions has a lateral distance across the first pixel region of less than 1.5 microns.

13. The imaging system of any of the preceding claims, wherein each of the first pixel regions has an area of less than about 1.5 microns squared.

14. The imaging system of claim 7, wherein each of the second pixel regions has a lateral distance across the second pixel region of less than 2.1 microns.

15. The imaging system of claim 7, wherein each of the second pixel regions has an area of less than about 2.1 microns squared.

16. The imaging system of claim 7, wherein each of the second pixel regions has a lateral distance across the second pixel region of less than 1.6 microns.

17. The imaging system of claim 7, wherein each of the second pixel regions has an area of less than about 1.6 microns squared.

18. The imaging system of claim 7, wherein each of the second pixel regions has a lateral distance across the second pixel region of less than 1.3 microns.

19. The imaging system of claim 7, wherein each of the second pixel regions has an area of less than about 1.3 microns squared.

20. The imaging system of claim 1, further comprising a third image sensor array and a third optical system configured to project a third image on the third image sensor array, the third optical system having a third zoom level;

wherein the third image sensor array and the third optical system are pointed in the same direction as the first image sensor array and the first optical system.

21. The imaging system of claim 20, wherein the third zoom level is greater than the second zoom level.

22. The imaging system of claim 20, wherein the third zoom level is less than the first zoom level.

23. The imaging system of claim 20, wherein the third image sensor array includes the same number of pixels as the second image sensor array.

24. The imaging system of claim 20, wherein the third image sensor array includes four megapixels or less.

25. The imaging system of claim 20, wherein the third image sensor array includes two megapixels or less.

26. The imaging system of claim 20, wherein the third image sensor array includes one megapixel or less.

27. The imaging system of claim 20, wherein the third image sensor array includes a third array of third pixel regions, wherein each of the third pixel regions is smaller than each of the first pixel regions.

28. The imaging system of claim 21, wherein each of the third pixel regions has a lateral distance across the pixel region of less than 1.9 microns.

29. The imaging system of claim 21, wherein each of the third pixel regions has an area of less than about 1.9 microns squared.

30. The imaging system of claim 21, wherein each of the third pixel regions has a lateral distance across the third pixel region of less than 1.4 microns.

31. The imaging system of claim 21, wherein each of the third pixel regions has an area of less than about 1.4 microns squared.

32. The imaging system of claim 21, wherein each of the third pixel regions has a lateral distance across the third pixel region of less than 1.2 microns.

33. The imaging system of claim 21, wherein each of the third pixel regions has an area of less than about 1.2 microns squared.

34. The imaging system of claim 1, wherein the first image sensor array and the second image sensor array are formed on the same substrate.

35. The imaging system of claim 20, wherein the third image sensor array is formed on the same substrate as the first image sensor array and the second image sensor array.

36. The imaging system of claim 1, further comprising a user interface control to select a zoom level and circuitry to read out images from the first sensor array and the second sensor array and generate an output image based on the selected zoom level.

37. The imaging system of claim 1, wherein the first image is to be selected for output when the first zoom level is selected.

38. The imaging system of claim 1, wherein the second image is to be used to enhance the first image for output when the first zoom level is selected.

39. The imaging system of claim 1, wherein the second image is to be selected for output when the first zoom level is selected and the first image is to be used to enhance the second image.

40. The imaging system of claim 1, wherein the imaging system is part of a camera device and wherein a user control may be selected to output both the first image and the second image from the camera device.

41. The imaging system of claim 20, wherein the imaging system is part of a camera device and wherein a user control may be selected to output the first image, the second image, and the third image from the camera device.

42. The imaging system of claim 1, further comprising:

first pixel circuitry to read image data from the first image sensor array;

second pixel circuitry to read image data from the second image sensor array; and

an electronic global shutter configured to stop charge integration between the first image sensor array and the first pixel circuitry and between the second image sensor array and the second pixel circuitry at substantially the same time.

43. The imaging system of claim 42, wherein the electronic global shutter is configured to stop the integration period for each of the pixel regions in the first pixel sensor array and the second pixel sensor array within one millisecond of one another.

44. The imaging system of claim 20, further comprising:

third pixel circuitry to read image data from the third image sensor array; and

an electronic global shutter configured to stop charge integration between the first image sensor array and the first pixel circuitry and between the second image sensor array and the second pixel circuitry, the electronic global shutter further configured to stop charge integration at the third image sensor array and the third pixel circuitry at substantially the same time as the first sensor array and the second sensor array.

45. The imaging system of claim 44, wherein the electronic global shutter is configured to stop the integration period for each of the third pixel regions in the third pixel sensor array within one millisecond of each of the pixel regions in the first image sensor array and the second image sensor array.

46. An imaging system comprising:

a primary image sensor array;

a primary optical system configured to project a primary image on the primary image sensor array, the primary optical system having a first zoom level;

a plurality of secondary image sensor arrays;

a secondary optical system for each of the secondary image sensor arrays, wherein each secondary optical system is configured to project a secondary image on a respective one of the secondary image sensor arrays, each of the secondary optical systems having a respective zoom level different than the first zoom level;

wherein each of the secondary image sensor arrays and each of the secondary optical systems are pointed in the same direction as the primary image sensor array and the primary optical system; and

wherein the primary image sensor array is larger than each of the secondary image sensor arrays.

47. The imaging system of claim 46, further comprising a control circuit to output a primary image output based on the first image projected onto the primary image sensor array during a first mode of operation, wherein the primary image output is not generated based on any of the secondary images projected onto the secondary image arrays.

48. The imaging system of claim 46, further comprising a control circuit to output a primary image output based on the first image projected onto the primary image sensor array during a first mode of operation, wherein the primary image output is enhanced based on at least one of the secondary images.

49. The imaging system of claim 48, wherein the control circuit is configured to output a zoomed image having a zoom level greater than the first zoom level during a second mode of operation, wherein the zoomed image is based on at least one of the secondary images and the primary image.

50. The imaging system of claim 46, wherein the number of secondary image sensor arrays is at least two.

51. The imaging system of claim 46, wherein the number of secondary image sensor arrays is at least four.

52. The imaging system of claim 46, wherein the number of secondary image sensor arrays is at least six.

53. The imaging system of claim 46, wherein each of the secondary optical systems has a different zoom level from one another.

54. The imaging system of claim 46, wherein at least some of the zoom levels of the plurality of secondary optical systems are greater than the first zoom level.

55. The imaging system of claim 46, wherein at least some of the zoom levels of the plurality of secondary optical systems are less than the first zoom level.

56. The imaging system of claim 46, wherein the plurality of secondary optical systems include at least two respective secondary optical systems having a zoom level greater than the first zoom level and at least two respective secondary optical systems having a zoom level less than the first zoom level.

57. The imaging system of claim 46, wherein the imaging system is part of a camera device, the imaging system further comprising control circuitry configured to output a plurality of images during a mode of operation, wherein the plurality of images is to include at least one image corresponding to each of the image sensor arrays.

58. The imaging system of claim 46, wherein the imaging system is part of a camera device, the imaging system further comprising control circuitry configured to output an image with super resolution generated from the first image and at least one of the secondary images.

59. The imaging system of claim 46, further comprising global electronic shutter circuitry configured to control an imaging period for the primary image sensor array and each of the secondary image sensor arrays to be substantially the same.

60. The imaging system of claim 46, further comprising global electronic shutter circuitry configured to control an integration period for the primary image sensor array and each of the secondary image sensor arrays to be substantially the same.

61. An imaging system comprising:

a semiconductor substrate;

a plurality of image sensor arrays, including a primary image sensor array and a plurality of secondary image sensor arrays;

a plurality of optical systems, including at least one optical system for each image sensor array;

wherein each of the optical systems has a different zoom level;

each of the image sensor arrays including pixel circuitry formed on the substrate for reading an image signal from the respective image sensor array, wherein the pixel circuitry for each of the image sensor arrays includes switching circuitry; and

a control circuit operatively coupled to the switching circuitry of each of the image sensor arrays.

62. The image sensor of claim 61, wherein the control circuit is configured to switch the switching circuitry at substantially the same time to provide a global electronic shutter for each of the image sensor arrays.

63. The image sensor of claim 61, wherein the control circuit is configured to switch the switching circuitry to end an integration period for each of the image sensor arrays at substantially the same time.

64. The imaging system of claim 61, wherein the number of secondary image sensor arrays is at least four.

65. The imaging system of claim 61, wherein the optical systems for the secondary image sensor arrays include at least two respective optical systems having a zoom level greater than the zoom level of the primary image sensor array and at least two respective optical systems having a zoom level less than the primary image sensor array.

66. The imaging system of claim 61, wherein the primary image sensor array is larger than each of the secondary image sensor arrays.

67. The imaging system of claim 61, wherein the pixel circuitry for each image sensor array includes a plurality of pixel circuits formed on the substrate corresponding to pixel regions of the respective image sensor array, each pixel circuit comprising a charge store and a switching element between the charge store and the respective pixel region.

68. The imaging system of claim 61, wherein the switching circuitry of each image sensor array is operatively coupled to each of the switching elements of the pixel circuits in the image sensor array, such that an integration period for each of the pixel circuits is configured to end at substantially the same time.

69. The imaging system of claim 67, wherein each pixel region comprises optically sensitive material over the pixel circuit for the respective pixel region.

70. The imaging system of claim 67, wherein each pixel region comprises an optically sensitive region on a first side of the semiconductor substrate, wherein the pixel circuit includes read out circuitry for the respective pixel region on the second side of the semiconductor substrate.

71. The imaging system of claim 67, wherein the charge store comprises a pinned diode.

72. The imaging system of claim 67, wherein the switching element is a transistor.

73. The imaging system of claim 67, wherein the switching element is a diode.

74. The imaging system of claim 67, wherein the switching element is a parasitic diode.

75. The imaging system of claim 67, wherein the control circuitry is configured to switch the switching element of each of the pixel circuits at substantially the same time.

76. The imaging system of claim 69, wherein each pixel region comprises a respective first electrode and a respective second electrode, wherein the optically sensitive material of the respective pixel region is positioned between the respective first electrode and the respective second electrode of the respective pixel region.

77. The imaging system of claim 76, wherein each pixel circuit is configured to transfer charge between the first electrode to the charge store when the switching element of the respective pixel region is in a first state and to block the transfer of the charge from the first electrode to the charge store when the switching element of the respective pixel region is in a second state.

78. The imaging system of claim 77, wherein the control circuitry is configured to switch the switching element of each of the pixel circuits from the first state to the second state at substantially the same time for each of the pixel circuits after an integration period of time.

79. The imaging system of claim 69, wherein each pixel circuit further comprises reset circuitry configured to reset the voltage difference across the optically sensitive material while the switching element is in the second state.

80. The imaging system of claim 67, wherein each pixel circuit further comprises a read out circuit formed on one side of the semiconductor substrate below the plurality of pixel regions.

81. The imaging system of claim 69, wherein the optically sensitive material is a continuous film of nanocrystal material.

82. The imaging system of claim 61, further comprising:

analog to digital conversion circuitry to generate digital pixel values from a signal read out of the pixel circuits for each of the image sensor arrays; and

a processor configured to process the pixel values corresponding to at least two of the image sensor arrays in a first mode of operation to generate an output image.

83. The imaging system of claim 82, wherein the output image has a zoom level between the zoom level of the primary image sensor array and at least one of the secondary image sensor arrays used to generate the output image.

84. The imaging system of claim 82, further comprising a processor configured to generate an output image during a selected mode of operation based on the pixel values corresponding to the primary image sensor array without modification based on images projected onto the secondary image sensor arrays.

85. The imaging system of claim 61, wherein the primary image sensor array includes a number of pixels corresponding to the full resolution of the imaging system and wherein each of the secondary image sensor arrays includes a number of pixels less than the full resolution of the imaging system.

86. The imaging system of claim 85, wherein an image corresponding to the primary image sensor array is output when the first zoom level is selected and an image generated from the primary image sensor array and at least one of the secondary image sensor arrays is output when a different zoom level is selected.

87. An imaging system comprising:

an image sensor comprising offset arrays of pixel electrodes to read out a signal from the image sensor, wherein the arrays of pixel electrodes are offset by less than the size of a pixel region of the image sensor; and

circuitry configured to select one of the offset arrays of pixel electrodes to read out a signal from the image sensor.

88. The imaging system of claim 87, further comprising circuitry to read out image data from each of the offset arrays of pixel electrodes and circuitry for combining the image data read out from each of the offset arrays of pixel electrodes to generate an output image.

89. An imaging system comprising:

a first image sensor array comprising offset arrays of pixel electrodes to read out a signal from the first image sensor array, wherein the arrays of pixel electrodes are offset by less than the size of a pixel region of the first image sensor;

a second image sensor array;

circuitry configured to select one of the offset arrays of pixel electrodes to read out a signal from the first image sensor array; and

circuitry to read out image data from the first image sensor array and the second image sensor array.

90. The imaging system of claim 89, further comprising circuitry to generate an output image from the image data for the first image sensor array and the second image sensor array.

91. The imaging system of claim 89, wherein the circuitry configured to select one of the offset arrays of pixel electrodes is configured to select the offset array of pixel electrodes that provides the highest super resolution when the image data from the first image sensor array is combined with the image data from the second image sensor array.

92. The imaging system of claim 89, wherein the circuitry configured to select one of the offset arrays of pixel electrodes is configured to select the offset array of pixel electrodes providing the least image overlap with the second image sensor array.

93. A method of generating an image from an image sensor system, the method comprising:

reading out a first image from a first image sensor array from a first set of locations corresponding to pixel regions of the first image sensor array; and

reading out a second image from the first image sensor array from a second set of locations corresponding to pixel regions of the first image sensor array.

94. The method of claim 93, further comprising generating an output image from the first image and the second image.

95. A method of generating an image from an image sensor system, the method comprising:

reading out a first image from a first image sensor array from a first set of locations corresponding to pixel regions of the first image sensor array;

reading out a second image from the first image sensor array from a second set of locations corresponding to pixel regions of the first image sensor array;

reading out a third image from a second image sensor array; and

using the first image, the second image, and the third image to select either the first set of locations or the second set of locations for reading out a subsequent image from the first image sensor array.

96. The method of claim 95, further comprising reading a subsequent image from the second image sensor array at substantially the same time as the subsequent image from the first image sensor array.

97. The method of claim 96, further comprising generating a super resolution image from the subsequent image read out from the second image sensor array and the subsequent image read out from the first image sensor array.

98. The method of claim 95, wherein the second image sensor array is pointed in the same direction as the first image sensor array and has a zoom level different than the first image sensor array.