DEVICES, METHODS, AND SYSTEMS FOR EXPANDED-FIELD-OF-VIEW IMAGE AND VIDEO CAPTURE

• US 20140118584A1
• Filed: 10/30/2013
• Published: 05/01/2014
• Est. Priority Date: 10/31/2012
• Status: Active Grant

• Current Assignee: InVisage Technologies, Inc. (Apple Inc.)
• Original Assignee: InVisage Technologies, Inc. (Apple Inc.)
• Inventors: Lee, Jess Jan Young, Hepp, Michael, Sargent, Edward Hartley
• Granted Patent: US 9,609,190 B2
• Time in Patent Office: Days
• US Class Current: 348/262
• CPC Class Codes: G03B 37/04 with cameras or projectors providing touching or overlapping fields of view
• CPC Class Codes: H01L 27/14605 Structural or functional details relating to the position of the pixel elements, e.g. smaller pixel elements in the center of the imager compared to pixel elements at the periphery
• CPC Class Codes: H01L 27/14612 involving a transistor
• CPC Class Codes: H01L 27/14623 Optical shielding
• CPC Class Codes: H01L 27/1464 Back illuminated imager structures
• CPC Class Codes: H01L 27/14643 Photodiode arrays; MOS imagers
• CPC Class Codes: H04N 2209/045 using mosaic colour filter
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 25/53 Control of the integration time
• CPC Class Codes: H04N 25/75 Circuitry for providing, modifying or processing image signals from the pixel array
• CPC Class Codes: H04N 25/76 Addressed sensors, e.g. MOS or CMOS sensors
• CPC Class Codes: H04N 25/77 Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components

Abstract

In various example embodiments, an imaging system and method are provided. In an embodiment, the system comprises a first image sensor array, a first optical system to project a first image on the first image sensor array, the first optical system having a first zoom level. A second optical system is to project a second image on a second image sensor array, the second optical system having a second zoom level. The second image sensor array and the second optical system are pointed in the same direction as the first image sensor array and the first optical system. The second zoom level is greater than the first zoom level such that the second image projected onto the second image sensor array is a zoomed in on portion of the first image projected on the first image sensor array. The first image sensor array includes at least four megapixels and the second image sensor array includes one-half or less than the number of pixels in the first image sensor array.

Claims

1. An imaging system, comprising:

a first image sensor array;

a first optical system configured to project a first image on the first image sensor array, the first optical system having a first zoom level;

a second image sensor array;

a second optical system configured to project a second image on the second image sensor array, the second optical system having a second zoom level;

wherein the second image sensor array and the second optical system are pointed in the same direction as the first image sensor array and the first optical system;

wherein the second zoom level is greater than the first zoom level such that the second image projected onto the second image sensor array is a zoomed in portion of the first image projected on the first image sensor array; and

wherein the first image sensor array includes at least four megapixels; and

wherein the second image sensor array includes one-half or less than the number of pixels in the first image sensor array.

2. The imaging system of claim 1, wherein the first image sensor array includes at least six megapixels.

3. The imaging system of claim 1, wherein the first image sensor array includes at least eight megapixels.

4. The imaging system of claim 1, wherein the second image sensor array includes four megapixels or less.

5. The imaging system of claim 1, wherein the second image sensor array includes two megapixels or less.

6. The imaging system of claim 1, wherein the second image sensor array includes one megapixel or less.

7. The imaging system of claim 1, wherein the first image sensor array includes a first array of first pixel regions and the second image sensor array includes a second array of second pixel regions, wherein each of the first pixel regions is larger than each of the second pixel regions.

8. The imaging system of claim 1, wherein each of the first pixel regions has a lateral distance across the first pixel region of less than 2.5 microns.

9. The imaging system of claim 1, wherein each of the first pixel regions has an area of less than about 2.5 microns squared.

10. The imaging system of claim 1, wherein each of the first pixel regions has a lateral distance across the first pixel region of less than 2 microns.

11. The imaging system of claim 1, wherein each of the first pixel regions has an area of less than about 2 microns squared.

12. The imaging system of claim 1, wherein each of the first pixel regions has a lateral distance across the first pixel region of less than 1.5 microns.

13. The imaging system of claim 1, wherein each of the first pixel regions has an area of less than about 1.5 microns squared.

14. The imaging system of claim 1, further comprising a third image sensor array and a third optical system configured to project a third image on the third image sensor array, the third optical system having a third zoom level;

wherein the third image sensor array and the third optical system are pointed in the same direction as the first image sensor array and the first optical system.

15. The imaging system of claim 14, wherein the third image sensor array includes the same number of pixels as the second image sensor array.

16. The imaging system of claim 1, wherein the first image sensor array and the second image sensor array are formed on the same substrate.

17. The imaging system of claim 1, further comprising a user interface control for selecting a zoom level and circuitry for reading out images from the first sensor array and the second sensor array and generating an output image based on the selected zoom level.

18. The imaging system of claim 1, wherein the first image is selected for output when the first zoom level is selected.

19. The imaging system of claim 1, wherein the second image is used to enhance the first image for output when the first zoom level is selected.

20. The imaging system of claim 1, wherein the second image is selected for output when the first zoom level is selected and the first image is used to enhance the second image.

21. The imaging system of claim 1, wherein the imaging system is part of a camera device and wherein a user control may be selected to output both the first image and the second image from the camera device.

22. The imaging system of claim 1, wherein the imaging system is part of a camera device and wherein a user control may be selected to output the first image, the second image and the third image from the camera device.

23. The imaging system of claim 1, further comprising first pixel circuitry for reading image data from the first image sensor array and second pixel circuitry for reading image data from the second image sensor array and an electronic global shutter configured to stop charge integration between the first image sensor array and the first pixel circuitry and between the second image sensor array and the second pixel circuitry at substantially the same time.

24. The imaging system of any claim 1, wherein the electronic global shutter is configured to stop the integration period for each of the pixel regions in the first pixel sensor array and the second pixel sensor array within one millisecond of one another.

25. The imaging system of claim 14, further comprising third pixel circuitry for reading image data from the third image sensor array, wherein the electronic global shutter is configured to stop charge integration between the third image sensor array and the third pixel circuitry at substantially the same time as the first sensor array and the second sensor array.

26. An imaging system, comprising:

a primary image sensor array;

a primary optical system configured to project a primary image on the primary image sensor array, the primary optical system having a first zoom level;

a plurality of secondary image sensor arrays;

a secondary optical system for each of the secondary image sensor arrays, wherein each secondary optical system is configured to project a secondary image on a respective one of the secondary image sensor arrays, each of the secondary optical systems having a respective zoom level different than the first zoom level;

wherein each of the secondary image sensor arrays and each of the secondary optical systems are pointed in the same direction as the primary image sensor array and the primary optical system; and

wherein the primary image sensor array is larger than each of the secondary image sensor arrays.

27. The imaging system of claim 26, further comprising a control circuit to output a primary image output based on the first image projected onto the primary image sensor array during a first mode of operation, wherein the primary image output is not generated based on any of the secondary images projected onto the secondary image arrays.

28. The imaging system of claim 26, further comprising a control circuit to output a primary image output based on the first image projected onto the primary image sensor array during a first mode of operation, wherein the primary image output is enhanced based on at least one of the secondary images.

29. The imaging system of claim 26, wherein the control circuit is configured to output a zoomed image having a zoom level greater than the first zoom level during a second mode of operation, wherein the zoomed image is based on at least one of the secondary images and the primary image.

30. The imaging system of claim 26, wherein each of the secondary optical systems has a different zoom level from one another.

31. The imaging system of claim 26, wherein at least some of the zoom levels of the plurality of secondary optical systems are greater than the first zoom level.

32. The imaging system of claim 26, wherein at least some of the zoom levels of the plurality of secondary optical systems are less than the first zoom level.

33. The imaging system of claim 26, wherein the plurality of secondary optical systems include at least two respective secondary optical systems having a zoom level greater than the first zoom level and at least two respective secondary optical systems having a zoom level less than the first zoom level.

34. The imaging system of claim 26, wherein the imaging system is part of a camera device, further comprising control circuitry configured to output a plurality of images during a mode of operation, wherein the plurality of images includes at least one image corresponding to each of the image sensor arrays.

35. The imaging system of claim 26, wherein the imaging system is part of a camera device, further comprising control circuitry configured to output an image with super resolution generated from the first image and at least one of the secondary images.

36. The imaging system of claim 26, further comprising global electronic shutter circuitry configured to control an imaging period for the primary image sensor array and each of the secondary image sensor arrays to be substantially the same.

37. The imaging system of claim 26, further comprising global electronic shutter circuitry configured to control an integration period for the primary image sensor array and each of the secondary image sensor arrays to be substantially the same.

38. An imaging system, comprising:

a semiconductor substrate;

a plurality of image sensor arrays, including a primary image sensor array and a plurality of secondary image sensor arrays;

a plurality of optical systems, including at least one optical system for each image sensor array;

wherein each of the optical systems has a different zoom level;

each of the image sensor arrays including pixel circuitry formed on the substrate for reading an image signal from the respective image sensor array, wherein the pixel circuitry for each of the image sensor arrays includes switching circuitry; and

a control circuit operatively coupled to the switching circuitry of each of the image sensor arrays.

39. The image sensor of claim 38, wherein the control circuit is configured to switch the switching circuitry at substantially the same time to provide a global electronic shutter for each of the image sensor arrays.

40. The image sensor of claim 38, wherein the control circuit is configured to switch the switching circuitry to end an integration period for each of the image sensor arrays at substantially the same time.

41. The imaging system of claim 38, wherein the number of secondary image sensor arrays is at least four.

42. The imaging system of claim 38, wherein the optical systems for the secondary image sensor arrays include at least two respective optical systems having a zoom level greater than the zoom level of the primary image sensor array and at least two respective optical systems having a zoom level less than the primary image sensor array.

43. The imaging system of claim 38, wherein the primary image sensor array is larger than each of the secondary image sensor arrays.

44. The imaging system of claim 38, wherein the pixel circuitry for each image sensor array includes a plurality of pixel circuits formed on the substrate corresponding to pixel regions of the respective image sensor array, each pixel circuit comprising a charge store and a switching element between the charge store and the respective pixel region.

45. The imaging system of claim 38, wherein the switching circuitry of each image sensor array is operatively coupled to each of the switching elements of the pixel circuits in the image sensor array, such that an integration period for each of the pixel circuits is configured to end at substantially the same time.

46. The imaging system of claim 38, wherein each pixel region comprises optically sensitive material over the pixel circuit for the respective pixel region.

47. The imaging system of claim 38, wherein each pixel region comprises an optically sensitive region on a first side of the semiconductor substrate, wherein the pixel circuit includes read out circuitry for the respective pixel region on the second side of the semiconductor substrate.

48. The imaging system of claim 38, wherein the charge store comprises a pinned diode.

49. The imaging system of claim 38, wherein the switching element is a transistor.

50. The imaging system of claim 38, wherein the switching element is a diode.

51. The imaging system of claim 38, wherein the switching element is a parasitic diode.

52. The imaging system of claim 38, wherein the control circuitry is configured to switch the switching element of each of the pixel circuits at substantially the same time.

53. The imaging system of claim 38, wherein each pixel region comprises a respective first electrode and a respective second electrode, wherein the optically sensitive material of the respective pixel region is positioned between the respective first electrode and the respective second electrode of the respective pixel region.

54. The imaging system of claim 38, wherein each pixel circuit is configured to transfer charge between the first electrode to the charge store when the switching element of the respective pixel region is in a first state and to block the transfer of the charge from the first electrode to the charge store when the switching element of the respective pixel region is in a second state.

55. The imaging system of claim 38, wherein the control circuitry is configured to switch the switching element of each of the pixel circuits from the first state to the second state at substantially the same time for each of the pixel circuits after an integration period of time.

56. The imaging system of claim 38, wherein each pixel circuit further comprises reset circuitry configured to reset the voltage difference across the optically sensitive material while the switching element is in the second state.

57. The imaging system of claim 38, wherein each pixel circuit further comprises a read out circuit formed on one side of the semiconductor substrate below the plurality of pixel regions.

58. The imaging system of claim 38, wherein the optically sensitive material is a continuous film of nanocrystal material.

59. The imaging system of claim 38, further comprising analog to digital conversion circuitry to generate digital pixel values from the signal read out of the pixel circuits for each of the image sensor arrays and a processor configured to process the pixel values corresponding to at least two of the image sensor arrays in a first mode of operation to generate an output image.

60. The imaging system of claim 59, wherein the output image has a zoom level between the zoom level of the primary image sensor array and at least one of the secondary image sensor arrays used to generate the output image.

61. The imaging system of claim 38, further comprising a processor configured to generate an output image during a selected mode of operation based on the pixel values corresponding to the primary image sensor array without modification based on the images projected onto the secondary image sensor arrays.

62. The imaging system of claim 38, wherein the primary image sensor array includes a number of pixels corresponding to the full resolution of the imaging system and wherein each of the secondary image sensor arrays includes a number of pixels less than the full resolution of the imaging system.

63. The imaging system of claim 62, wherein an image corresponding to the primary image sensor array is output when the first zoom level is selected and an image generated from the primary image sensor array and at least one of the secondary image sensor arrays is output when a different zoom level is selected.

64. An imaging system, comprising:

an image sensor comprising offset arrays of pixel electrodes for reading out a signal from the image sensor, wherein the arrays of pixel electrodes are offset by less than the size of a pixel region of the image sensor; and

circuitry configured to select one of the offset arrays of pixel electrodes for reading out a signal from the image sensor.

65. The imaging system of claim 64, further comprising circuitry to read out image data from each of the offset arrays of pixel electrodes and circuitry for combining the image data read out from each of the offset arrays of pixel electrodes to generate an output image.

66. An imaging system, comprising:

a first image sensor array comprising offset arrays of pixel electrodes for reading out a signal from the first image sensor array, wherein the arrays of pixel electrodes are offset by less than the size of a pixel region of the first image sensor;

a second image sensor array;

circuitry configured to select one of the offset arrays of pixel electrodes for reading out a signal from the first image sensor array; and

circuitry for reading out image data from the first image sensor array and the second image sensor array.

67. The imaging system of claim 66, further comprising circuitry for generating an output image from the image data for the first image sensor array and the second image sensor array.

68. The imaging system of claim 67, wherein the circuitry configured to select one of the offset arrays of pixel electrodes is configured to select the offset array of pixel electrodes that provides the highest super resolution when the image data from the first image sensor array is combined with the image data from the second image sensor array.

69. The imaging system of claim 67, wherein the circuitry configured to select one of the offset arrays of pixel electrodes is configured to select the offset array of pixel electrodes providing the least image overlap with the second image sensor array.

70. A method of generating an image from an image sensor system, the method comprising:

reading out a first image from a first image sensor array from a first set of locations corresponding to pixel regions of the first image sensor array; and

reading out a second image from the first image sensor array from a second set of locations corresponding to pixel regions of the first image sensor array.

71. The method of claim 70, further comprising generating an output image from the first image and the second image.

72. A method of generating an image from an image sensor system, the method comprising:

reading out a first image from a first image sensor array from a first set of locations corresponding to pixel regions of the first image sensor array;

reading out a second image from the first image sensor array from a second set of locations corresponding to pixel regions of the first image sensor array;

reading out a third image from a second image sensor array; and

using the first image, the second image and the third image to select either the first set of locations or the second set of locations for reading out a subsequent image from the first image sensor array.

73. The method of claim 72, further comprising reading a subsequent image from the second image sensor array at substantially the same time as the subsequent image from the first image sensor array.

74. The method of claim 73, further comprising generating a super resolution image from the subsequent image read out from the second image sensor array and the subsequent image read out from the first image sensor array.

75. The method of claim 73, wherein the second image sensor array is pointed in the same direction as the first image sensor array and has a zoom level different than the first image sensor array.