Multiple aperture video image enhancement system

• US 9,380,273 B1
• Filed: 10/02/2009
• Issued: 06/28/2016
• Est. Priority Date: 10/02/2009
• Status: Active Grant

• Current Assignee: Rockwell Collins, Inc. (Rtx Corporation)
• Original Assignee: Rockwell Collins, Inc. (Rtx Corporation)
• Inventors: Jensen, David W., Koenck, Steven E.
• Primary Examiner(s): Winder, Patrice
• Assistant Examiner(s): De Rodrig, Angela Widhalm
• Application Number:
  US12/572,492
• Time in Patent Office: 2,461 Days
• Field of Search: 348/218.1, 345 7- 9
• US Class Current: 1/1
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Disclosed is a helmet mounted enhanced vision system which includes at least one camera, having a circuit board therein with an array thereon of closely spaced apertures, where each aperture includes an electronic light sensor and a computer die coupled to and associated therewith, such that each aperture behaves much like a digital camera coupled via to the other apertures by a network. The array of apertures each including varied sensor types such sensors having: different resolutions, distance to optimal focus point, and light sensitivity, so as to enable foveal displays, multi-focal point images and dynamic range enhancement, respectively.

Claims

1. An electronic vision system comprising in operative combination:

a support, configured to be worn;

a plurality of apertures, disposed on said support, where each aperture of said plurality of apertures comprises;

an electronic light sensor die configured to generate electronic image data and an internal to an aperture computing die configured to receive the electronic image data from said electronic light sensor die; and

a mesh network segment configured for at least indirectly interconnecting said aperture to a nearest neighboring aperture;

a computer coupled to said plurality of apertures and configured to receive a first electronic image data including a first region of a scene having overexposed pixel samples from a first aperture of the plurality of apertures and a second electronic image data including the first region of the scene having no overexposed pixel samples from a second aperture of the plurality of apertures and to combine the first and second electronic image data and increase a bit-depth of the pixel samples to render display data including an enhanced image of the first region of the scene having no overexposed pixel samples to be displayed;

a display device, disposed on said support, and coupled to receive display data rendered by said computer;

said display device being configured to display visual images of said display data; and

said computer is one of said computing die of one of said apertures configured to display visual images of said display data to only one person at any time, where said only one person is a person wearing said support.

2. The vision system of claim 1 wherein at least one of said apertures has an optical characteristic which is not the same as another of said plurality of apertures, where the optical characteristic is a non image resolution characteristic.

3. The vision system of claim 2 wherein the support is head-mounted.

4. The vision system of claim 1 further comprising a circuit board upon which is disposed, in an array, said plurality of apertures and wherein said electronic light sensor die and said internal to an aperture computing die are arranged in a stacked configuration.

5. The vision system of claim 1 wherein said computer further comprises a port to an external network and

further comprising:

an analog to digital convertor disposed between said internal to an aperture computing die and said electronic light sensor die.

6. The vision system of claim 1 wherein said computer is configured with software to render image data to display a foveated image where a central region of said foveated image has a higher resolution than a peripheral region of said foveated image;

with respect to at least one of said plurality of apertures, a fovea aperture disposed centrally in said a plurality of apertures;

which outputs an image signal having a higher resolution characteristic than a peripheral aperture, which is disposed in a non-central location in said plurality of apertures.

7. The vision system of claim 1 wherein said first aperture and said second aperture are separate and distinct apertures with at least one different optical characteristic.

8. The vision system of claim 7 further comprising a circuit board upon which is disposed, in an array, said plurality of apertures:

wherein said computer is further configured with software to combine regions of a scene that are in focus in different aperture generated images, into a single enhanced image.

9. The vision system of claim 8 wherein said computer is configured with software to render image data to display a foveated image where a central region of said foveated image has a higher resolution than a peripheral region of said foveated image;

with respect to at least one of said plurality of apertures, a fovea aperture disposed centrally in said a plurality of apertures;

which outputs an image signal having a higher resolution characteristic than a peripheral aperture, which is disposed in a non-central location in said plurality of apertures.