System and method for high performance image processing

• US 8,830,340 B2
• Filed: 09/10/2007
• Issued: 09/09/2014
• Est. Priority Date: 09/08/2006
• Status: Active Grant

• Current Assignee: SRI International, Inc.
• Original Assignee: SRI International, Inc.
• Inventors: Burt, Peter Jeffrey, Tower, John Robertson, Van der Wal, Gooitzen Siemen, Ackerman, David Alan
• Primary Examiner(s): Haskins, Twyler
• Assistant Examiner(s): Trehan, Akshay
• Application Number:
  US11/852,632
• Publication Number:
  US 20080063294A1
• Time in Patent Office: 2,556 Days
• Field of Search: 348/143, 348/222.1, 348/571, 348/572, 348/294, 348/298, 348/302, 348/308, 348/312, 358/448, 358/530, 250/370.09, 250/208.1, 257290-292
• US Class Current: 348/222.1
• CPC Class Codes: H04N 23/676 Bracketing for image capture at varying focusing conditions
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 23/743 Bracketing, i.e. taking a series of images with varying exposure conditions
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 23/959 by adjusting depth of field during image capture, e.g. maximising or setting range based on scene characteristics
• CPC Class Codes: H04N 25/00 Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 25/57 Control of the dynamic range
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

The present invention provides a method and a system for high performance image signal processing of continuous images in real time. The system comprising a focal plane array for generating continuous source image frames in real time. The focal plane array divided logically into blocks of sub-frames. The system also comprising an analog to digital converter (ADC) layer having an array of ADC elements for converting the source image frames into a digital data. The system further comprising a digital processor layer having an array of processing elements for processing the digital data and an interconnecting layer for connecting each one of the ADC elements and the digital processing elements substantially vertically to the focal plane and substantially parallel to one another. The processing comprising reducing image motion blur, increasing image dynamic range, increasing image depth of field and obtaining features of the images.

Claims

1. An image signal processing system for an integrated sensor processor for capturing different portions of a single two-dimensional (2D) viewpoint, comprising:

a focal plane array comprising an array of focal plane elements for generating continuous source image frames in real time of the single 2D viewpoint, said focal plane array divided logically causing each focal plane element in the array of focal plane elements to produce a block of pixels;

an analog to digital converter (ADC) layer having an array of ADC elements, the array of ADC elements coupled in an element to element correspondence with the array of focal plane elements, for converting the source image frames into digital image data, wherein each of the ADC elements has an adjustable quantization step size for extending dynamic range of the sensor;

a digital processor layer having an array of programmable digital processing elements for performing one or more substantially simultaneous programmable operations on the digital image data, wherein each digital processing element is assigned to one or more of the ADC elements, for integrating the digital image data to form enhanced output frames for each operation performed, wherein a sequence of the enhanced output frames has a lesser frame rate than the rate of generating the continuous source image frames, wherein each of the digital processing elements is dedicated to processing a block of pixels; and

an interconnecting layer fabricated on the ADC layer for connecting each one of the ADC elements and the digital processing elements creating a physical element coupling substantially vertically to the focal plane and substantially parallel to one another.

2. The system of claim 1 wherein the interconnecting layer provides for communications of the digital image data between the ADC layer and the digital processor layer.

3. The system of claim 1 wherein the interconnecting layer provides for communications of the digital image data between an external device and each of the ADC layer and the digital processor layer.

4. The system of claim 1 further comprising a memory layer having an array of memory elements for storing the digital image data processed by the digital processor layer.

5. The system of claim 1 wherein the digital processor layer is mounted back to back with the focal plane array.

6. The system of claim 1 wherein the digital processor layer is mounted side to side with the focal plane array.

7. The system of claim 1 wherein the digital processor layer is displaced from the focal plane array, and the digital processor layer and the focal plane array are connected to each other by a data communications link having a sensing and processing connectivity.

8. The system of claim 1 further comprising a feedback control signal component to control functions of the focal plane array.

9. The system of claim 8 wherein the feedback control signal component controls functions of the digital processor layer.