System and Method for High Performance Image Processing

• US 20080063294A1
• Filed: 09/10/2007
• Published: 03/13/2008
• Est. Priority Date: 09/08/2006
• Status: Active Grant

• Current Assignee: SRI International, Inc.
• Original Assignee: SRI International, Inc.
• Inventors: Van der Wal, Gooitzen Siemen, Tower, John Robertson, Ackerman, David Alan, Burt, Peter Jeffrey
• Granted Patent: US 8,830,340 B2
• Time in Patent Office: Days
• US Class Current: 382/255
• CPC Class Codes: H04N 23/676 Bracketing for image capture at varying focusing conditions
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 23/743 Bracketing, i.e. taking a series of images with varying exposure conditions
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 23/959 by adjusting depth of field during image capture, e.g. maximising or setting range based on scene characteristics
• CPC Class Codes: H04N 25/00 Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 25/57 Control of the dynamic range
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

The present invention provides a method and a system for high performance image signal processing of continuous images in real time. The system comprising a focal plane array for generating continuous source image frames in real time. The focal plane array divided logically into blocks of sub-frames. The system also comprising an analog to digital converter (ADC) layer having an array of ADC elements for converting the source image frames into a digital data. The system further comprising a digital processor layer having an array of processing elements for processing the digital data and an interconnecting layer for connecting each one of the ADC elements and the digital processing elements substantially vertically to the focal plane and substantially parallel to one another. The processing comprising reducing image motion blur, increasing image dynamic range, increasing image depth of field and obtaining features of the images.

Claims

1. An image signal processing system for an integrated sensor processor, comprising:

a focal plane array for generating continuous source image frames in real time, said focal plane array divided logically into blocks of pixels;

an analog to digital converter (ADC) layer having an array of ADC elements for converting the source image frames into a digital data;

a digital processor layer having an array of processing elements for processing the digital data; and

an interconnecting layer for connecting each one of said ADC elements, said digital processing elements substantially vertically to the focal plane and substantially parallel to one another.

3. The system of claim 1 wherein said interconnecting layer provides for communications of the digital data between an external device and each of said ADC layer and said digital processor layer.

4. The system of claim 1 further comprising an memory layer having an array of memory elements for storing the processed digital data.

5. The system of claim 1 wherein said processor array is mounted back to back with said focal plane array.

6. The system of claim 1 wherein said processor array is mounted side to side with said focal plane array.

7. The system of claim 1 wherein said processor array is displaced from the focal array, said processor array and said focal plane array are connected to each other by a data communications link having a sensing and processing connectivity.

8. The system of claim 1 further comprising a feedback control signal component to control the functions of the focal plane array.

9. The system of claim 8 wherein said feedback control signal component controls the functions of the processor array.

2. The system of claim 2 wherein said interconnecting layer provides for communications of the digital data between the ADC layer and each of the said digital processor layer.

10. An image processing method for reducing image motion blur in a scene, the method comprising the steps of:

capturing continuous source image frames of the image scene from a sensor pixel array in real time at a relatively high frame rate;

dividing said source image frames into blocks of sub-frames;

digitizing each of said sub-frames;

estimating the offset between each successive frame of said sub-frames through digital image analysis;

aligning sets of the successive frames to one another based on the offset estimates; and

combining each of the aligned sets to form an enhanced image of the scene at a relatively lower frame rate.

11. The method of claim 10 wherein said combining comprising summing each of the aligned sets of the frames to form said enhanced image with an increased signal/noise ratio.

12. The method of claim 10 wherein said combining comprising resampling to a higher sample rate image to form said enhanced image with an enhanced resolution.

13. An image processing method for increasing dynamic range of a scene, the method comprising the steps of:

capturing continuous source image frames of the scene from a sensor pixel array in real time at a relatively high frame rate, wherein each of said captured source frames having different exposures of the scene;

dividing said source image frames into blocks of sub-frames;

digitizing each of said sub-frames;

combining each of said digitized sub-frames to form an extended dynamic range image of said scene.

14. An image processing method for increasing dynamic range of a scene, the method comprising the steps of:

capturing continuous source image frames of the scene from a sensor pixel array in real time at a relatively high frame rate;

dividing said source image frames into blocks of sub-frames;

digitizing each of said sub-frames at various step sizes;

selecting regions of each of the digitized sub-frames; and

combining the selected regions to form an extended dynamic range image of said scene.

17. The method of claim 14 wherein said features comprise global and local statistical measures, pattern selective filters, image transforms, pattern detection, 3D range estimate or combinations thereof.

15. An image processing method for increasing depth of field of a scene, the method comprising the steps of:

capturing continuous source image frames of the scene from a sensor pixel array in real time at a relatively high frame rate, wherein each of said captured source frames have different focal settings of the scene;

dividing said source image frames into blocks of sub-frames;

digitizing each of said sub-frames; and

combining each of said digitized sub-frames to form an extended depth of field image of said scene.

16. An image processing method for obtaining features of an image scene, the method comprising the steps of:

capturing continuous source image frames of the scene from a sensor pixel array in real time at a relatively high frame rate;

dividing said source image frames into blocks of sub-frames;

digitizing each of said sub-frames;

generating a set of the features from said digitized sub-frames.

18. The method of claim 16 wherein said generating comprising convolving a filter with a data within the block of sub-frames.

19. The method of claim 16 wherein said generating comprising computing statistic measure within a neighborhood of each pixel in the block.

20. An image processing method for reducing image motion blur in a scene, the method comprising the steps of:

capturing a block of image frames of the image scene at a relatively high frame rate;

digitizing each of the image frames;

estimating the offset between each successive frame of said image frames through digital image analysis;

aligning sets of the successive frames to one another based on the offset estimates; and

combining each of the aligned sets to form an enhanced image of the scene at a relatively lower frame rate.

21. An image processing method for increasing dynamic range of a scene, the method comprising the steps of:

capturing a block of image frames of the image scene at a relatively high frame rate;

wherein each of said captured image frames having different exposures of the scene;

digitizing each of the image frames;

combining each of said digitized image frames to form an extended dynamic range image of said scene.

22. An image processing method for increasing dynamic range of a scene, the method comprising the steps of:

capturing a block of image frames of the image scene at a relatively high frame rate;

digitizing each of the image frames at various step sizes;

selecting regions of each of the digitized frames; and

combining the selected regions to form an extended dynamic range image of said scene.

23. An image processing method for increasing depth of field of a scene, the method comprising the steps of:

capturing a block of image frames of the image scene at a relatively high frame rate;

wherein each of said captured image frames have different focal settings of the scene;

digitizing each of the image frames; and

combining each of said digitized frames to form an extended depth of field image of said scene.

24. An image processing method for obtaining features of an image scene, the method comprising the steps of:

capturing a block of image frames of the image scene at a relatively high frame rate;

digitizing each of said frames;

generating a set of the features from said digitized frames.