SCENE BASED NON-UNIFORMITY CORRECTION SYSTEMS AND METHODS

• US 20090257679A1
• Filed: 04/15/2008
• Published: 10/15/2009
• Est. Priority Date: 04/15/2008
• Status: Active Grant

• Current Assignee: Teledyne FLIR LLC (Teledyne Technologies Incorporated)
• Original Assignee: FLIR Systems, Inc. (Teledyne Technologies Incorporated)
• Inventors: Hogasten, Nicholas
• Granted Patent: US 7,995,859 B2
• Time in Patent Office: Days
• US Class Current: 382/275
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: H04N 25/674 based on the scene itself, e.g. defocusing
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Systems and methods provide scene-based non-uniformity correction for infrared images, in accordance with one or more embodiments. For example in one embodiment, a method of processing infrared images includes storing a template frame comprising a first plurality of pixel data of an infrared image; receiving an input frame comprising a second plurality of pixel data of an infrared image; determining frame-to-frame motion between at least some of the first and second plurality of pixel data; warping the template frame based on the determining of the frame-to-frame motion; comparing the first plurality of pixel data to the second plurality of pixel data to determine irradiance differences based on the determining; propagating pixel offset information for scene based non uniformity correction terms, based on the determining of the frame-to-frame motion, for at least some of the scene based non uniformity correction terms to other ones of the scene based non uniformity correction terms; updating the scene based non uniformity correction terms based on the comparing and the propagating; applying the scene based non uniformity correction terms to the second plurality of pixel data to reduce fixed pattern noise; and providing an output infrared image after the applying.

Claims

1. A method of processing infrared images, the method comprising:

storing a template frame comprising a first plurality of pixel data of an infrared image;

receiving an input frame comprising a second plurality of pixel data of an infrared image;

determining frame-to-frame motion between at least some of the first and second plurality of pixel data;

warping the template frame based on the determining of the frame-to-frame motion;

comparing the first plurality of pixel data to the second plurality of pixel data to determine irradiance differences based on the determining;

propagating pixel offset information for scene based non uniformity correction terms, based on the determining of the frame-to-frame motion, for at least some of the scene based non uniformity correction terms to other ones of the scene based non uniformity correction terms;

updating the scene based non uniformity correction terms based on the comparing and the propagating;

applying the scene based non uniformity correction terms to the second plurality of pixel data to reduce fixed pattern noise; and

providing an output infrared image after the applying.

2. The method of claim 1, wherein at least a portion of the pixel offset information corresponds to a golden pixel having an approximately constant offset term.

3. The method of claim 2, wherein the pixel offset information corresponding to the golden pixel is propagated to learning pixels and/or enlightened pixels corresponding to the other ones of the scene based non uniformity correction terms.

4. The method of claim 3, wherein the comparing generates a difference frame, and wherein the method further comprises:

determining a certainty estimate for pixel data in the difference frame;

performing a masking process on the pixel data in the difference frame based on the determining of the certainty estimate and on the determining of the frame-to-frame motion; and

wherein at least some of the learning pixels become the enlightened pixels, which propagate their pixel offset information and/or accumulated certainty estimate to other pixels corresponding to additional ones of the scene based non uniformity correction terms, upon accumulation of a sufficient level of the certainty estimate for the learning pixels.

5. The method of claim 4, wherein updated values for the scene based non uniformity correction terms are weighted mean values, with weighting determined by the accumulated certainty estimates corresponding to the golden or enlightened pixels from which data are being propagated, certainties of the frame-to-frame motion, and the accumulated certainty estimate of the enlightened or learning pixels to which the data are being propagated.

6. The method of claim 1, wherein the comparing generates a difference frame, and wherein the method further comprises:

determining a certainty estimate for pixel data in the difference frame; and

performing a masking process on the pixel data in the difference frame based on the determining of the certainty estimate and on the determining of the frame-to-frame motion.

7. The method of claim 6, wherein the masking process further comprises a threshold decay process for a difference threshold for the irradiance differences from the comparing.

8. The method of claim 1, wherein the applying of the scene based non uniformity correction terms is performed only if the scene based non uniformity correction terms are determined to be stable based on the determining of the certainty estimate, and wherein the determining of the frame-to-frame motion comprises at least one of a motion prediction process and an image registration process.

9. The method of claim 1, further comprising subtracting a mean value of one or more of the input frames from the input frame during low signal-to-noise ratio conditions.

10. A system comprising:

an infrared image sensor adapted to provide pixel data of an infrared image;

a memory adapted to store the pixel data; and

a controller adapted to perform infrared image processing on the pixel data, the processing comprising;

generating a template frame comprising a first plurality of pixel data;

receiving an input frame comprising a second plurality of pixel data from the infrared image sensor;

determining frame-to-frame motion between at least some of the first and second plurality of pixel data;

comparing the first plurality of pixel data to the second plurality of pixel data to determine irradiance differences;

propagating pixel offset information for scene based non uniformity correction terms, based on the determining of the frame-to-frame motion, for at least some of the scene based non uniformity correction terms to other ones of the scene based non uniformity correction terms;

updating the scene based non uniformity correction terms based on the comparing and the propagating; and

applying the scene based non uniformity correction terms to the second plurality of pixel data to reduce fixed pattern noise.

11. The system of claim 10, wherein the comparing generates a difference frame, and wherein the processing further comprises:

determining a certainty estimate for pixel data in the difference frame; and

performing a masking process on the pixel data in the difference frame based on the determining of the certainty estimate.

12. The system of claim 11, wherein at least a portion of the pixel offset information corresponds to a golden pixel having an approximately constant offset term.

13. The system of claim 12, wherein the pixel offset information corresponding to the golden pixel is propagated to learning pixels and/or enlightened pixels corresponding to the other ones of the scene based non uniformity correction terms.

14. The system of claim 13, wherein at least some of the learning pixels become the enlightened pixels, which propagate their pixel offset information and/or accumulated certainty estimate to other pixels corresponding to additional ones of the scene based non uniformity correction terms, upon accumulation of a sufficient level of the certainty estimate for the learning pixels.

15. The system of claim 14, wherein updated values for the scene based non uniformity correction terms are weighted mean values, with weighting determined by the accumulated certainty estimates corresponding to the golden or enlightened pixels from which data are being propagated, certainties of the frame-to-frame motion, and the accumulated certainty estimate of the enlightened or learning pixels to which the data are being propagated.

16. The system of claim 14, wherein the system comprises an infrared camera system having a display, with the controller coupled to the memory and the infrared image sensor to perform the processing based on information stored in the memory, and wherein the processing further comprises:

storing in the memory an output infrared image after the applying;

providing to the display the output infrared image after the applying;

warping the template frame based on the determining of the frame-to-frame motion; and

wherein the applying of the scene based non uniformity correction terms is performed only if the scene based non uniformity correction terms are determined to be stable based on the determining of the certainty estimate.

17. A computer-readable medium on which is stored information for performing an infrared image processing method, the method comprising:

generating a template frame comprising a first plurality of pixel data generated by an infrared image sensor;

receiving a second plurality of pixel data generated by the infrared image sensor;

comparing the first plurality of pixel data to the second plurality of pixel data, based on frame-to-frame motion between at least some of the first and second plurality of pixel data, to determine irradiance differences;

propagating pixel offset information for scene based non uniformity correction terms, based on the frame-to-frame motion, for at least some of the scene based non uniformity correction terms to other ones of the scene based non uniformity correction terms;

updating scene based non uniformity correction terms based on the comparing and the propagating;

applying the scene based non uniformity correction terms to the second plurality of pixel data to reduce fixed pattern noise and provide an output infrared image; and

storing the output infrared image.

18. The method of claim 17, wherein at least a portion of the pixel offset information corresponds to a golden pixel having an approximately constant offset term.

19. The method of claim 18, wherein the pixel offset information corresponding to the golden pixel is propagated to learning pixels and/or enlightened pixels corresponding to the other ones of the scene based non uniformity correction terms.

20. The method of claim 19, wherein the comparing generates a difference frame, and wherein the method further comprises:

determining a certainty estimate for pixel data in the difference frame; and

performing a masking process on the pixel data in the difference frame based on the determining of the certainty estimate.

21. The method of claim 20, wherein at least some of the learning pixels become the enlightened pixels, which propagate their pixel offset information and/or accumulated certainty estimate to other pixels corresponding to additional ones of the scene based non uniformity correction terms, upon accumulation of a sufficient level of the certainty estimate for the learning pixels.

22. The method of claim 21, further comprising:

determining the frame-to-frame motion between at least some of the first and second plurality of pixel data; and

warping the template frame based on the determining of the frame-to-frame motion.

23. The method of claim 22, wherein updated values for the scene based non uniformity correction terms are weighted mean values, with weighting determined by the accumulated certainty estimates corresponding to the golden or enlightened pixels from which data are being propagated, certainties of the frame-to-frame motion, and the accumulated certainty estimate of the enlightened or learning pixels to which the data are being propagated.

24. The method of claim 23, wherein the applying of the scene based non uniformity correction terms is performed only if the scene based non uniformity correction terms are determined to be stable based on the determining of the certainty estimate.