Pixel-wise noise reduction in thermal images

• US 9,208,542 B2
• Filed: 09/17/2013
• Issued: 12/08/2015
• Est. Priority Date: 03/02/2009
• Status: Active Grant

• Current Assignee: Teledyne FLIR LLC (Teledyne Technologies Incorporated)
• Original Assignee: FLIR Systems, Inc. (Teledyne Technologies Incorporated)
• Inventors: Hgasten, Nicholas, Nussmeier, Mark, Kurth, Eric A., Hoelter, Theodore R., Strandemar, Katrin, Boulanger, Pierre, Sharp, Barbara, Ingerhed, Malin
• Primary Examiner(s): Patel, Kanjibhai
• Application Number:
  US14/029,683
• Publication Number:
  US 20140016879A1
• Time in Patent Office: 812 Days
• Field of Search: 382/260, 382274-275, 382263-264
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/20182 Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: H04N 25/671 for non-uniformity detection or correction
• CPC Class Codes: H04N 25/677 for reducing the column or line fixed pattern noise
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Methods and systems are provided to reduce noise in thermal images. In one example, a method includes receiving an image frame comprising a plurality of pixels arranged in a plurality of rows and columns. The pixels comprise thermal image data associated with a scene and noise introduced by an infrared imaging device. The image frame may be processed to determine a plurality of column correction terms, each associated with a corresponding one of the columns and determined based on relative relationships between the pixels of the corresponding column and the pixels of a neighborhood of columns. In another example, the image frame may be processed to determine a plurality of non-uniformity correction terms, each associated with a corresponding one of the pixels and determined based on relative relationships between the corresponding one of the pixels and associated neighborhood pixels within a selected distance.

Claims

1. A method comprising:

receiving an image frame comprising a plurality of pixels arranged in a plurality of rows and columns, wherein the pixels comprise thermal image data associated with a scene and noise introduced by an infrared imaging device; and

processing the image frame to determine a plurality of non-uniformity correction (NUC) terms to reduce at least a portion of the noise, wherein each NUC term is associated with a corresponding one of the pixels and is determined based on relative relationships between the corresponding one of the pixels and associated neighborhood pixels within a selected distance from the corresponding one of the pixels, wherein the processing comprises;

for each pixel of the image frame, comparing the pixel to each of its associated neighborhood pixels,

for each comparison, adjusting a first counter if the pixel has a value greater than the compared neighborhood pixel,

for each comparison, adjusting a second counter if the pixel has a value less than the compared neighborhood pixel, and

selectively updating the NUC term associated with the pixel based on the first and second counters.

2. The method of claim 1, further comprising, for each comparison, adjusting a third counter if the pixel has a value equal to the compared neighborhood pixel, wherein the updating is also based on the third counter.

4. The method of claim 1, wherein the associated neighborhood pixels comprise at least one pixel in a different column and a different row than the corresponding one of the pixels.

5. The method of claim 1, wherein the distance is a first distance, wherein the portion of the noise is a first portion, the method further comprising repeating the processing using a second distance to update the NUC terms to reduce at least a second portion of the noise.

6. The method of claim 1, wherein the noise comprises spatially uncorrelated fixed pattern noise (FPN) and spatially correlated FPN.

7. The method of claim 1, further comprising applying the NUC terms to the image frame to remove the portion of the noise.

8. The method of claim 1, further comprising temporally damping the NUC terms.

9. The method of claim 1, further comprising weighting the NUC terms based on gradients between the pixels.

10. The method of claim 1, further comprising:

processing the image frame to determine a plurality of spatially correlated fixed pattern noise (FPN) terms to reduce a portion of the noise comprising spatially correlated FPN associated with rows or columns of infrared sensors of the infrared imaging device; and

applying the spatially correlated FPN terms to the image frame before the NUC terms are determined.

11. The method of claim 1, wherein the image frame is an intentionally blurred image frame, wherein the thermal image data is blurred thermal image data.

3. A method comprising:

receiving an image frame comprising a plurality of pixels arranged in a plurality of rows and columns, wherein the pixels comprise thermal image data associated with a scene and noise introduced by an infrared imaging device; and

processing the image frame to determine a plurality of non-uniformity correction (NUC) terms to reduce at least a portion of the noise, wherein each NUC term is associated with a corresponding one of the pixels and is determined based on relative relationships between the corresponding one of the pixels and associated neighborhood pixels within a selected distance from the corresponding one of the pixels, wherein the processing comprises;

for each pixel of the image frame, comparing the pixel to each of its associated neighborhood pixels,

for each comparison, adjusting a counter in a first manner if the pixel has a value greater than the compared neighborhood pixel or a second manner if the pixel has a value less than the compared neighborhood pixel, and

selectively updating the NUC term associated with the pixel based on the counter.

12. A system comprising:

a memory component adapted to receive an image frame comprising a plurality of pixels arranged in a plurality of rows and columns, wherein the pixels comprise thermal image data associated with a scene and noise introduced by an infrared imaging device; and

a processor adapted to execute instructions to process the image frame to determine a plurality of non-uniformity correction (NUC) terms to reduce at least a portion of the noise, wherein each NUC term is associated with a corresponding one of the pixels and is determined based on relative relationships between the corresponding one of the pixels and associated neighborhood pixels within a selected distance from the corresponding one of the pixels, wherein the instructions to process the image frame are adapted to cause the processor to;

for each pixel of the image frame, compare the pixel to each of its associated neighborhood pixels,

for each comparison, adjust a first counter if the pixel has a value greater than the compared neighborhood pixel,

for each comparison, adjust a second counter if the pixel has a value less than the compared neighborhood pixel, and

selectively update the NUC term associated with the pixel based on the first and second counters.

13. The system of claim 12, wherein:

the processor is adapted to execute instructions to, for each comparison, adjust a third counter if the pixel has a value equal to the compared neighborhood pixel; and

the instructions to update the NUC term are adapted to cause the processor to selectively update the NUC term also based on the third counter.

15. The system of claim 12, wherein the associated neighborhood pixels comprise at least one pixel in a different column and a different row than the corresponding one of the pixels.

16. The system of claim 12, wherein the distance is a first distance, wherein the portion of the noise is a first portion, wherein the processor is adapted to execute instructions to repeat the process using a second distance to update the NUC terms to reduce at least a second portion of the noise.

17. The system of claim 12, wherein the noise comprises spatially uncorrelated fixed pattern noise (FPN) and spatially correlated FPN.

18. The system of claim 12, wherein the processor is adapted to execute instructions to apply the NUC terms to the image frame to remove the portion of the noise.

19. The system of claim 12, wherein the processor is adapted to execute instructions to temporally damp the NUC terms.

20. The system of claim 12, wherein the processor is adapted to execute instructions to weight the NUC terms based on gradients between the pixels.

21. The system of claim 12, wherein the processor is adapted to execute instructions to:

process the image frame to determine a plurality of spatially correlated fixed pattern noise (FPN) terms to reduce a portion of the noise comprising spatially correlated FPN associated with rows or columns of infrared sensors of the infrared imaging device; and

apply the spatially correlated FPN terms to the image frame before the NUC terms are determined.

22. The system of claim 12, wherein the image frame is an intentionally blurred image frame, wherein the thermal image data is blurred thermal image data.

23. The system of claim 12, further comprising a focal plane array (FPA) adapted to provide the image frame, wherein the FPA comprises an array of microbolometers adapted to receive a bias voltage selected from a range of approximately 0.2 to approximately 0.7 volts.

14. A system comprising:

a memory component adapted to receive an image frame comprising a plurality of pixels arranged in a plurality of rows and columns, wherein the pixels comprise thermal image data associated with a scene and noise introduced by an infrared imaging device; and

a processor adapted to execute instructions to process the image frame to determine a plurality of non-uniformity correction (NUC) terms to reduce at least a portion of the noise, wherein each NUC term is associated with a corresponding one of the pixels and is determined based on relative relationships between the corresponding one of the pixels and associated neighborhood pixels within a selected distance from the corresponding one of the pixels, wherein the instructions to process the image frame are adapted to cause the processor to;

for each pixel of the image frame, compare the pixel to each of its associated neighborhood pixels,

for each comparison, adjust a counter in a first manner if the pixel has a value greater than the compared neighborhood pixel or a second manner if the pixel has a value less than the compared neighborhood pixel, and

selectively update the NUC term associated with the pixel based on the counter.