Methods, systems, and computer readable media for reconstructing images using blurring and noise-enhanced pixel intensity resampling

• US 10,349,087 B2
• Filed: 07/13/2018
• Issued: 07/09/2019
• Est. Priority Date: 07/13/2017
• Status: Active Grant

• Current Assignee: University of North Carolina At Chapel Hill (University of North Carolina System)
• Original Assignee: University of North Carolina At Chapel Hill (University of North Carolina System)
• Inventors: Taylor, II, Russell Morton, Shao, Chong, Mayer-Patel, Ketan Dasharath
• Primary Examiner(s): Safaipour, Bobbak
• Application Number:
  US16/035,016
• Publication Number:
  US 20190020903A1
• Time in Patent Office: 361 Days
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20182 Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness

Abstract

Methods, systems, and non-transitory computer readable media for reconstructing electronic images using blurring and noise-enhanced pixel intensity resampling are disclosed. One method includes, at an image reconstructor including at least one processor, receiving an image compressed using an image compression algorithm. The method further includes decompressing the image using an image decompression algorithm. The method further includes modifying pixel intensities in the image by blurring and/or resampling the pixel intensities according to a noise distribution generated by the reconstructor without receiving a noise estimate from an image compressor. The method further includes outputting the image with the modified pixel intensities.

Claims

1. A method for reconstructing an electronic image using blurring and noise-enhanced pixel resampling, the method comprising:

at an image reconstructor including a decompressor, a blurrer/resampler, and

an image quality analyzer, and at least one processor;

receiving, at the decompressor, an image compressed using an image compression algorithm;

decompressing, by the decompressor, the image using an image decompression algorithm and producing a decompressed image;

analyzing, by the image quality analyzer, the decompressed image for blockiness or banding and detecting blockiness or banding in the decompressed image;

modifying, by the blurrer/resampler, pixel intensities in the decompressed image by selecting new pixel intensity values using a noise distribution generated by the reconstructor without receiving a noise estimate from an image compressor and thereby producing a decompressed image with modified pixel intensities and less blockiness or banding than the decompressed image produced by the decompressor; and

outputting, by the blurrer/resampler, the decompressed image with the modified pixel intensities and less blockiness or banding than the decompressed image produced by the decompressor.

2. The method of claim 1 modifying the pixel intensities includes, at the reconstructor, using a parameter representing an amount of noise expected in the pixel intensities to construct the noise distribution.

3. The method of claim 2 wherein the parameter comprises a scale factor representing a number of photons per pixel intensity count and wherein selecting new values for the pixel intensities includes selecting a scale factor to apply to each pixel intensity before sampling from a Poisson distribution separately for red, green, and blue intensities for each pixel and then re-scaling by the inverse of the scale factor to produce new pixel intensities.

4. The method of claim 2 wherein the parameter is a width of a Gaussian distribution in pixel intensity counts and wherein selecting new values for the pixel intensities includes sampling pixel intensity counts from the Gaussian distribution and adding or subtracting the pixel intensity counts from the pixel intensities in the decompressed image.

5. The method of claim 2 wherein the parameter is a width of a Gaussian distribution centered around 1 and wherein selecting new values for the pixel intensities includes sampling values from the Gaussian and multiplying sampled values by the pixel intensities in the decompressed image.

6. The method of claim 1 wherein selecting new values for the pixel intensities includes blurring the image using a Gaussian distribution having a width scaled by a parameter representing an expected size of a smallest correctly represented feature in the decompressed image.

7. The method of claim 2 wherein the parameter represents an expected spatial extent of a smallest properly-encoded feature in the image and wherein selecting new values for the pixel intensities includes sampling values from the pixel intensities from neighboring pixels using a spatial sampling distribution centered at each pixel and scaled according the parameter.

8. The method of claim 1 wherein the image is a first image in a sequence of video frames and wherein the method further comprises:

receiving a second image in the sequence of video frames and compressed using the image compression algorithm;

decompressing the second image using the image decompression algorithm to produce a second decompressed image;

modifying pixel intensities in the second decompressed image by selecting new values for the pixel intensities using a noise distribution generated by the reconstructor without receiving a noise estimate from an image compressor; and

selecting, for each pixel position in the second decompressed image, a pixel intensity for the pixel position in the first image or a pixel intensity for the pixel position in the second image using a probability distribution that varies over time.

9. A system for reconstructing an electronic image, the system comprising:

an image reconstructor including at least one processor, the image reconstructor including;

an image decompressor for receiving an image compressed using an image compression algorithm and decompressing the image using an image decompression algorithm to produce a decompressed image;

an image quality analyzer for analyzing the decompressed image for blockiness or banding and detecting blockiness or banding in the decompressed image; and

a pixel intensity blurrer/resampler for modifying pixel intensities in the decompressed image by selecting new values for the pixel intensities using a noise distribution generated by the reconstructor without receiving a noise estimate from an image compressor, thereby producing a decompressed image with modified pixel intensities and less blockiness or banding than the decompressed image produced by the decompressor, and outputting the decompressed image with the modified pixel intensities and less blockiness or banding than the decompressed image produced by the decompressor.

10. The system of claim 9 wherein modifying the pixel intensities includes, at the reconstructor, using a parameter representing an amount of noise expected in the pixel intensities to construct the noise distribution.

11. The system of claim 10 wherein the parameter comprises a scale factor representing a number of photons per pixel intensity count and wherein selecting new values for the pixel intensities includes selecting the scale factor to apply to each pixel intensity before sampling from a Poisson distribution separately for red, green, and blue intensities for each pixel and then rescaling the samples by the inverse of the parameter to determine the new pixel intensities.

12. The system of claim 10 wherein the parameter is a width of a Gaussian distribution in pixel intensity counts and wherein selecting new values for the pixel intensities includes sampling pixel intensity counts from the Gaussian distribution and adding or subtracting the pixel intensity counts from the pixel intensities in the decompressed image.

13. The system of claim 10 wherein the parameter is a width of a Gaussian distribution centered around 1 and wherein selecting new values for the pixel intensities includes sampling values from the Gaussian and multiplying sampled values by the pixel intensities in the decompressed image.

14. The system of claim 9 wherein selecting new values for the pixel intensities includes blurring the image using a Gaussian distribution having a width scaled by a parameter representing an expected size of a smallest correctly represented feature in the decompressed image.

15. The system of claim 10 wherein the parameter represents an expected spatial extent of a smallest properly-encoded feature in the image and wherein selecting new values for the pixel intensities sampling the pixel intensities from neighboring pixels using a spatial sampling distribution centered at each pixel and scaled according the parameter.

16. The system of claim 9 wherein:

the image is a first image in a sequence of video frames;

the image reconstructor receives a second image in the sequence of video frames and compressed using the image compression algorithm;

the image decompressor decompresses the second image using the image decompression algorithm;

the pixel intensity blurrer/resampler modifies pixel intensities in the second image selecting new values for the pixel intensities using a noise distribution generated by the reconstructor without receiving a noise estimate from an image compressor; and

the pixel intensity blurrer/resampler selects, for each pixel position in the second image, a pixel intensity for the pixel position in the first image or a pixel intensity for the pixel position in the second image using a probability distribution that varies over time.

17. A non-transitory computer readable medium having stored thereon executable instructions that when executed by a processor of a computer control the computer to perform steps comprising:

at an image reconstructor including a decompressor, a blurrer/resampler, and an image quality analyzer, and at least one processor;

receiving, at the decompressor, an image compressed using an image compression algorithm;

decompressing, by the decompressor, the image using an image decompression algorithm to produce a decompressed image;

analyzing, by the image quality analyzer, the decompressed image for blockiness or banding and detecting blockiness or banding in the decompressed image;

modifying, by the blurrer/resampler, pixel intensities in the decompressed image by selecting new values for the pixel intensities according to a noise distribution generated by the reconstructor without receiving a noise estimate from an image compressor and thereby producing a decompressed image with modified pixel intensities and less blockiness or banding than the decompressed image produced by the decompressor; and

outputting, by the blurrer/resampler, the decompressed image with the modified pixel intensities and less blockiness or banding than the decompressed image produced by the decompressor.