Methods and systems for image enhancement and estimation of compression noise

• US 8,538,193 B2
• Filed: 03/29/2012
• Issued: 09/17/2013
• Est. Priority Date: 09/28/2010
• Status: Expired due to Fees

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Laboratories of America Incorporated (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Segall, Christopher A.
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US13/434,717
• Publication Number:
  US 20120189216A1
• Time in Patent Office: 537 Days
• Field of Search: 382/232, 382/266, 382/275, 382/298, 37524026-24029
• US Class Current: 382/275
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness

Abstract

Aspects of the present invention are related to systems and methods for determining the compression noise present in an image or frame in a video sequence and for enhancing an image or frame in a video sequence.

Claims

1. A method for estimating compression noise in an image, said method comprising:

receiving an image in a noise-estimation system;

determining a first compression-noise estimate, wherein said first compression-noise estimate is associated with no scaling of said image;

determining if said image has been scaled;

when said image has been scaled, estimating a second compression-noise estimate, for said image, based on said first compression-noise estimate and a third compression-noise estimate, wherein said third compression-noise estimate is associated with a scaling of said image at a first scaling factor; and

when said image has not been scaled, estimating said second compression-noise estimate, for said image, based on said first compression-noise estimate.

2. The method according to claim 1, wherein:

said determining if said image has been scaled comprises;

determining if said image has been scaled in a horizontal direction; and

determining if said image has been scaled in a vertical direction.

3. The method according to claim 1, wherein:

said determining said first compression-noise estimate comprises determining a first vertical compression-noise estimate and a first horizontal compression-noise estimate;

wherein said determining said first vertical compression-noise estimate comprises;

computing a vertical difference value for each pixel in a first plurality of pixels in said image, thereby producing a plurality of vertical difference values;

computing a plurality of vertical standard deviation values, wherein each of said vertical standard deviation values in said plurality of vertical standard deviation values is associated with a vertical offset within said image;

calculating a first vertical statistic associated with said plurality of vertical standard deviation values;

calculating a second vertical statistic associated with said plurality of vertical standard deviation values; and

combining said first vertical statistic and said second vertical statistic to form said first vertical compression-noise estimate; and

wherein said determining said first horizontal compression-noise estimate comprises;

computing a horizontal difference value for each pixel in a second plurality of pixels in said image, thereby producing a plurality of horizontal difference values;

computing a plurality of horizontal standard deviation values, wherein each of said horizontal standard deviation values in said plurality of horizontal standard deviation values is associated with a horizontal offset within said image;

calculating a first horizontal statistic associated with said plurality of horizontal standard deviation values;

calculating a second horizontal statistic associated with said plurality of horizontal standard deviation values; and

combining said first horizontal statistic and said second horizontal statistic to form said first horizontal compression-noise estimate.

4. The method as described in claim 3, wherein said horizontal offset and said vertical offset are related to a coding block offset associated with a codec associated with said image.

5. The method as described in claim 3, wherein:

said first horizontal statistic is a mean of said horizontal standard deviation values; and

said first vertical statistic is a mean of said vertical standard deviation values.

6. The method as described in claim 3, wherein:

said second horizontal statistic is a maximum of said horizontal standard deviation values; and

said second vertical statistic is a maximum of said vertical standard deviation values.

7. The method as described in claim 3, wherein:

said combining said first horizontal statistic and said second horizontal statistic comprises computing a weighted average of said first horizontal statistic and said second horizontal statistic; and

said combining said first vertical statistic and said second vertical statistic comprises computing a weighted average of said first vertical statistic and said second vertical statistic.

8. The method as described in claim 3, wherein said image is a first frame in a video sequence and further comprising:

estimating a horizontal block-boundary location associated with each frame in a plurality of frames in said video sequence, wherein said first frame is in said plurality of frames, thereby producing a plurality of horizontal block-boundary locations;

estimating a vertical block-boundary location associated with each frame in said plurality of frames in said video sequence, thereby producing a plurality of vertical block-boundary locations;

determining a number of unique horizontal block-boundary locations in said plurality of horizontal block-boundary locations;

determining a number of unique vertical block-boundary locations in said plurality of vertical block-boundary locations;

comparing said number of unique horizontal block-boundary locations in said plurality of horizontal block-boundary locations to a horizontal threshold;

comparing said number of unique vertical block-boundary locations in said plurality of vertical block-boundary locations to a vertical threshold;

setting said first horizontal compression-noise estimate to zero when said number of unique horizontal block-boundary locations in said plurality of horizontal block-boundary locations and said horizontal threshold meet a horizontal criterion; and

setting said first vertical compression-noise estimate to zero when said number of unique vertical block-boundary locations in said plurality of vertical block-boundary locations and said vertical threshold meet a vertical criterion.

9. The method as described in claim 1, wherein said receiving said image comprises:

receiving a bitstream associated with said image; and

decoding said bitstream.

10. The method as described in claim 1, wherein said receiving said image comprises:

receiving a bitstream associated with said image;

decoding said bitstream, thereby producing a decoded image; and

resizing said decoded image, thereby producing said image.

11. The method as described in claim 10, wherein said resizing comprises resizing said decoded image one direction selected from the group consisting of the horizontal direction and the vertical direction.

12. The method as described in claim 10, wherein said resizing comprises resizing said decoded image in the vertical direction and in the horizontal direction.

13. A method for estimating compression noise in an image, said method comprising:

receiving an image in a noise-estimation system;

determining a first compression-noise estimate, wherein said first compression-noise estimate is associated with no scaling of said image;

determining if said image has been scaled in a first direction;

when said image has been scaled in said first direction, estimating a second compression-noise estimate, for said image, based on said first compression-noise estimate and a third compression-noise estimate, wherein said third compression-noise estimate is associated with a scaling of said image, in said first direction, at a first scaling factor; and

when said image has not been scaled in said first direction, estimating said second compression-noise estimate, for said image, based on said first compression-noise estimate.

14. The method as described in claim 13, wherein said determining said third compression-noise estimate comprises:

computing a first-direction difference value for each pixel in a first plurality of pixels in said image, thereby producing a plurality of first-direction difference values;

computing a first plurality of first-direction standard deviation values, wherein each of said first-direction standard deviation values in said plurality of first-direction standard deviation values is associated with a subset of offsets associated with a first-direction base offset within said image;

combining said first plurality of first-direction standard deviation values to form a second plurality of first-direction standard deviation values;

calculating a first first-direction statistic associated with said second plurality of first-direction standard deviation values;

calculating a second first-direction statistic associated with said second plurality of first-direction standard deviation values; and

combining said first first-direction statistic and said second first-direction statistic to form said third compression-noise estimate.

15. The method as described in claim 14, wherein:

said first first-direction statistic is a mean of said second plurality of first-direction standard deviation values; and

said second horizontal statistic is a maximum of said second plurality of first-direction standard deviation values.

16. The method as described in claim 14, wherein said combining said first first-direction statistic and said second first-direction statistic comprises computing a weighted average of said first first-direction statistic and said second first-direction statistic.

17. The method as described in claim 14, wherein said image is a first frame in a video sequence and further comprising:

estimating a first-direction block-boundary location associated with each frame in a plurality of frames in said video sequence, wherein said first frame is in said plurality of frames, thereby producing a plurality of first-direction block-boundary locations;

determining a number of unique first-direction block-boundary locations in said plurality of first-direction block-boundary locations;

comparing said number of unique first-direction block-boundary locations in said plurality of first-direction block-boundary locations to a first-direction threshold;

setting said second first-direction compression-noise estimate to zero when said number of unique first-direction block-boundary locations in said plurality of first-direction block-boundary locations and said first-direction threshold meet a first-direction criterion.

18. A method for estimating compression noise in an image, said method comprising:

receiving an image in a noise-estimation system;

estimating a vertical compression noise associated with no image scaling;

estimating a horizontal compression noise associated with no image scaling;

setting a vertical compression-noise estimate to said vertical compression noise associated with no image scaling;

estimating a horizontal compression noise associated with a first scaling;

setting a horizontal compression-noise estimate to the maximum of said horizontal compression noise associated with no image scaling and said horizontal compression noise associated with a first scaling; and

setting a compression-noise estimate to the maximum of said vertical compression-noise estimate and said horizontal compression-noise estimate.

19. A system for enhancing an input image, said system comprising:

an edge-preserving filter for generating, from an input image, a first image comprising image content, from said input image, to sharpen and a second image comprising image content, from said input image, to attenuate; and

a noise estimator for determining a noise estimate associated with said input image, wherein;

a parameter value associated with said edge-preserving filter is based on said noise estimate; and

said noise estimator comprises;

a first compression-noise estimator for determining a first compression-noise estimate, wherein said first compression-noise estimate is associated with no scaling of said input image;

a scale determiner for determining if said input image has been scaled;

a second compression-noise estimator for determining a second compression-noise estimate, wherein;

when said image has been scaled, estimating said second compression-noise estimate, for said input image, based on said first compression-noise estimate and a third compression-noise estimate, wherein said third compression-noise estimate is associated with a scaling of said input image at a first scaling factor; and

when said input image has not been scaled, estimating said second compression-noise estimate, for said input image, based on said first compression-noise estimate.

20. The system as described in claim 19, wherein said edge-preserving filter comprises a bi-lateral filter.

21. The system as described in claim 19, wherein said noise estimator further comprises:

an image-noise estimator for estimating at least one of grain noise, sensor noise and thermal noise in said input image, thereby producing an image-noise statistic; and

a combiner for combining said image-noise statistic and said second compression-noise estimate, wherein said noise estimate is based on said combined image-noise statistic and said second compression-noise estimate.