Processing an input image to reduce compression-related artifacts

• US 7,965,900 B2
• Filed: 09/26/2007
• Issued: 06/21/2011
• Est. Priority Date: 09/26/2007
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Maurer, Ron, Staelin, Carl
• Primary Examiner(s): Ahmed; Samir A
• Assistant Examiner(s): Liu; Li
• Application Number:
  US11/862,125
• Publication Number:
  US 20090080798A1
• Time in Patent Office: 1,364 Days
• Field of Search: None
• US Class Current: 382/268
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/48 using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data motion estimation in a transform domain H04N19/547; processing of decoded motion vectors H04N19/513
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness

Abstract

In a method for processing an input image to reduce compression-related artifacts, a plurality of block transforms of the input image having respective plurality of block-grid locations are generated, where each of the block-grid locations is shifted with respect to the block-grid locations in each of the other block transformed images. In addition, a plurality of respective intermediate transform domain filtered images having modified transform coefficients are generated. Per-pixel relative weights are assigned to each of the intermediate transform domain filtered images, where each of the intermediate transform domain filtered images has different contexts, and where the per-pixel relative weights are adaptive to the local context in each of the intermediate transform domain filtered images. Moreover, a weighted average of inverse-transforms of the intermediate transform domain filtered images from the per-pixel relative weights to produce an artifact-reduced image is calculated and the artifact-reduced image is outputted.

Claims

1. A method for processing an input image to reduce compression-related artifacts, said method comprising:

generating a plurality of block transforms of the input image having respective plurality of block-grid locations, wherein each of the block-grid locations is shifted with respect to the block-grid locations in each of the other block transformed images;

generating a plurality of respective intermediate transform domain filtered images having modified transform coefficients;

assigning per-pixel relative weights to each of the intermediate transform domain filtered images, wherein each of the intermediate transform domain filtered images has different contexts, and wherein the per-pixel relative weights are adaptive to the local context in each of the intermediate transform domain filtered images;

computing a weighted average of inverse-transforms of the intermediate transform domain filtered images from the per-pixel relative weights to produce a artifact-reduced image; and

outputting the artifact-reduced image.

2. The method according to claim 1, further comprising:

computing an inverse block transform of the intermediate transform domain filtered images to generate a plurality of respective candidate reconstructions; and

wherein assigning the per-pixel relative weights further comprises calculating the per-pixel relative weights from either the intermediate transform domain filtered images or the respective candidate reconstructions.

3. The method according to claim 2, further comprising:

combining one of the plurality of intermediate transform domain filtered images and the plurality of candidate reconstructions along with their corresponding per-pixel relative weights to produce the artifact-reduced image by multiplying values of the intermediate transform domain filtered images or the candidate reconstructions with their corresponding per-pixel relative weights to obtain respective products and accumulating the products to obtain a summed product per-pixel, and by accumulating the relative weights to obtain a normalization sum per-pixel and dividing the summed product per-pixel by the normalization sum per-pixel.

4. The method according to claim 3, further comprising:

accumulating the product of the per-pixel relative weights with one of the plurality of intermediate transform domain filtered images and the plurality of candidate reconstructions and accumulating a sum of per-pixel weights for producing the artifact-reduced image without having to store intermediate per-pixel weights, thereby improving the computational efficiency in processing the input image and increasing a range of hardware platforms that accommodate the processing.

5. The method according to claim 2, further comprising:

combining one of the plurality of intermediate transform domain filtered images and the plurality of candidate reconstructions (û

_s (b,m)) along with their corresponding per-pixel relative weights (w_s(b,m)) to produce the artifact-reduced image (û

_AR(n)) according to the following equation;

6. The method according to claim 2, wherein assigning the per-pixel relative weights to one of the plurality of intermediate transform domain filtered images and the plurality of candidate reconstructions further comprises determining activity measures of each of the blocks aligned with the shifted block-transform grid in the intermediate transform domain filtered images or the candidate reconstructions to obtain relative self-confidences of each of the intermediate transform domain filtered images or the candidate reconstructions with respect to a pixel of interest.

7. The method according to claim 6, wherein determining the activity measure per each block in one of the plurality of intermediate transform domain filtered images and the plurality of candidate reconstructions further comprises computing the activity measures based upon a quantity that is related to a magnitude of differences between the pixels contained in the respective blocks of the shifted block-transform grid corresponding to that intermediate transform domain filtered image or candidate reconstruction.

8. The method according to claim 7, wherein the activity measure is block variance computed from one of directly from the pixel values and from transform coefficient values.

9. The method according to claim 8, wherein determining the activity measures further comprises determining the activity measures A_s(b) for each of the blocks b in one of the plurality of intermediate transform domain filtered images and the plurality of candidate reconstructions (û

_s(b,m)), which are frequency-weighted variances, for a block-grid shift s based upon the following equation;

A_s(b)=Σ

<sub>k≠

0</sub>α

[Σ

_mT_k,mû

_s(b,m)]²=Σ

<sub>k≠

0</sub>α

_k·

[ψ

_k(σ

_k|Σ

_jT_k,jf_s(b,j))]²∀

b,s,

wherein b is a block-index of the pixel of interest, m is the block in which the pixel of interest is located s denotes the shift of the block-transform rid α and

α

_k are frequency weighting matrices, σ

_k is a local feature-contrast scale, k corresponds to the kth 2D-transform coefficient, j denotes indices of the quantization element, f is the compressed image, T denotes the block transform employed, and Ψ

_k denotes the set of non-linear mappings that are each applied to their corresponding transform coefficient k.

10. The method according to claim 6, wherein assigning per-pixel relative weights of one of the plurality of intermediate transform domain filtered images and the plurality of candidate reconstructions further comprises assigning relatively lower self-confidences to the pixels in the intermediate transform domain filtered images or the candidate reconstructions having relatively higher activity measures and assigning relatively higher self-confidences to the pixels in the intermediate transform domain filtered images or the candidate reconstructions having relatively lower activity measures.

11. The method according to claim 10, wherein assigning per-pixel relative weights between two intermediate transform domain filtered images or between two candidate reconstructions corresponding to two shifts of block-grids (s1) and (s2) further comprises calculating per-pixel relative weights based upon the following equation to determine a relative preference (R_s1,s2(b,m)) between the two intermediate transform domain filtered images or the two candidate reconstructions (s1) and (s2):

12. The method according to claim 2, wherein assigning per-pixel relative weights of one of the plurality of intermediate transform domain filtered images and the plurality of candidate reconstructions further comprises calculating the per-pixel relative weights w_s(b,m) in a manner designed to protect desired features from filtering while allowing artifacts to be filtered according to the following equation:

w_s(b,m)=C_s(b)·

g<sub>σ

(n)</sub>(|û

_s(n)−

f_s(n)|),

wherein C_s(b) is a self-confidence attribute measure for block b and a shift s, n is a pixel address of a pixel of interest, m is the block in which the pixel of interest is located, g_σ

(t) is a decreasing function of the difference between û

_s(b,m) and the original pixel value in that location f_s(n) divided by a positive local feature contrast scale σ

(n), such that g_σ

(0)=1, g_σ

(σ

)˜

0.5, g_σ

(t>

>

σ

)<

<

1, and the local feature contrast scale σ

(n) is computed per pixel by a feature detector that is insensitive to the features desired for protection, and sensitive to artifacts desired to be filtered.

13. The method according to claim 12, wherein the local characteristic feature contrast scale is computed according to the following equation:

14. A system for processing an input image to reduce compression-related artifacts, said system comprising:

a processor to implement;

a forward transform module configured to generate a plurality of block transforms of the input image having respective plurality of block-grid locations, wherein each of the block-grid locations is shifted with respect to the block-grid locations in each of the other block transformed images, said forward transform module being further configured to generate a plurality of respective intermediate transform domain filtered images having modified transform coefficients;

a block transform filter configured to compute inverse transforms of the intermediate transform domain filtered images to generate a plurality of respective candidate reconstructions;

a local weights generator configured to assign per-pixel relative weights to one of the intermediate transform domain filtered images and the candidate reconstructions, wherein each of the intermediate transform domain filtered images and the candidate reconstructions has different contexts, and wherein the per-pixel relative weights are adaptive to the local context in each of the intermediate transform domain filtered images or candidate reconstructions;

a weighted average generator configured to compute a weighted average of one of the intermediate transform domain filtered images and the candidate reconstructions from the per-pixel relative weights to produce an artifact-reduced image; and

an output for outputting the artifact-reduced image.

15. The system according to claim 14, wherein the weighted average generator is further configured to combine one of the plurality of intermediate transform domain images and the plurality of candidate reconstructions (û

_s (b,m)) along with their corresponding per-pixel relative weights (w_s(b,m)) to produce the artifact-reduced image (û

_AR(n)) according to the following equation;

16. The system according to claim 15, wherein the local weights generator is further configured to accumulate the product of the per-pixel relative weights with one of the plurality of intermediate transform domain filtered images and the plurality of candidate reconstructions and accumulating a sum of per-pixel weights for producing the artifact-reduced image (û

_AR(n)) without having to store intermediate per-pixel weights, thereby improving the computational efficiency in processing the input image and increasing a range of hardware platforms that accommodate the processing.

17. The system according to claim 14, wherein the local weights generator is further configured to determine activity measures A_s(b) for each of the blocks b in one of the plurality of intermediate transform domain filtered images and the plurality of candidate reconstructions (û

_s(b,m)), which are frequency-weighted variances for a block-grid shift s based upon the following equation;

A_s(b)=Σ

<sub>k≠

0</sub>α

[Σ

_mT_k,mû

_s(b,m)]²=Σ

<sub>k≠

0</sub>α

_k·

[ψ

_k(σ

_k|Σ

_jT_k,jf_s(b,j))]²∀

b,s,

wherein b is a block-index of the pixel of interest, m is the block in which the pixel of interest is located, s denotes the shift of the block-transform grid, α

_k is a frequency weighting matrix, σ

_k is a local feature-contrast scale, k corresponds to the kth 2D-transform coefficient, j denotes indices of the quantization element, f is the compressed image, T denotes the block transform employed, and Ψ

_k denotes the set of non-linear mappings that are each applied to their corresponding transform coefficient k.

18. The system according to claim 14, wherein the local weights generator is further configured to calculate the per-pixel relative weights w_s(b,m) in a manner designed to protect desired features from filtering while allowing artifacts to be filtered according to the following equation:

w_s(b,m)=C_s(b)·

g<sub>σ

(n)</sub>(|û

_s(n)−

f_s(n)|),

wherein C_s(b) is a self-confidence attribute measure for block b and a shift s, m is the block in which the pixel of interest is located, n is a pixel address of a pixel of interest, g_σ

(t) is a decreasing function of the difference between û

_s (b,m) and the original pixel value in that location f_s(n) divided by a positive local feature contrast scale σ

(n), such that g_σ

(0)=1, g_σ

(σ

)˜

0.5, g_σ

(t>

>

σ

)<

<

1, and the local feature contrast scale σ

(n) is computed per pixel by a feature detector that is insensitive to the features desired for protection, and sensitive to artifacts desired to be filtered.

19. A computer readable storage medium on which is embedded one or more computer programs, said one or more computer programs implementing a method for processing an input image, said one or more computer programs comprising computer readable code for:

generating a plurality of block transforms of the input image having respective plurality of block-grid locations, wherein each of the block-grid locations is shifted with respect to the block-grid locations in each of the other block transformed images;

generating a plurality of respective intermediate transform domain filtered images having modified transform coefficients;

assigning per-pixel relative weights to each of the intermediate images, wherein each of the intermediate images has different contexts, and wherein the per-pixel relative weights are adaptive to the local context in each of the intermediate images;

computing a weighted average of inverse-transforms of the intermediate images from the per-pixel relative weights of the intermediate images to produce an artifact-reduced image; and

outputting the artifact-reduced image.

20. The computer readable medium according to claim 19, said one or more computer programs further comprising computer readable code for:

combining each of the intermediate transform domain filtered images (û

_s (b,m)) along with their corresponding per-pixel relative weights (w_s(b,m)) to produce the artifact-reduced image (û

_AR(n)) according to the following equation;