Enhanced image/video quality through artifact evaluation

• US 20070206871A1
• Filed: 03/01/2006
• Published: 09/06/2007
• Est. Priority Date: 03/01/2006
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Chang, Chienchung, Jalil, Suhail, El-Malch, Khaled
• Granted Patent: US 7,873,224 B2
• Time in Patent Office: Days
• US Class Current: 382/233
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/423 characterised by memory arrangements H04N19/433 takes precedence
• CPC Class Codes: H04N 19/527 Global motion vector estimation
• CPC Class Codes: H04N 19/60 using transform coding
• CPC Class Codes: H04N 19/82 involving filtering within a prediction loop

Abstract

In an image/video encoding and decoding system employing an artifact evaluator a method and/or apparatus to process video blocks comprising a decoder operable to synthesize an un-filtered reconstructed video block or frame and an artifact filter operable to receive the un-filtered reconstructed video block or frame, which generates a filtered reconstructed video block or frame. A memory buffer operable to store either the filtered reconstructed video block or frame or the un-filtered reconstructed video block or frame, and an artifact evaluator operable to update the memory buffer after evaluating and determining which of the filtered video block or frame, or the un-filtered video block or frame yields better image/video quality.

Claims

1. An apparatus configured to process video blocks comprising:

a decoder operable to synthesize an un-filtered reconstructed video unit, wherein a video unit is at least one of video block and video frame;

an artifact filter operable to receive the un-filtered reconstructed video unit, and that generates a filtered reconstructed video unit, wherein video unit is at least one of video video block or frame;

a memory buffer operable to store either the filtered reconstructed video unit or the un-filtered reconstructed video unit; and

an artifact evaluator operable to update the memory buffer.

2. The apparatus of claim 1, wherein the artifact evaluator comprises at least one artifact metric generator and a decision logic block.

3. The apparatus of claim 2, wherein any artifact metric generator amongst the at least one artifact metric generator is configured to receive the filtered video unit or the un-filtered reconstructed video unit.

4. The apparatus of claim 3, wherein the at least one artifact metric generators is further configured to receive the original video unit.

5. The apparatus of claim 4, wherein any artifact metric generator amongst the at least one artifact metric generator comprises generating a non-original reference (NR) artifact metric or a full-original reference (FR) artifact metric.

6. The apparatus of claim 5, wherein a first artifact metric generator from the at least one artifact metric generator and a second artifact metric generator from the at least one artifact metric generator are coupled to a first decision logic block or a second decision logic block.

7. The apparatus of claim 6, wherein the first decision logic block is configured to receive the filtered reconstructed video unit and the un-filtered reconstructed video unit, and is further configured to compare a first NR artifact metric with a second NR artifact metric or compare a first FR artifact metric with a second FR artifact metric, and based on either comparison decide to output the filtered reconstructed video unit or the un-filtered reconstructed video unit.

8. The apparatus of claim 6, wherein the second decision logic block is configured to receive the filtered reconstructed video unit, and the unfiltered reconstructed video unit, and is further configured to compare a first set of NR artifact metrics with a first set of NR artifact metrics or compare a first set of FR artifact metrics with a second set of FR artifact metrics, and based on either comparison decide to output either the filtered reconstructed video unit, or the unfiltered reconstructed video unit.

9. The apparatus of claim 7, wherein the output is sent to the memory buffer in-loop in a playback device, mobile device, or computer.

10. The apparatus of claim 8, wherein the output is sent to the memory buffer in-loop in a playback device, mobile device, or computer.

11. An artifact evaluator device comprising:

means for generating at least one artifact metric from an un-filtered reconstructed video unit, wherein a video unit is at least one of video block and frame, and an original video unit, wherein a video unit is at least one of video block and frame;

means for generating at least one artifact metric from a filtered reconstructed video unit, wherein a video unit is at least one of video block and frame, and the original video unit;

means for comparing a first artifact metric with a second artifact metric; and

means for deciding an output based on the means for comparing.

12. The artifact evaluator device of claim 11, wherein the output is coupled to an in-loop memory buffer of an image/video encoder.

13. The device of claim 12, wherein the first artifact metric and the second artifact metric is an non-original reference (NR) artifact metric or a full-original reference (FR) artifact metric.

14. The device of claim 13, wherein the output is the filtered reconstructed video unit, or the un-filtered video unit.

15. An artifact evaluator device comprising:

means for generating at least one artifact metric from an un-filtered reconstructed video unit, wherein a video unit is at least one of video block and frame, and an original video unit, wherein a video unit is at least one of video block and frame;

means for generating at least one artifact metric from a filtered reconstructed video unit, wherein a video unit is at least one of video block and frame, and the original video unit;

means for a first comparison of a first artifact metric with a first artifact threshold;

means for a second comparison of a second artifact metric with a second artifact threshold;

means for a third comparison of a third artifact metric with a third artifact threshold;

means for a fourth comparison of a fourth artifact metric with a fourth artifact threshold; and

means for deciding an output based on various combinations of the means for comparisons.

16. The artifact evaluator device of claim 15, wherein the output is coupled to an in-loop memory buffer of an image/video encoder.

17. The device of claim 16, wherein the first, the second, the third, or the fourth artifact metric is an non-original reference (NR) artifact metric or a full-original reference (FR) artifact metric.

18. The device of claim 17, wherein the output is the filtered reconstructed video unit, or the un-filtered reconstructed video unit.

19. A computer-readable medium configured to store a set of instructions, where the instructions are executable process video blocks, comprising:

computer-readable program code means for generating at least one artifact metric from an un-filtered reconstructed video unit, wherein a video unit is at least one of video block and frame, and an original video unit, wherein a video unit is at least one of video and frame;

computer-readable program code means for generating at least one artifact metric from a filtered reconstructed video unit, wherein a video unit is at least one of video block and frame, and the original video unit;

computer-readable program code means for comparing a first artifact metric with a second artifact metric;

computer-readable program code means for deciding the output based on the means for comparing; and

computer-readable program code means for updating a memory buffer in-loop with an output based on the means for deciding.

20. The computer-readable medium of claim 19, wherein the means for generating at least one artifact metric is an non-original reference (NR) artifact metric or a full-original reference (FR) artifact metric.

21. The computer-readable medium of claim 20, wherein the means for deciding means for updating a memory buffer in-loop further comprises updating the memory buffer with the filtered reconstructed video unit, or the un-filtered reconstructed video unit.

22. The computer-readable medium configured to process video blocks, comprising:

computer-readable program code means for generating at least one artifact metric from an un-filtered reconstructed video unit, wherein a video unit is at least one of video block and frame, and an original video unit, wherein a video unit is at least one of video block and frame;

computer-readable program code means for generating at least one artifact metric from a filtered reconstructed video unit and the original video unit;

computer-readable program code means for a first comparison of a first artifact metric with a first artifact threshold;

computer-readable program code means for a second comparison of a second artifact metric with a second artifact threshold;

computer-readable program code means for a third comparison of a third artifact metric with a third artifact threshold;

computer-readable program code means for a fourth comparison of a fourth artifact metric with a fourth artifact threshold;

computer-readable program code means for deciding the output based on various combinations of the means for comparisons; and

computer-readable program code means for updating a memory buffer in-loop with the output based on the means for deciding.

23. The computer-readable medium of claim 22, wherein the means for generating at least one artifact metric is an non-(original) reference (NR) artifact metric or a full-reference (FR) artifact metric.

24. The computer-readable medium of claim 23, wherein the means for deciding means for updating a memory buffer in-loop further comprises updating the memory buffer with the filtered reconstructed video unit, or the unfiltered reconstructed video unit.

25. A method of artifact evaluation comprising:

inputting an original video unit, wherein a video unit is at least one of video block and frame;

inputting an un-filtered reconstructed video unit, wherein a video unit is at least one of video block and frame;

inputting a filtered reconstructed video unit, wherein a video unit is at least one of video block and frame;

generating at least one artifact metric from the original video unit and the un-filtered reconstructed video unit; and

generating at least one artifact metric from the original video unit and the filtered reconstructed video unit;

26. The method of claim 25, wherein any of the artifact metrics generated from the original video unit and the un-filtered reconstructed video unit and any of the artifact metrics generated from the original video unit and the filtered reconstructed video block measure the amount of blockiness or blurriness.

27. The method of claim 26, further comprising:

comparing any one of the artifact metrics generated from the original video unit and the unfiltered reconstructed video unit with any one of the artifact metrics generated from the original video unit and the filtered reconstructed video unit;

deciding to output the un-filtered reconstructed video unit or the filtered reconstructed video block based on the comparison; and

re-encoding based on the comparison if necessary.

28. The method of claim 27, further comprising:

making a first comparison of any one of the artifact metrics generated from the original video unit and the un-filtered reconstructed video unit with a first artifact threshold;

if any of the artifact metrics used in the first comparison is less than the first artifact threshold, making a second comparison of any one of the artifact metrics generated from the original video unit and the un-filtered reconstructed video unit with a second artifact threshold;

if any of the artifact metrics used in the first comparison is less than the first artifact threshold and any of the artifact metrics used in the second comparison is less than the second artifact threshold, making a third comparison of any one of the artifact metrics generated from the original video unit and the filtered reconstructed video unit with a third artifact threshold;

if any of the artifact metrics used in the third comparison is less than the third artifact threshold, making a fourth comparison of any one of the artifact metrics generated from the original video unit and the filtered reconstructed video unit with a fourth artifact threshold;

deciding to output the filtered reconstructed video unit based on the first and second comparisons;

deciding to output the un-filtered reconstructed video unit based on the third and fourth comparisons; and

if necessary, re-encoding based on either the third or fourth comparison.

29. The method of claim 27, wherein the filtered reconstructed video unit, or the un-filtered reconstructed video unit is stored in a memory buffer being used in-loop as part of an encoding process involving a texture encoder.

30. The method of claim 29, wherein the filtered reconstructed video unit, or the un-filtered reconstructed video unit is stored in a memory buffer being used in-loop as part of an encoding process involving a texture encoder.

31. A method in a image/video encoder comprising:

updating a memory buffer with an output of an artifact evaluator, wherein, the artifact evaluator is used in-loop; and

making a decision with a decision logic block, as part of the artifact evaluator, that

uses a first set of artifact metrics and a second set of artifact metrics to make a comparison, and based on the comparison the artifact evaluator outputs a filtered reconstructed video unit, wherein a video unit is at least one of video block and frame, or an un-filtered reconstructed video unit, wherein a video unit is at least one of video block and frame.

32. The method of claim 31, wherein the memory buffer stores either the filtered reconstructed video unit, or the un-filtered reconstructed video unit.

33. The method of claim 32, wherein the first set of metrics is based on the original video unit and the un-filtered reconstructed video unit.

34. The method of claim 33, wherein the second set of metrics is based on the original video unit and the filtered reconstructed video unit.

35. The method of claim 32, wherein the first set of metrics is based on the un-filtered reconstructed video unit.

36. The method of claim 33, wherein the second set of metrics is based on the filtered reconstructed video unit.

37. A method of measuring an artifact comprising:

using either an un-filtered reconstructed video unit, wherein a video unit is at least one of video block and frame, or a filtered reconstructed video unit, wherein a video unit is at least one of video block and frame, with the following artifact metric implementation;

$\mathrm{ASNR}<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(y\right)=10*{\log }_{10}<br><br>\left(\frac{\mathrm{PKS}·<br><br>W_S·<br><br>W_P·<br><br>W_T}{F<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(y\right)}\right),\mathrm{where}$

 

y represents either a un-filtered reconstructed video unit or a filtered reconstructed video unit;

 

PKS is the peak value of the pixel squared;

 

W_S is a weight that affects the un-filtered reconstructed video unit, based on spatial factors;

 

W_P is a weight that affects the un-filtered reconstructed video unit, based on perceptual factors;

 

W_T is a weight that affects the un-filtered reconstructed video unit, based on temporal factors; and

 

F(y) is a function of y.

38. The method of claim 37, wherein ASNR(y) is DSNR(y) if F(y) is

Norm_Factor(MSDS(y));

and

Norm_Factor involves taking either an absolute value or a squaring.

39. The method of claim 38, wherein the measuring of the artifact further comprises:

emphasizing or de-emphasizing a blockiness artifact through a combination of values W_S, W_p, or W_T.

40. A method of measuring an artifact comprising:

using either an unfiltered reconstructed video unit, wherein a video unit is at least one of video block and frame, or a filtered reconstructed video unit, wherein a video unit is at least one of video block and frame, in conjunction with an original video unit with the following artifact metric implementation;

$\mathrm{ASNR}<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(x,y\right)=10*{\log }_{10}<br><br>\left(\frac{\mathrm{PKS}·<br><br>W_S·<br><br>W_P·<br><br>W_T}{F<br><br>\text{\hspace{1em}<br><br>}<br><br>\left(x,y\right)}\right),\mathrm{where}$

 

y represents either a un-filtered reconstructed video unit or a filtered reconstructed video unit;

 

x represents the original video unit, wherein a video unit is at least one of video block and frame;

 

PKS is the peak value of the pixel squared;

 

W_S is a weight that affects the filtered reconstructed video unit, based on spatial factors;

 

W_P is a weight that affects the filtered reconstructed video unit, based on perceptual factors;

 

W_T is a weight that affects the filtered reconstructed video unit, based on temporal factors; and

 

F(x,y) is a function of x and y.

41. The method of claim 40, wherein ASNR(x,y) is DSNR(x,y) if

F(x,y) is MSDS_error(x,y);

MSDS_error(x,y)=Norm_Factor(MSDS(x)−

MSDS(y));

MSDS(x) is the Mean Square Slope of Differences of (x);

MSDS(y) is the Mean Square Slope of Differences of (y); and

Norm_Factor involves taking either an absolute value or a squaring.

42. The method of claim 41, wherein the measuring of the artifact further comprises:

emphasizing or de-emphasizing a blockiness artifact through a combination of values W_S, W_P, or W_T.