Method and apparatus for sparsity-based de-artifact filtering for video encoding and decoding

• US 9,723,330 B2
• Filed: 11/18/2009
• Issued: 08/01/2017
• Est. Priority Date: 11/25/2008
• Status: Active Grant

• Current Assignee: InterDigital Madison Patent Holdings (InterDigital, Inc.)
• Original Assignee: Thomson Licensing Dtv
• Inventors: Yin, Peng, Sole, Joel, Zheng, Yunfei, Xu, Jun
• Primary Examiner(s): Czekaj, Dave
• Assistant Examiner(s): ABIMBOLA, KEHINDE
• Application Number:
  US12/998,721
• Publication Number:
  US 20110222597A1
• Time in Patent Office: 2,813 Days
• Field of Search: 37524029
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/157 Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
• CPC Class Codes: H04N 19/18 the unit being a set of transform coefficients
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness

Abstract

Methods and apparatus are provided for sparsity-based de-artifact filtering for video encoding and decoding. An apparatus includes an encoder (400) for encoding at least a portion of an image by grouping regions within the portion based on a grouping metric, transforming the grouped regions, adaptively performing de-artifact filtering on the transformed regions using a de-artifacting filter (413) included in the encoder, inverse transforming the de-artifacted regions to create replacement regions, and restoring the replacement regions to positions with the image from which the regions were taken prior to the grouping.

Claims

1. An apparatus, comprising:

an encoder encoding at least a portion of an image using de-artifact filtering, said encoder comprising a de-artifacting filter that performs the de-artifact filtering by grouping regions within the portion based on dimensions and characteristics of the region or portion, transforming the grouped regions, adaptively performing the de-artifact filtering on the transformed regions, inverse transforming the de-artifacted regions to create replacement regions, and restoring the replacement regions to positions with the image from which the regions were taken prior to the grouping, wherein multiple estimates of the same pixel are obtained from overlapping regions and, for each pixel, adaptive sparsity-based filtering is used to fuse multiple estimates from redundant representations of the overlapping regions, and wherein a type and a strength of the de-artifact filtering is adaptively selected responsive to characteristics of at least one of quantization noise statistics, coding modes and motion information, local coding conditions, and compression requirements.

2. The apparatus of claim 1, wherein the de-artifacting filter is useable in any of an in-loop configuration and an out-of-loop configuration.

3. The apparatus of claim 2, wherein the de-artifacting filter replaces a deblocking filter when said de-artifacting filter is applied the in-loop configuration.

4. The apparatus of claim 2, wherein the encoder includes a deblocking filter, and said de-artifacting filter is configured for use subsequent to the deblocking filter.

5. The apparatus of claim 1, wherein the de-artifacting filter is a Wiener filter.

6. The apparatus of claim 1, wherein the replacement regions are combined using weights prior to restoring.

7. The apparatus of claim 6, wherein the grouped regions are transformed to obtain transform sparsity values there for, and the weights are determined responsive to the transform sparsity values.

8. A method performed in a video encoder, comprising:

encoding at least a portion of an image using de-artifact filtering,

wherein said de-artifact filtering comprises;

grouping regions within the portion based on dimensions and characteristics of the region or portion;

transforming the grouped regions;

adaptively performing de-artifact filtering on the transformed regions using a de-artifacting filter;

inverse transforming the de-artifacted regions to create replacement regions; and

restoring the replacement regions to positions with the image from which the regions were taken prior to the grouping, wherein multiple estimates of the same pixel are obtained from overlapping regions and, for each pixel, adaptive sparsity-based filtering is used to fuse multiple estimates from redundant representations of the overlapping regions, and,

wherein a type and a strength of the de-artifact filtering is adaptively selected responsive to characteristics of at least one of quantization noise statistics, coding modes and motion information, local coding conditions, and compression requirements.

9. The method of claim 8, further comprising configuring the de-artifacting filter for use in any of an in-loop configuration and an out-of-loop configuration.

10. The method of claim 9, further comprising replacing a deblocking filter with the de-artifacting filter when the de-artifacting filter is applied the in-loop configuration.

11. The method of claim 9, wherein the encoder includes a deblocking filter, and the method further comprises configuring the de-artifacting filter for use subsequent to the deblocking filter.

12. The method of claim 8, further comprising combining the replacement regions using weights prior to said restoring step.

13. The method of claim 12, wherein the grouped regions are transformed to obtain transform sparsity values there for, and the weights are determined responsive to the transform sparsity values.

14. The method of claim 8, further comprising applying a secondary filter before or after the de-artifacting filter.

15. An apparatus, comprising:

a decoder decoding at least a portion of an image using de-artifact filtering, said decoder comprising a de-artifact filter for performing the de-artifact filtering by grouping regions within the portion based on dimensions and characteristics of the region or portion, transforming the grouped regions, adaptively performing the de-artifact filtering on the transformed regions, inverse transforming the de-artifacted regions to create replacement regions, and restoring the replacement regions to positions with the image from which the regions were taken prior to the grouping, wherein multiple estimates of the same pixel are obtained from overlapping regions and, for each pixel, adaptive sparsity-based filtering is used to fuse multiple estimates from redundant representations of the overlapping regions, and,

wherein a type and a strength of the de-artifact filtering is adaptively selected responsive to characteristics of at least one of quantization noise statistics, coding modes and motion information, local coding conditions, and compression requirements.

16. The apparatus of claim 15, wherein the de-artifacting filter is useable in any of an in-loop configuration and an out-of-loop configuration.

17. The apparatus of claim 16, wherein the de-artifacting filter replaces a deblocking filter when said de-artifacting filter is applied the in-loop configuration.

18. The apparatus of claim 16, wherein the decoder includes a deblocking filter, and said de-artifacting filter is configured for use subsequent to the deblocking filter.

19. The apparatus of claim 15, wherein the de-artifacting filter is a Wiener filter.

20. The apparatus of claim 15, wherein the replacement regions are combined using weights prior to restoring.

21. The apparatus of claim 20, wherein the grouped regions are transformed to obtain transform sparsity values there for, and the weights are determined responsive to the transform sparsity values.

22. A method performed in a video decoder, comprising:

decoding at least a portion of an image using de-artifact filtering,

wherein said de-artifact filtering comprises;

grouping regions within the portion based on dimensions and characteristics of the region or portion;

transforming the grouped regions;

adaptively performing the de-artifact filtering on the transformed regions using a de-artifacting filter;

inverse transforming the de-artifacted regions to create replacement regions; and

restoring the replacement regions to positions with the image from which the regions were taken prior to the grouping, wherein multiple estimates of the same pixel are obtained from overlapping regions and, for each pixel, adaptive sparsity-based filtering is used to fuse multiple estimates from redundant representations of the overlapping regions, and,

wherein a type and a strength of the de-artifact filtering is adaptively selected responsive to characteristics of at least one of quantization noise statistics, coding modes and motion information, local coding conditions, and compression requirements.

23. The method of claim 22, further comprising configuring the de-artifacting filter for use in any of an in-loop configuration and an out-of-loop configuration.

24. The method of claim 23, further comprising replacing a deblocking filter with the de-artifacting filter when the de-artifacting filter is applied the in-loop configuration.

25. The method of claim 23, wherein the decoder includes a deblocking filter, and the method further comprises configuring the de-artifacting filter for use subsequent to the deblocking filter.

26. The method of claim 22, wherein the de-artifacting filter is a Wiener filter.

27. The method of claim 22, further comprising combining the replacement regions using weights prior to said restoring step.

28. The method of claim 27, wherein the grouped regions are transformed to obtain transform sparsity values there for, and the weights are determined responsive to the transform sparsity values.

29. A non-transitory computer-readable storage media having video signal data encoded thereupon executable by a computer for performing a method, the method comprising:

encoding at least a portion of an image by an encoder using de-artifact filtering, said encoder comprising a de-artifacting filter that performs the de-artifact filtering by grouping regions within the portion based on dimensions and characteristics of the region or portion, transforming the grouped regions, adaptively performing de-artifact filtering on the transformed regions, inverse transforming the de-artifacted regions to create replacement regions, and restoring the replacement regions to positions with the image from which the regions were taken prior to the grouping, wherein multiple estimates of the same pixel are obtained from overlapping regions and, for each pixel, adaptive sparsity-based filtering is used to fuse multiple estimates from redundant representations of the overlapping regions, and,

wherein a type and a strength of the de-artifact filtering is adaptively selected responsive to characteristics of at least one of quantization noise statistics, coding modes and motion information, local coding conditions, and compression requirements.