Filter description signaling for multi-filter adaptive filtering

• US 9,819,966 B2
• Filed: 07/07/2011
• Issued: 11/14/2017
• Est. Priority Date: 09/01/2010
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Chong, In Suk, Karczewicz, Marta, Chien, Wei-Jung
• Primary Examiner(s): Hallenbeck-Huber, Jeremaiah C
• Assistant Examiner(s): Kir, Albert
• Application Number:
  US13/178,415
• Publication Number:
  US 20120051438A1
• Time in Patent Office: 2,322 Days
• Field of Search: 37524012, 37524025, 37524013, 37524016, 37524029
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/182 the unit being a pixel
• CPC Class Codes: H04N 19/42 characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/82 involving filtering within a prediction loop
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression

Abstract

Filtering is applied at an encoder, and filter information describing a set of filters is encoded in the bitstream to enable a decoder to identify the filtering that was applied at the encoder. The decoder receives encoded video data that includes the filter information, decodes the video data, and applies filtering based on the filtering information. The decoder applies the same filtering that was applied at the encoder. To potentially reduce the number of bits needed to reconstruct the set of filters and other filter information as well as to potentially improve decoder performance, filter description syntax describing the set of filters can be included in the bitstream. The filter description syntax may identify a number of filters in a set of filters, a maximum number of filters in a set of filters, and/or identify a shape of filters in a set of filters.

Claims

1. A method of decoding a bitstream of encoded video data, the method comprising:

receiving in the bitstream of encoded video data as part of a higher level coded unit, at a video decoder comprising one or more processors, a first syntax element, wherein a value of the first syntax element identifies a maximum number of filters allowed in a set of filters for a series of video blocks;

receiving, in the bitstream of encoded video data as part of a lower level coded unit, at the video decoder, a second syntax element, wherein a value of the second syntax element identifies a number of filters in the set of filters for the series of video blocks, wherein the number of filters in the set of filters is less than the maximum number of filters, wherein the higher level coded unit is hierarchically higher than the lower level coded unit;

reconstructing the set of filters based on the value of the first syntax element and the value of the second syntax element;

determining a mapping of ranges of an activity metric to filters in the set of filters based on the value of the first syntax element, wherein a number of ranges of the activity metric to be mapped to filters in the set of filters is a function of the maximum number of filters allowed in the set of filters as indicated by the value of the first syntax element;

generating a reconstructed block from a predictive block in a reference frame and a residual block;

determining a value of an activity metric for the reconstructed block;

selecting a filter from the set of filters based on the determined value of the activity metric and the mapping of the ranges of the activity metric to the filters in the set of filters;

applying the filter to samples of the reconstructed block of video as part of a loop filtering operation; and

outputting decoded video data comprising the filtered reconstructed block of video.

2. The method of claim 1, further comprising:

allocating a portion of memory to storing filters based on the value of the first syntax element.

3. The method of claim 1, wherein the video decoder implements a Quadtree-based Adaptive Loop Filter (QALF) scheme with multiple filters.

4. The method of claim 1, wherein the first syntax element is part of a header of a series of video blocks.

5. The method of claim 1, further comprising:

receiving third syntax element, wherein a value of the third syntax element identifies a shape of filters in the set of filters;

wherein reconstructing the set of filters is further based on the value of the third syntax element.

6. The method of claim 1, wherein the activity metric comprises a sum-modified Laplacian value.

7. The method of claim 6, wherein the sum-modified Laplacian value comprises var(i,j) defined approximately according to the following equation:

8. The method of claim 1, wherein the video decoder is implemented as a component of a video encoder.

36. The method of claim 1, further comprising:

receiving the encoded video data at a receiver of a wireless communication device;

storing the encoded video data in a memory of the wireless communication device; and

processing the encoded video data on one or more processors of the wireless communication device.

37. The method of claim 36, wherein the wireless communication device comprises a telephone handset and wherein receiving the video data at the receiver of the wireless communication device comprises demodulating, according to a wireless communication standard, a signal comprising the encoded video data.

40. The method of claim 1, wherein the higher level coded unit comprises one of a sequence parameter set or a picture parameter set.

41. The method of claim 1, wherein the lower level coded unit comprises one of a slice header or a largest coding unit.

9. A method of video encoding, the method comprising:

determining a maximum number of filters allowed in a set of filters for a series of video blocks;

determining a number of filters to be included in the set of filters for the series of video blocks;

determining a mapping of ranges of an activity metric to filters in the set of filters, wherein a number of ranges of the activity metric is a function of the maximum number of filters allowed in the set of filters as indicated by the value of the first syntax element;

generating for inclusion in a bitstream of encoded video data, as part of a higher level coded unit, a first syntax element, wherein a value of the first syntax element identifies the maximum number of filters;

generating for inclusion in the bitstream of encoded video data, as part of a lower level coded unit, a second syntax element, wherein a value of the second syntax element identifies the number of filters, wherein the number of filters in the set of filters is less than the maximum number of filters, wherein the higher level coded unit is hierarchically higher than the lower level coded unit; and

outputting the bitstream of encoded video data comprising the first syntax element, the second syntax element, the set of filters, and for each range of the ranges of the activity metric, information for mapping the range of the activity metric to a filter in the set of filters;

as part of a decoding loop;

generating a reconstructed block from a predictive block in a reference frame and a residual block;

determining a value of an activity metric for the reconstructed block;

selecting a filter from the set of filters based on the determined value of the activity metric and the mapping of the ranges of the activity metric to the filters in the set of filters;

applying the filter to samples of the reconstructed block of video.

10. The method of claim 9, wherein the video encoder implements a Quadtree-based Adaptive Loop Filter (QALF) scheme with multiple filters.

11. The method of claim 9, wherein the first syntax element is part of a header of a series of video blocks.

12. The method of claim 9, further comprising:

determining a shape of filters to be used for the set of filters;

generating a third syntax element identifying the shape of filters in the set of filters; and

including the third syntax element in the bitstream.

42. The method of claim 9, wherein the higher level coded unit comprises one of a sequence parameter set or a picture parameter set.

43. The method of claim 9, wherein the lower level coded unit comprises one of a slice header or a largest coding unit.

13. A video decoding device comprising:

a memory storing a bitstream of encoded video data;

a video decoder comprising one or more processors configured to;

receive the bitstream of encoded video data;

receive, in the bitstream of encoded video data as part of a higher level coded unit, a first syntax element, wherein a value of the first syntax element identifies a maximum number of filters allowed in a set of filters for a series of video blocks;

receive, in the bitstream of encoded video data as part of a lower level coded unit, a second syntax element, wherein a value of the second syntax element identifies a number of filters in the set of filters for the series of video blocks, wherein the number of filters in the set of filters is less than the maximum number of filters, wherein the higher level coded unit is hierarchically higher than the lower level coded unit;

reconstruct the set of filters based on the first syntax element and the second filter description;

determine a mapping of ranges of an activity metric to filters in the set of filters based on the syntax element, wherein a number of ranges of the activity metric to be mapped to filters in the set of filters is a function of the maximum number of filters allowed in the set of filters as indicated by the value of the first syntax element;

generate a reconstructed block from a predictive block in a reference frame and a residual block;

determine a value of an activity metric for the reconstructed block;

select a filter from the set of filters based on the determined value of the activity metric and the mapping of ranges of the activity metric to the filters in the set of filters;

apply the filter to samples of the reconstructed block of video as part of a loop filtering operation; and

output decoded video data comprising the filtered reconstructed block of video data.

14. The video decoding device of claim 13, wherein the one or more processors are further configured to allocate a portion of memory to storing filters based on the value of the first syntax element.

15. The video decoding device of claim 13, wherein the video decoding device implements a Quadtree-based Adaptive Loop Filter (QALF) scheme with multiple inputs.

16. The video decoding device of claim 13, wherein the first syntax element is part of a header of a series of video blocks.

17. The video decoding device of claim 13, wherein the one or more processors are further configured to:

receive third syntax element, wherein a value of the third syntax element identifies a shape of filters in the set of filters; and

reconstructing the set of filters based on the third syntax element.

18. The video decoding device of claim 13, wherein the activity metric comprises a sum-modified Laplacian value.

19. The video decoding device of claim 18, wherein the sum-modified Laplacian value comprises var(i,j) defined approximately according to the following equation:

38. The device of claim 13, wherein the device comprises a wireless communication device comprising a receiver configured to receive encoded video data.

39. The device of claim 38, wherein the wireless communication device comprises a telephone handset and wherein the receiver is configured to demodulate, according to a wireless communication standard, a signal comprising the encoded video data.

20. A video encoding device comprising:

a memory storing video data;

a video encoder comprising one or more processors configured to;

determine a maximum number of filters allowed in a set of filters for a series of video blocks;

determine a number of filters to be included in the set of filters for the series of video blocks;

determine a mapping of ranges of an activity metric to filters in the set of filters, wherein a number of ranges of the activity metric is a function of the maximum number of filters allowed in the set of filters as indicated by the value of the first syntax element;

generate for inclusion in a bitstream of encoded video data, as part of a higher level coded unit, a first syntax element, wherein a value of the first syntax element identifies the maximum number of filters;

generate for inclusion in a bitstream of encoded video data, as part of a lower level coded unit, a second syntax element, wherein a value of the second syntax element identifies the number of filters, wherein the number of filters in the set of filters is less than the maximum number of filters, wherein the higher level coded unit is hierarchically higher than the lower level coded unit; and

output the bitstream of encoded video data comprising the first syntax element, the second syntax element, the set of filters, and for each range of the ranges of the activity metric, information for mapping the range of the activity metric to a filter in the set of filters; and

as part of a decoding loop;

generate a reconstructed block from a predictive block in a reference frame and a residual block;

determine a value of an activity metric for the reconstructed block;

select a filter from the set of filters based on the determined value of the activity metric and the mapping of the ranges of the activity metric to the filters in the set of filters;

apply the filter to samples of the reconstructed block of video.

21. The video encoding device of claim 20, wherein the one or more processors are configured to implement a Quadtree-based Adaptive Loop Filter (QALF) scheme with multiple inputs.

22. The video encoding device of claim 20, wherein the first syntax element is part of a header of a series of video blocks.

23. The video encoding device of claim 20, wherein the one or more processors are further configured to:

determine a shape of filters to be used for the set of filters;

generate a third syntax element, wherein a value of the third syntax element identifies the shape of filters in the set of filters; and

including the third syntax element in the bitstream.

24. A non-transitory computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors of a device for decoding video data to:

receive a bitstream of encoded video data;

receive in the bitstream of encoded video data as part of a higher level coded unit, at a video decoder, first syntax element, wherein a value of the first syntax element identifies a maximum number of filters allowed in a set of filters for a series of video blocks;

receive, in the bitstream of encoded video data as part of a lower level coded unit, at the video decoder, second syntax element, wherein a value of the second syntax element identifies a number of filters in the set of filters for the series of video blocks, wherein the number of filters in the set of filters is less than the maximum number of filters, wherein the higher level coded unit is hierarchically higher than the lower level coded unit;

reconstruct the set of filters based on the first syntax element and the second syntax element;

determine a mapping of ranges of an activity metric to filters in the set of filters based on the first syntax element, wherein a number of ranges of the activity metric to be mapped to filters in the set of filters is a function of the maximum number of filters allowed in the set of filters as indicated by the value of the first syntax element;

generate a reconstructed block from a predictive block in a reference frame and a residual block;

determine a value of an activity metric for the reconstructed block;

select a filter from the set of filters based on the determined value of the activity metric and the mapping of the ranges of the activity metric to the filters in the set of filters;

apply the filter to samples of the reconstructed block of video as part of a loop filtering operation; and

output decoded video data comprising the filtered reconstructed block of video.

25. The non-transitory, computer-readable storage medium of claim 24, further comprising instructions that cause the one or more processors to allocate a portion of memory to storing filters based on the first syntax element.

26. The non-transitory, computer-readable storage medium of claim 24, wherein the device for video decoding video data implements a Quadtree-based Adaptive Loop Filter (QALF) scheme with multiple filters.

27. The non-transitory, computer-readable storage medium of claim 24, wherein the first syntax element is part of a header of a series of video blocks.

28. The non-transitory, computer-readable storage medium of claim 24, further comprising instructions that cause the one or more processors to:

receive a third syntax element, wherein a value of the third syntax element identifies a shape of filters in the set of filters;

wherein reconstructing the set of filters is further based on the third syntax element.

29. The non-transitory, computer-readable storage medium of claim 24, wherein the activity metric comprises a sum-modified Laplacian value.

30. The non-transitory, computer-readable storage medium of claim 29, wherein the sum-modified Laplacian value comprises var(i,j) defined approximately according to the following equation:

31. The non-transitory, computer-readable storage medium of claim 24, wherein the device for video decoding video data is implemented as a component of a video encoder.

32. A non-transitory, computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors of a device for encoding video data to:

determine a maximum number of filters allowed in a set of filters for a series of video blocks;

determine a number of filters to be included in the set of filters for the series of video blocks;

determine a mapping of ranges of an activity metric to filters in the set of filters, wherein a number of ranges of the activity metric is a function of the maximum number of filters allowed in the set of filters as indicated by the value of the first syntax element;

generate for inclusion in a bitstream of encoded video data, as part of a higher level coded unit, a first syntax element, wherein a value of the first syntax element identifies the maximum number of filters;

generate for inclusion in the bitstream of encoded video data, as part of a lower level coded unit, a second syntax element, wherein a value of the second syntax element identifies the number of filters, wherein the number of filters in the set of filters is less than the maximum number of filters, wherein the higher level coded unit is hierarchically higher than the lower level coded unit; and

output the bitstream of encoded video data comprising the first syntax element, the second syntax element, the set of filters, and for each range of the ranges of the activity metric, information for mapping the range of the activity metric to a filter in the set of filters; and

as part of a decoding loop;

generate a reconstructed block from a predictive block in a reference frame and a residual block;

determine a value of an activity metric for the reconstructed block;

select a filter from the set of filters based on the determined value of the activity metric and the mapping of the ranges of the activity metric to the filters in the set of filters;

apply the filter to samples of the reconstructed block of video.

33. The non-transitory, computer-readable storage medium of claim 32, wherein the device for encoding video data implements a Quadtree-based Adaptive Loop Filter (QALF) scheme with multiple filters.

34. The non-transitory, computer-readable storage medium of claim 32, wherein the first syntax element is part of a header of a series of video blocks.

35. The non-transitory, computer-readable storage medium of claim 32, further comprising instructions that cause the one or more processors to:

determine a shape of filters to be used for the set of filters;

generate a third syntax element, wherein a value of the third syntax element identifies the shape of filters in the set of filters; and

signal the third syntax element in the bitstream.