Downsampling process for linear model prediction mode

• US 10,455,249 B2
• Filed: 03/17/2016
• Issued: 10/22/2019
• Est. Priority Date: 03/20/2015
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Zhang, Li, Chen, Jianle, Karczewicz, Marta
• Primary Examiner(s): Hasan, Mainul
• Application Number:
  US15/073,171
• Publication Number:
  US 20160277762A1
• Time in Patent Office: 1,314 Days
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/593 involving spatial prediction techniques

Abstract

Example techniques related to linear model (LM) prediction decoding or encoding are described. A video decoder or video encoder determines which filter to apply from a set of filters to downsample samples of a luma block and generate a predictive block for a corresponding chroma block based on characteristics of the chroma block.

Claims

1. A method of linear model (LM) prediction decoding video data, the method comprising:

determining, with a video decoder, a luma block of the video data that corresponds to a chroma block of the video data that is being decoded;

determining, with the video decoder, a filter from a set of filters to apply to luma samples that are internal to the luma block based on a characteristic of the chroma block, wherein determining the filter to apply comprises;

determining to apply a first filter to luma samples that are internal to the luma block that correspond to chroma samples of the chroma block that are located at or near a boundary, wherein the boundary comprises a boundary of one of a picture, a slice, a coding unit (CU), a prediction unit (PU), or a transform unit (TU); and

determining to apply a second filter, different from the first filter, to luma samples that are internal to the luma block that correspond to chroma samples of the chroma block that are not located at or near the boundary;

downsampling, with the video decoder, the luma samples that are internal to the luma block based on the determined filter to generate a downsampled luma block;

determining, with the video decoder, parameters based on one or more neighboring luma blocks that neighbor the luma block and one or more neighboring chroma blocks that neighbor the chroma block;

determining, with the video decoder, a predictive block based on the downsampled luma block and the determined parameters; and

reconstructing, with the video decoder, the chroma block based on the predictive block and a residual block.

2. The method of claim 1, wherein the boundary comprises one of a left picture boundary, a left slice boundary, a left CU boundary, a left PU boundary, or a left TU boundary.

3. The method of claim 1, wherein the first filter comprises a two-tap filter and the second filter comprises a six-tap filter.

4. The method of claim 1, wherein determining parameters based on one or more neighboring luma blocks that neighbor the luma block and one or more neighboring chroma blocks that neighbor the chroma block comprises:

downsampling the one or more neighboring luma blocks that neighbor the luma block; and

determining a first parameter and second parameter based on the downsampled one or more neighboring luma blocks and the one or more neighboring chroma blocks that neighbor the chroma block, and

wherein determining the predictive block comprises determining the predictive block based on the downsampled luma block and the first parameter and the second parameter.

5. The method of claim 1, wherein the luma block is a first luma block, and the chroma block is a first chroma block, the method further comprising:

in one instance of a second chroma block, different than the first chroma block, being of a first type, determining to apply a third filter to a second luma block, different than the first luma block, based on the second chroma block being of the first type; and

in another instance of the second chroma block being of a second type, determining to apply a fourth filter to the second luma block based on the second chroma block being of the second type.

6. The method of claim 1, wherein one or more luma samples of the luma block are outside one or more of the picture, the CU, the PU, or the TU, the method further comprising:

padding luma sample values for the one or more luma samples that are outside with padding values.

7. The method of claim 1, wherein the luma block comprises a luma residual block, and wherein the chroma block comprises a chroma residual block.

33. The method of claim 1, wherein determining to apply a first filter to luma samples that are internal to the luma block comprises determining to apply a first filter to luma samples that are of the luma block.

34. The method of claim 1, wherein determining to apply a first filter to luma samples that are internal to the luma block that correspond to chroma samples of the block that are located at or near a boundary comprises determining to apply a first filter to luma samples that are internal to the luma block that correspond to chroma samples of the block that are closest to the boundary including chroma samples that are directly on the boundary.

35. The method of claim 1, wherein determining to apply a second filter, different from the first filter, to luma samples that are internal to the luma block that correspond to chroma samples of the block that are not located at or near the boundary comprises determining to apply a second filter, different from the first filter, to luma samples that are internal to the luma block that correspond to chroma samples of the block that are not closest to the boundary including chroma samples that are not directly on the boundary.

8. A method of linear model (LM) prediction encoding video data, the method comprising:

determining, with a video encoder, a luma block of the video data that corresponds to a chroma block of the video data that is being encoded;

determining, with the video encoder, a filter from a set of filters to apply to luma samples that are internal to the luma block based on a characteristic of the chroma block, wherein determining the filter to apply comprises;

determining to apply a first filter to luma samples that are internal to the luma block that correspond to chroma samples of the chroma block that are located at or near a boundary, wherein the boundary comprises a boundary of one of a picture, a slice, a coding unit (CU), a prediction unit (PU), or a transform unit (TU); and

determining to apply a second filter, different from the first filter, to luma samples that are internal to the luma block that correspond to chroma samples of the chroma block that are not located at or near the boundary;

downsampling, with the video encoder, the luma samples that are internal to the luma block based on the determined filter to generate a downsampled luma block;

determining, with the video encoder, parameters based on one or more neighboring luma blocks that neighbor the luma block and one or more neighboring chroma blocks that neighbor the chroma block;

determining, with the video encoder, a predictive block based on the downsampled luma block and the determined parameters; and

generating, with the video encoder, a residual block based on the chroma block and the predictive block.

9. The method of claim 8, wherein the boundary comprises one of a left picture boundary, a left slice boundary, a left CU boundary, a left PU boundary, or a left TU boundary.

10. The method of claim 8, wherein the first filter comprises a two-tap filter and the second filter comprises a six-tap filter.

11. The method of claim 8, wherein determining parameters based on one or more neighboring luma blocks that neighbor the luma block and one or more neighboring chroma blocks that neighbor the chroma block comprises:

downsampling the one or more neighboring luma blocks that neighbor the luma block; and

determining a first parameter and second parameter based on the downsampled one or more neighboring luma blocks and the one or more neighboring chroma blocks that neighbor the chroma block, and

wherein determining the predictive block comprises determining the predictive block based on the downsampled luma block and the first parameter and second parameter.

12. The method of claim 8, wherein the luma block is a first luma block, and the chroma block is a first chroma block, the method further comprising:

in one instance of a second chroma block, different than the first chroma block, being of a first type, determining to apply a third filter to a second luma block, different than the first luma block, based on the second chroma block being of the first type; and

in another instance of the second chroma block being of a second type, determining to apply a fourth filter to the second luma block based on the second chroma block being of the second type.

13. The method of claim 8, wherein one or more luma samples of the luma block are outside one or more of the picture, the CU, the PU, or the TU, the method further comprising:

padding luma sample values for the one or more luma samples that are outside with padding values.

14. The method of claim 8, wherein the luma block comprises a luma residual block, and wherein the chroma block comprises a chroma residual block.

36. The method of claim 8, wherein determining to apply a first filter to luma samples that are internal to the luma block comprises determining to apply a first filter to luma samples that are of the luma block.

37. The method of claim 8,

wherein determining to apply a first filter to luma samples that are internal to the luma block that correspond to chroma samples of the block that are located at or near a boundary comprises determining to apply a first filter to luma samples that are internal to the luma block that correspond to chroma samples of the block that are closest to the boundary including chroma samples that are directly on the boundary, and

wherein determining to apply a second filter, different from the first filter, to luma samples that are internal to the luma block that correspond to chroma samples of the block that are not located at or near the boundary comprises determining to apply a second filter, different from the first filter, to luma samples that are internal to the luma block that correspond to chroma samples of the block that are not closest to the boundary including chroma samples that are not directly on the boundary.

15. A device for linear model (LM) prediction decoding video data, the device comprising:

a video data memory configured to store the video data; and

a video decoder comprising integrated circuitry, the video decoder configured to;

determine a luma block of the video data stored in the video data memory that corresponds to a chroma block of video data that is being decoded;

determine a filter from a set of filters to apply to luma samples that are internal to the luma block based on a characteristic of the chroma block, wherein to determine the filter, the video decoder is configured to;

determine to apply a first filter to luma samples that are internal to the luma block that correspond to chroma samples of the chroma block that are located at or near a boundary, wherein the boundary comprises a boundary of one of a picture, a slice, a coding unit (CU), a prediction unit (PU), or a transform unit (TU); and

determine to apply a second filter, different from the first filter, to luma samples that are internal to the luma block that correspond to chroma samples of the chroma block that are not located at or near the boundary;

downsample the luma samples that are internal to the luma block based on the determined filter to generate a downsampled luma block;

determine parameters based on one or more neighboring luma blocks that neighbor the luma block and one or more neighboring chroma blocks that neighbor the chroma block;

determine a predictive block based on the downsampled luma block and the determined parameters; and

reconstruct the chroma block based on the predictive block and a residual block.

16. The device of claim 15, wherein the boundary comprises one of a left picture boundary, a left slice boundary, a left CU boundary, a left PU boundary, or a left TU boundary.

17. The device of claim 15, wherein the first filter comprises a two-tap filter and the second filter comprises a six-tap filter.

18. The device of claim 15, wherein to determine parameters based on one or more neighboring luma blocks that neighbor the luma block and one or more neighboring chroma blocks that neighbor the chroma block, the video decoder is configured to:

downsample the one or more neighboring luma blocks that neighbor the luma block; and

determine a first parameter and second parameter based on the downsampled one or more neighboring luma blocks and the one or more neighboring chroma blocks that neighbor the chroma block, and

wherein to determine the predictive block, the video decoder is configured to determine the predictive block based on the downsampled luma block and the first parameter and the second parameter.

19. The device of claim 15, wherein the luma block is a first luma block, and the chroma block is a first chroma block, and wherein the video decoder is configured to:

in one instance of a second chroma block, different than the first chroma block, being of a first type, determine to apply a third filter to a second luma block, different than the first luma block, based on the second chroma block being of the first type; and

in another instance of the second chroma block being of a second type, determine to apply a fourth filter to the second luma block based on the second chroma block being of the second type.

20. The device of claim 15, wherein one or more luma samples of the luma block are outside one or more of the picture, the CU, the PU, or the TU, the video decoder is configured to:

pad luma sample values for the one or more luma samples that are outside with padding values.

21. The device of claim 15, wherein the luma block comprises a luma residual block, and wherein the chroma block comprises a chroma residual block.

38. The device of claim 15, wherein to determine to apply a first filter to luma samples that are internal to the luma block, the video decoder is configured to determine to apply a first filter to luma samples that are of the luma block.

39. The device of claim 15, wherein to determine to apply a first filter to luma samples that are internal to the luma block that correspond to chroma samples of the block that are located at or near a boundary, the video decoder is configured to determine to apply a first filter to luma samples that are internal to the luma block that correspond to chroma samples of the block that are closest to the boundary including chroma samples that are directly on the boundary.

40. The device of claim 15, wherein to determine to apply a second filter, different from the first filter, to luma samples that are internal to the luma block that correspond to chroma samples of the block that are not located at or near the boundary, the video decoder is configured to determine to apply a second filter, different from the first filter, to luma samples that are internal to the luma block that correspond to chroma samples of the block that are not closest to the boundary including chroma samples that are not directly on the boundary.

22. A device for linear model (LM) prediction encoding video data, the device comprising:

a video data memory configured to store the video data; and

a video encoder comprising integrated circuitry, the video encoder configured to;

determine a luma block of the video data stored in the video data memory that corresponds to a chroma block of the video data that is being encoded;

determine a filter from a set of filters to apply to luma samples that are internal to the luma block based on a characteristic of the chroma block, wherein to determine the filter, the video encoder is configured to;

determine to apply a first filter to luma samples that are internal to the luma block that correspond to chroma samples of the chroma block that are located at or near a boundary, wherein the boundary comprises a boundary of one of a picture, a slice, a coding unit (CU), a prediction unit (PU), or a transform unit (TU); and

determine to apply a second filter, different from the first filter, to luma samples that are internal to the luma block that correspond to chroma samples of the chroma block that are not located at or near the boundary;

downsample the luma samples that are internal to the luma block based on the determined filter to generate a downsampled luma block;

determine parameters based on one or more neighboring luma blocks that neighbor the luma block and one or more neighboring chroma blocks that neighbor the chroma block;

determine a predictive block based on the downsampled luma block and the determined parameters; and

generate a residual block based on the chroma block and the predictive block.

23. The device of claim 22, wherein the boundary comprises one of a left picture boundary, a left slice boundary, a left CU boundary, a left PU boundary, or a left TU boundary.

24. The device of claim 22, wherein the first filter comprises a two-tap filter and the second filter comprises a six-tap filter.

41. The device of claim 22, wherein to determine to apply a first filter to luma samples that are internal to the luma block, the video encoder is configured to determine to apply a first filter to luma samples that are of the luma block.

42. The device of claim 22,

wherein to determine to apply a first filter to luma samples that are internal to the luma block that correspond to chroma samples of the block that are located at or near a boundary, the video encoder is configured to determine to apply a first filter to luma samples that are internal to the luma block that correspond to chroma samples of the block that are closest to the boundary including chroma samples that are directly on the boundary, and

wherein to determine to apply a second filter, different from the first filter, to luma samples that are internal to the luma block that correspond to chroma samples of the block that are not located at or near the boundary, the video encoder is configured to determine to apply a second filter, different from the first filter, to luma samples that are internal to the luma block that correspond to chroma samples of the block that are not closest to the boundary including chroma samples that are not directly on the boundary.

25. A device for linear model (LM) prediction decoding video data, the device comprising:

means for determining a luma block of the video data that corresponds to a chroma block of the video data that is being decoded;

means for determining a filter from a set of filters to apply to luma samples that are internal to the luma block based on a characteristic of the chroma block, wherein the means for determining the filter to apply comprises;

means for determining to apply a first filter to luma samples that are internal to the luma block that correspond to chroma samples of the chroma block that are located at or near a boundary, wherein the boundary comprises a boundary of one of a picture, a slice, a coding unit (CU), a prediction unit (PU), or a transform unit (TU); and

means for determining to apply a second filter, different from the first filter, to luma samples that are internal to the luma block that correspond to chroma samples of the chroma block that are not located at or near the boundary;

means for downsampling the luma samples that are internal to the luma block based on the determined filter to generate a downsampled luma block;

means for determining parameters based on one or more neighboring luma blocks that neighbor the luma block and one or more neighboring chroma blocks that neighbor the chroma block;

means for determining a predictive block based on the downsampled luma block and the determined parameters; and

means for reconstructing the chroma block based on the predictive block and a residual block.

26. The device of claim 25, wherein the boundary comprises one of a left picture boundary, a left slice boundary, a left CU boundary, a left PU boundary, or a left TU boundary.

27. The device of claim 25, wherein the first filter comprises a two-tap filter and the second filter comprises a six-tap filter.

43. The device of claim 25, wherein the means for determining to apply a first filter to luma samples that are internal to the luma block comprises means for determining to apply a first filter that are of the luma block.

44. The device of claim 25, wherein the means for determining to apply a first filter to luma samples that are internal to the luma block that correspond to chroma samples of the block that are located at or near a boundary comprises means for determining to apply a first filter to luma samples that are internal to the luma block that correspond to chroma samples of the block that are closest to the boundary including chroma samples that are directly on the boundary.

45. The device of claim 25, wherein the means for determining to apply a second filter, different from the first filter, to luma samples that are internal to the luma block that correspond to chroma samples of the block that are not located at or near the boundary comprises means for determining to apply a second filter, different from the first filter, to luma samples that are internal to the luma block that correspond to chroma samples of the block that are not closest to the boundary including chroma samples that are not directly on the boundary.

28. A device for linear model (LM) prediction encoding video data, the device comprising:

means for determining a luma block of the video data that corresponds to a chroma block of the video data that is being encoded;

means for determining a filter from a set of filters to apply to luma samples that are internal to the luma block based on a characteristic of the chroma block, wherein the means for determining the filter to apply comprises;

means for determining to apply a first filter to luma samples that are internal to the luma block that correspond to chroma samples of the chroma block that are located at or near a boundary, wherein the boundary comprises a boundary of one of a picture, a slice, a coding unit (CU), a prediction unit (PU), or a transform unit (TU); and

means for determining to apply a second filter, different from the first filter, to luma samples that are internal to the luma block that correspond to chroma samples of the chroma block that are not located at or near the boundary;

means for downsampling the luma samples that are internal to the luma block based on the determined filter to generate a downsampled luma block;

means for determining parameters based on one or more neighboring luma blocks that neighbor the luma block and one or more neighboring chroma blocks that neighbor the chroma block;

means for determining a predictive block based on the downsampled luma block and the determined parameters; and

means for generating a residual block based on the chroma block and the predictive block.

29. The device of claim 28, wherein the boundary comprises one of a left picture boundary, a left slice boundary, a left CU boundary, a left PU boundary, or a left TU boundary.

30. The device of claim 28, wherein the first filter comprises a two-tap filter and the second filter comprises a six-tap filter.

31. A non-transitory computer-readable storage medium storing instructions that when executed cause one or more processors of a device for linear model (LM) prediction decoding video data to:

determine a luma block of the video data that corresponds to a chroma block of the video data that is being decoded;

determine a filter from a set of filters to apply to luma samples that are internal to the luma block based on a characteristic of the chroma block, wherein the instructions that cause the one or more processors to determine the filter to apply comprise instructions that cause the one or more processors to;

determine to apply a first filter to luma samples that are internal to the luma block that correspond to chroma samples of the chroma block that are located at or near a boundary, wherein the boundary comprises a boundary of one of a picture, a slice, a coding unit (CU), a prediction unit (PU), or a transform unit (TU); and

determine to apply a second filter, different from the first filter, to luma samples that are internal to the luma block that correspond to chroma samples of the chroma block that are not located at or near the boundary;

downsample the luma samples that are internal to the luma block based on the determined filter to generate a downsampled luma block;

determine parameters based on one or more neighboring luma blocks that neighbor the luma block and one or more neighboring chroma blocks that neighbor the chroma block;

determine a predictive block based on the downsampled luma block and the determined parameters; and

reconstruct the chroma block based on the predictive block and a residual block.

46. The non-transitory computer-readable storage medium of claim 31, wherein the instructions that cause the one or more processors to determine to apply a first filter to luma samples that are internal to the luma block comprise instructions that cause the one or more processors to determine to apply a first filter that are of the luma block.

47. The non-transitory computer-readable storage medium of claim 31,

wherein the instructions that cause the one or more processors to determine to apply a first filter to luma samples that are internal to the luma block that correspond to chroma samples of the block that are located at or near a boundary comprise instructions that cause the one or more processors to determine to apply a first filter to luma samples that are internal to the luma block that correspond to chroma samples of the block that are closest to the boundary including chroma samples that are directly on the boundary, and

wherein the instructions that cause the one or more processors to determine to apply a second filter, different from the first filter, to luma samples that are internal to the luma block that correspond to chroma samples of the block that are not located at or near the boundary comprise instructions that cause the one or more processors to determine to apply a second filter, different from the first filter, to luma samples that are internal to the luma block that correspond to chroma samples of the block that are not closest to the boundary including chroma samples that are not directly on the boundary.

32. A non-transitory computer-readable storage medium storing instructions that when executed cause one or more processors of a device for linear model (LM) prediction encoding video data to:

determine a luma block of the video data that corresponds to a chroma block of the video data that is being encoded;

determine a filter from a set of filters to apply to luma samples that are internal to the luma block based on a characteristic of the chroma block, wherein the instructions that cause the one or more processors to determine the filter to apply comprise instructions that cause the one or more processors to;

determine to apply a first filter to luma samples that are internal to the luma block that correspond to chroma samples of the chroma block that are located at or near a boundary, wherein the boundary comprises a boundary of one of a picture, a slice, a coding unit (CU), a prediction unit (PU), or a transform unit (TU); and

determine to apply a second filter, different from the first filter, to luma samples that are internal to the luma block that correspond to chroma samples of the chroma block that are not located at or near the boundary;

downsample the luma samples that are internal to the luma block based on the determined filter to generate a downsampled luma block;

determine parameters based on one or more neighboring luma blocks that neighbor the luma block and one or more neighboring chroma blocks that neighbor the chroma block;

determine a predictive block based on the downsampled luma block and the determined parameters; and

generate a residual block based on the chroma block and the predictive block.