Motion vector prediction for affine motion models in video coding

• US 10,448,010 B2
• Filed: 10/04/2017
• Issued: 10/15/2019
• Est. Priority Date: 10/05/2016
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Chen, Yi-Wen, Chien, Wei-Jung, Zhang, Li, Sun, Yu-Chen, Chen, Jianle, Karczewicz, Marta
• Primary Examiner(s): Mikeska, Neil R
• Application Number:
  US15/725,052
• Publication Number:
  US 20180098063A1
• Time in Patent Office: 741 Days
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/52 by predictive encoding
• CPC Class Codes: H04N 19/537 Motion estimation other than block-based

Abstract

A video decoder selects a source affine block. The source affine block is an affine-coded block that spatially neighbors a current block. Additionally, the video decoder extrapolates motion vectors of control points of the source affine block to determine motion vector predictors for control points of the current block. The video decoder inserts, into an affine motion vector predictor (MVP) set candidate list, an affine MVP set that includes the motion vector predictors for the control points of the current block. The video decoder also determines, based on an index signaled in a bitstream, a selected affine MVP set in the affine MVP set candidate list. The video decoder obtains, from the bitstream, motion vector differences (MVDs) that indicate differences between motion vectors of the control points of the current block and motion vector predictors in the selected affine MVP set.

Claims

1. A method of decoding video data, the method comprising:

selecting a source affine block, the source affine block being an affine-coded block that spatially neighbors a current block;

extrapolating motion vectors of a plurality of control points of the source affine block to determine motion vector predictors for a plurality of control points of the current block, wherein the control points of the source affine block include a first control point at a top-left of the source affine block and a second control point at a top-right of the source affine block, and the motion vectors of the first and second control points of the source affine block are used for affine prediction of the source affine block;

inserting, into an affine motion vector predictor (MVP) set candidate list, an affine MVP set that includes the motion vector predictors for the control points of the current block;

determining, based on an index signaled in a bitstream, a selected affine MVP set in the affine MVP set candidate list;

obtaining, from the bitstream, motion vector differences (MVDs) that indicate differences between motion vectors of the control points of the current block and motion vector predictors in the selected affine MVP set;

determining, based on the motion vector predictors included in the selected affine MVP set and the MVDs, motion vectors of the control points of the current block;

generating, based on the motion vectors of the control points of the current block, a predictive block; and

reconstructing the current block based on residual data and the predictive block.

2. The method of claim 1, wherein the control points of the current block include a first control point of the current block and a second control point of the current block, the method further comprising:

determining a first motion vector predictor as a motion vector of a block adjacent to the first control point of the current block;

determining a second motion vector predictor as a motion vector of a block adjacent to the second control point of the current block; and

inserting, into the affine MVP set candidate list, an affine MVP set that includes the first motion vector predictor and the second motion vector predictor.

3. The method of claim 1, wherein the current block is adjacent to a plurality of neighbor blocks, and selecting the source affine block comprises:

determining that the source affine block is a first-occurring affine-coded block of the plurality of neighbor blocks visited in a predefined visiting order.

4. The method of claim 1, wherein the current block is adjacent to a plurality of neighbor blocks, and selecting the source affine block comprises:

determining that the source affine block is a first-occurring available affine-coded block of the plurality of neighbor blocks according to a plurality of predefined priority sets based on a predefined visiting order, wherein an affine-coded block is not considered available if the affine-coded block is not in one of the predefined priority sets.

5. The method of claim 4, wherein:

the plurality of predefined priority sets includes a first priority set and a second priority set, and

the first priority set and the second priority set are defined such that, for each respective neighbor block of the plurality of neighbor blocks;

the respective neighbor block is in the first priority set if the respective neighbor block has a List X reference picture that is the same as a List X reference picture of the current block, X being 0 or 1; and

the respective neighbor block is in the second priority set if the respective neighbor block has a List Y reference picture that is the same as the List X reference picture of the current block, Y being equal to 1-X.

6. The method of claim 4, wherein:

the plurality of predefined priority sets includes a first priority set and a second priority set, and

the first priority set and the second priority set are defined such that, for each respective neighbor block of the plurality of neighbor blocks;

the respective neighbor block is in the first priority set if the respective neighbor block has a List X reference picture different from a List X reference picture of the current block, X being 0 or 1; and

the respective neighbor block is in the second priority set if the respective neighbor block has a List Y reference picture that is different from the List X reference picture of the current block, Y being equal to 1-X.

7. The method of claim 4, wherein:

the plurality of predefined priority sets includes a first priority set and a second priority set, and

the first priority set and the second priority set are defined such that, for each respective neighbor block of the plurality of neighbor blocks;

the respective neighbor block is in the first priority set if the respective neighbor block is coded in an affine inter mode; and

the respective neighbor block is in the second priority set if the respective neighbor block is coded in an affine merge mode.

8. The method of claim 1, wherein the affine source block is a first affine source block, the method further comprising:

selecting a second source affine block, the second source affine block being a different affine-coded block that spatially neighbors the current block;

extrapolating motion vectors of control points of the second source affine block to determine second motion vector predictors for the control points of the current block; and

inserting a second affine MVP set into the affine MVP set candidate list, the second affine MVP set including the second motion vector predictors for the control points of the current block.

9. A method of encoding video data, the method comprising:

selecting a source affine block, the source affine block being an affine-coded block that spatially neighbors a current block;

extrapolating motion vectors of a plurality of control points of the source affine block to determine motion vector predictors for a plurality of control points of the current block wherein the control points of the source affine block include a first control point at a top-left of the source affine block and a second control point at a top-right of the source affine block, and the motion vectors of the first and second control points of the source affine block are used for affine prediction of the source affine block;

inserting, into an affine motion vector predictor (MVP) set candidate list, an affine MVP set that includes the motion vector predictors for the control points of the current block;

selecting an affine MVP set in the affine MVP set candidate list;

signaling, in a bitstream, motion vector differences (MVDs) that indicate differences between motion vectors of the control points of the current block and motion vector predictors in the selected affine MVP set; and

signaling, in the bitstream, an index indicating a position in the affine MVP set candidate list of the selected affine MVP set.

10. The method of claim 9, wherein the control points of the current block include a first control point of the current block and a second control point of the current block, the method further comprising:

determining a first motion vector predictor as a motion vector of a block adjacent to the first control point of the current block;

determining a second motion vector predictor as a motion vector of a block adjacent to the second control point of the current block; and

inserting, into the affine MVP set candidate list, an affine MVP set that includes the first motion vector predictor and the second motion vector predictor.

11. The method of claim 9, wherein the current block is adjacent to a plurality of neighbor blocks, and selecting the source affine block comprises:

determining that the source affine block is a first-occurring affine-coded block of the plurality of neighbor blocks visited in a predefined visiting order.

12. The method of claim 9, wherein the current block is adjacent to a plurality of neighbor blocks, and selecting the source affine block comprises:

determining that the source affine block is a first-occurring available affine-coded block of the plurality of neighbor blocks according to a plurality of predefined priority sets based on a predefined visiting order, wherein an affine-coded block is not considered available if the affine-coded block is not in one of the predefined priority sets.

13. The method of claim 12, wherein:

the plurality of predefined priority sets includes a first priority set and a second priority set, and

the first priority set and the second priority set are defined such that, for each respective neighbor block of the plurality of neighbor blocks;

the respective neighbor block is in the first priority set if the respective neighbor block has a List X reference picture that is the same as a List X reference picture of the current block, X being 0 or 1; and

the respective neighbor block is in the second priority set if the respective neighbor block has a List Y reference picture that is the same as the List X reference picture of the current block, Y being equal to 1-X.

14. The method of claim 12, wherein:

the plurality of predefined priority sets includes a first priority set and a second priority set, and

the first priority set and the second priority set are defined such that, for each respective neighbor block of the plurality of neighbor blocks;

the respective neighbor block is in the first priority set if the respective neighbor block has a List X reference picture different from a List X reference picture of the current block, X being 0 or 1; and

the respective neighbor block is in the second priority set if the respective neighbor block has a List Y reference picture that is different from the List X reference picture of the current block, Y being equal to 1-X.

15. The method of claim 12, wherein:

the plurality of predefined priority sets includes a first priority set and a second priority set, and

the first priority set and the second priority set are defined such that, for each respective neighbor block of the plurality of neighbor blocks;

the respective neighbor block is in the first priority set if the respective neighbor block is coded in an affine inter mode; and

the respective neighbor block is in the second priority set if the respective neighbor block is coded in an affine merge mode.

16. The method of claim 9, wherein the affine source block is a first affine source block, the method further comprising:

selecting a second source affine block, the second source affine block being a different affine-coded block that spatially neighbors the current block;

extrapolating motion vectors of control points of the second source affine block to determine second motion vector predictors for the control points of the current block; and

inserting a second affine MVP set into the affine MVP set candidate list, the second affine MVP set including the second motion vector predictors for the control points of the current block.

17. A device for decoding video data, the device comprising:

a memory configured to store the video data; and

one or more processing circuits configured to;

select a source affine block, the source affine block being an affine-coded block that spatially neighbors a current block;

extrapolate motion vectors of a plurality of control points of the source affine block to determine motion vector predictors for a plurality of control points of the current block, wherein the control points of the source affine block include a first control point at a top-left of the source affine block and a second control point at a top-right of the source affine block, and the motion vectors of the first and second control points of the source affine block are used for affine prediction of the source affine block;

insert, into an affine motion vector predictor (MVP) set candidate list, an affine MVP set that includes the motion vector predictors for the control points of the current block;

determine, based on an index signaled in a bitstream, a selected affine MVP set in the affine MVP set candidate list;

obtain, from the bitstream, motion vector differences (MVDs) that indicate differences between motion vectors of the control points of the current block and motion vector predictors in the selected affine MVP set; and

determine, based on the motion vector predictors included in the selected affine MVP set and the MVDs, motion vectors of the control points of the current block;

generate, based on the motion vectors of the control points of the current block, a predictive block; and

reconstruct the current block based on residual data and the predictive block.

18. The device of claim 17, wherein the control points of the current block include a first control point of the current block and a second control point of the current block, the one or more processing circuits further configured to:

determine a first motion vector predictor as a motion vector of a block adjacent to the first control point of the current block;

determine a second motion vector predictor as a motion vector of a block adjacent to the second control point of the current block; and

insert, into the affine MVP set candidate list, an affine MVP set that includes the first motion vector predictor and the second motion vector predictor.

19. The device of claim 17, wherein the current block is adjacent to a plurality of neighbor blocks, and the one or more processing circuits are configured such that, as part of selecting the source affine block, the one or more processing circuits:

determine that the source affine block is a first-occurring affine-coded block of the plurality of neighbor blocks visited in a predefined visiting order.

20. The device of claim 17, wherein the current block is adjacent to a plurality of neighbor blocks, and the one or more processing circuits are configured such that, as part of selecting the source affine block, the one or more processing circuits:

determine that the source affine block is a first-occurring available affine-coded block of the plurality of neighbor blocks according to a plurality of predefined priority sets based on a predefined visiting order, wherein an affine-coded block is not considered available if the affine-coded block is not in one of the predefined priority sets.

21. The device of claim 20, wherein:

the plurality of predefined priority sets includes a first priority set and a second priority set, and

the first priority set and the second priority set are defined such that, for each respective neighbor block of the plurality of neighbor blocks;

the respective neighbor block is in the first priority set if the respective neighbor block has a List X reference picture that is the same as a List X reference picture of the current block, X being 0 or 1; and

the respective neighbor block is in the second priority set if the respective neighbor block has a List Y reference picture that is the same as the List X reference picture of the current block, Y being equal to 1-X.

22. The device of claim 20, wherein:

the plurality of predefined priority sets includes a first priority set and a second priority set, and

the first priority set and the second priority set are defined such that, for each respective neighbor block of the plurality of neighbor blocks;

the respective neighbor block is in the first priority set if the respective neighbor block has a List X reference picture different from a List X reference picture of the current block, X being 0 or 1; and

the respective neighbor block is in the second priority set if the respective neighbor block has a List Y reference picture that is different from the List X reference picture of the current block, Y being equal to 1-X.

23. The device of claim 20, wherein:

the plurality of predefined priority sets includes a first priority set and a second priority set, and

the first priority set and the second priority set are defined such that, for each respective neighbor block of the plurality of neighbor blocks;

the respective neighbor block is in the first priority set if the respective neighbor block is coded in an affine inter mode; and

the respective neighbor block is in the second priority set if the respective neighbor block is coded in an affine merge mode.

24. The device of claim 17, wherein the affine source block is a first affine source block, the one or more processing circuits further configured to:

select a second source affine block, the second source affine block being a different affine-coded block that spatially neighbors the current block;

extrapolate motion vectors of control points of the second source affine block to determine second motion vector predictors for the control points of the current block; and

insert a second affine MVP set into the affine MVP set candidate list, the second affine MVP set including the second motion vector predictors for the control points of the current block.

25. A device for encoding video data, the device comprising:

a memory configured to store the video data; and

one or more processing circuits configured to;

select a source affine block, the source affine block being an affine-coded block that spatially neighbors a current block;

extrapolate motion vectors of a plurality of control points of the source affine block to determine motion vector predictors for a plurality of control points of the current block, wherein the control points of the source affine block include a first control point at a top-left of the source affine block and a second control point at a top-right of the source affine block, and the motion vectors of the first and second control points of the source affine block are used for affine prediction of the source affine block;

insert, into an affine motion vector predictor (MVP) set candidate list, an affine MVP set that includes the motion vector predictors for the control points of the current block;

select an affine MVP set in the affine MVP set candidate list;

signal, in a bitstream, motion vector differences (MVDs) that indicate differences between motion vectors of the control points of the current block and motion vector predictors in the selected affine MVP set; and

signal, in the bitstream, an index indicating a position in the affine MVP set candidate list of the selected affine MVP set.

26. The device of claim 25, wherein the control points of the current block include a first control point of the current block and a second control point of the current block, the one or more processing circuits further configured to:

determine a first motion vector predictor as a motion vector of a block adjacent to the first control point of the current block;

determine a second motion vector predictor as a motion vector of a block adjacent to the second control point of the current block; and

insert, into the affine MVP set candidate list, an affine MVP set that includes the first motion vector predictor and the second motion vector predictor.

27. The device of claim 25, wherein the current block is adjacent to a plurality of neighbor blocks, and the one or more processing circuits are configured such that, as part of selecting the source affine block, the one or more processing circuits:

determine that the source affine block is a first-occurring affine-coded block of the plurality of neighbor blocks visited in a predefined visiting order.

28. The device of claim 25, wherein the current block is adjacent to a plurality of neighbor blocks, and the one or more processing circuits are configured such that, as part of selecting the source affine block, the one or more processing circuits:

determine that the source affine block is a first-occurring available affine-coded block of the plurality of neighbor blocks according to a plurality of predefined priority sets based on a predefined visiting order, wherein an affine-coded block is not considered available if the affine-coded block is not in one of the predefined priority sets.

29. The device of claim 28, wherein:

the plurality of predefined priority sets includes a first priority set and a second priority set, and

the first priority set and the second priority set are defined such that, for each respective neighbor block of the plurality of neighbor blocks;

the respective neighbor block is in the first priority set if the respective neighbor block has a List X reference picture that is the same as a List X reference picture of the current block, X being 0 or 1; and

the respective neighbor block is in the second priority set if the respective neighbor block has a List Y reference picture that is the same as the List X reference picture of the current block, Y being equal to 1-X.

30. The device of claim 28, wherein:

the plurality of predefined priority sets includes a first priority set and a second priority set, and

the first priority set and the second priority set are defined such that, for each respective neighbor block of the plurality of neighbor blocks;

the respective neighbor block is in the first priority set if the respective neighbor block has a List X reference picture different from a List X reference picture of the current block, X being 0 or 1; and

the respective neighbor block is in the second priority set if the respective neighbor block has a List Y reference picture that is different from the List X reference picture of the current block, Y being equal to 1-X.

31. The device of claim 28, wherein:

the plurality of predefined priority sets includes a first priority set and a second priority set, and

the first priority set and the second priority set are defined such that, for each respective neighbor block of the plurality of neighbor blocks;

the respective neighbor block is in the first priority set if the respective neighbor block is coded in an affine inter mode; and

the respective neighbor block is in the second priority set if the respective neighbor block is coded in an affine merge mode.

32. The device of claim 25, wherein the affine source block is a first affine source block, the one or more processing circuits further configured to:

select a second source affine block, the second source affine block being a different affine-coded block that spatially neighbors the current block;

extrapolate motion vectors of control points of the second source affine block to determine second motion vector predictors for the control points of the current block; and

insert a second affine MVP set into the affine MVP set candidate list, the second affine MVP set including the second motion vector predictors for the control points of the current block.

33. A device for decoding video data, the device comprising:

means for selecting a source affine block, the source affine block being an affine-coded block that spatially neighbors a current block;

means for extrapolating motion vectors of a plurality of control points of the source affine block to determine motion vector predictors for a plurality of control points of the current block, wherein the control points of the source affine block include a first control point at a top-left of the source affine block and a second control point at a top-right of the source affine block, and the motion vectors of the first and second control points of the source affine block are used for affine prediction of the source affine block;

means for inserting, into an affine motion vector predictor (MVP) set candidate list, an affine MVP set that includes the motion vector predictors for the control points of the current block;

means for determining, based on an index signaled in a bitstream, a selected affine MVP set in the affine MVP set candidate list;

means for obtaining, from the bitstream, motion vector differences (MVDs) that indicate differences between motion vectors of the control points of the current block and motion vector predictors in the selected affine MVP set;

means for determining, based on the motion vector predictors included in the selected affine MVP set and the MVDs, motion vectors of the control points of the current block;

means for generating, based on the motion vectors of the control points of the current block, a predictive block; and

means for reconstructing the current block based on residual data and the predictive block.

34. A device for encoding video data, the device comprising:

means for selecting a source affine block, the source affine block being an affine-coded block that spatially neighbors a current block;

means for extrapolating motion vectors of a plurality of control points of the source affine block to determine motion vector predictors for a plurality of control points of the current block, wherein the control points of the source affine block include a first control point at a top-left of the source affine block and a second control point at a top-right of the source affine block, and the motion vectors of the first and second control points of the source affine block are used for affine prediction of the source affine block;

means for inserting, into an affine motion vector predictor (MVP) set candidate list, an affine MVP set that includes the motion vector predictors for the control points of the current block;

means for selecting an affine MVP set in the affine MVP set candidate list;

means for signaling, in a bitstream, motion vector differences (MVDs) that indicate differences between motion vectors of the control points of the current block and motion vector predictors in the selected affine MVP set; and

means for signaling, in the bitstream, an index indicating a position in the affine MVP set candidate list of the selected affine MVP set.

35. A computer-readable storage medium storing instructions that, when executed, cause one or more processing circuits of a device for video decoding to:

select a source affine block, the source affine block being an affine-coded block that spatially neighbors a current block;

extrapolate motion vectors of a plurality of control points of the source affine block to determine motion vector predictors for a plurality of control points of the current block, wherein the control points of the source affine block include a first control point at a top-left of the source affine block and a second control point at a top-right of the source affine block, and the motion vectors of the first and second control points of the source affine block are used for affine prediction of the source affine block;

insert, into an affine motion vector predictor (MVP) set candidate list, an affine MVP set that includes the motion vector predictors for the control points of the current block;

determine, based on an index signaled in a bitstream, a selected affine MVP set in the affine MVP set candidate list;

obtain, from the bitstream, motion vector differences (MVDs) that indicate differences between motion vectors of the control points of the current block and motion vector predictors in the selected affine MVP set;

determine, based on the motion vector predictors included in the selected affine MVP set and the MVDs, motion vectors of the control points of the current block;

generate, based on the motion vectors of the control points of the current block, a predictive block; and

reconstruct the current block based on residual data and the predictive block.

36. A computer-readable storage medium storing instructions that, when executed, cause one or more processing circuits of a device for encoding video data to:

select a source affine block, the source affine block being an affine-coded block that spatially neighbors a current block;

extrapolate motion vectors of a plurality of control points of the source affine block to determine motion vector predictors for a plurality of control points of the current block, wherein the control points of the source affine block include a first control point at a top-left of the source affine block and a second control point at a top-right of the source affine block, and the motion vectors of the first and second control points of the source affine block are used for affine prediction of the source affine block;

insert, into an affine motion vector predictor (MVP) set candidate list, an affine MVP set that includes the motion vector predictors for the control points of the current block;

select an affine MVP set in the affine MVP set candidate list;

signal, in a bitstream, motion vector differences (MVDs) that indicate differences between motion vectors of the control points of the current block and motion vector predictors in the selected affine MVP set; and

signal, in the bitstream, an index indicating a position in the affine MVP set candidate list of the selected affine MVP set.