IMAGE PREDICTION METHOD AND RELATED DEVICE

• US 20190098312A1
• Filed: 11/21/2018
• Published: 03/28/2019
• Est. Priority Date: 05/24/2016
• Status: Abandoned Application

• Current Assignee: University of Science and Technology of China (Chinese Academy of Sciences), Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: University of Science and Technology of China (Chinese Academy of Sciences), Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: CHEN, Huanbang, YANG, Haitao, LI, Houqiang
• Application Number:
  US16/197,585
• Publication Number:
  US 20190098312A1
• Time in Patent Office: Days
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/184 the unit being bits, e.g. of the compressed video stream
• CPC Class Codes: H04N 19/513 Processing of motion vectors
• CPC Class Codes: H04N 19/52 by predictive encoding
• CPC Class Codes: H04N 19/54 using feature points or meshes

Abstract

An image prediction method and a related apparatus are disclosed. The method includes: parsing first bitstream information, used for indicating a motion information unit, to obtain motion information of each first pixel sample and predicted motion information of each second pixel sample; parsing second bitstream information, used for representing differential motion information of each second pixel sample, to obtain motion information of each second pixel sample; and obtaining a predictor of a current image block based on a motion model of the current image block, the motion information of the first pixel sample, and the motion information of the second pixel sample.

Claims

1. An image prediction method, wherein a current image block comprises at least one first pixel sample and at least one second pixel sample, and the method comprises:

parsing first bitstream information, wherein the first bitstream information indicates motion information units respectively corresponding to each first pixel sample and each second pixel sample;

obtaining motion information of each first pixel sample and predicted motion information of each second pixel sample based on the parsed first bitstream information, wherein the predicted motion information is predicted information of motion information;

parsing second bitstream information, wherein the second bitstream information is used to represent differential motion information of each second pixel sample, and the differential motion information is a difference between motion information and predicted motion information;

obtaining motion information of each second pixel sample based on the parsed second bitstream information and the corresponding predicted motion information of each second pixel sample; and

obtaining a predictor of the current image block based on a motion model of the current image block, the motion information of each first pixel sample, and the motion information of each second pixel sample.

2. The method according to claim 1, wherein the first bitstream information comprises an index, and the index is used to indicate the motion information units respectively corresponding to each first pixel sample and each second pixel sample.

3. The method according to claim 1, wherein the second bitstream information comprises a difference, and the difference is a motion vector difference between a motion vector and a predicted motion vector of any second pixel sample.

4. The method according to claim 1, wherein the obtaining motion information of each first pixel sample and predicted motion information of each second pixel sample based on the parsed first bitstream information comprises:

determining candidate motion information unit sets respectively corresponding to each first pixel sample and each second pixel sample, wherein any candidate motion information unit set comprises at least one motion information unit;

determining a combined motion information unit set of the current image block, wherein each motion information unit in the combined motion information unit set is selected from at least some motion information units in the candidate motion information unit sets respectively corresponding to each first pixel sample and each second pixel sample, and motion information of the motion information unit comprises a motion vector with a forward prediction direction and/or a motion vector with a backward prediction direction;

determining, from the combined motion information unit set based on the parsed first bitstream information, the motion information units respectively corresponding to each first pixel sample and each second pixel sample;

using motion information of the motion information unit corresponding to the first pixel sample as the motion information of the first pixel sample; and

using motion information of the motion information unit corresponding to the second pixel sample as the predicted motion information of the second pixel sample.

5. The method according to claim 4, wherein the determining a combined motion information unit set of the current image block comprises:

determining, from N candidate combined motion information unit sets, a combined motion information unit set comprising the motion information units respectively corresponding to each first pixel sample and each second pixel sample, wherein each motion information unit comprised in each of the N candidate combined motion information unit sets is selected from at least some motion information units that meet a constraint in the candidate motion information unit sets respectively corresponding to each first pixel sample and each second pixel sample, N is a positive integer, and the N candidate combined motion information unit sets are different from each other.

6. The method according to claim 5, wherein the N candidate combined motion information unit sets meet at least one of a first condition, a second condition, a third condition, a fourth condition, and a fifth condition, wherein

the first condition comprises:

a motion manner that is of the current image block and that is indicated by a motion information unit in any one of the N candidate combined motion information unit sets is non-translational motion;

the second condition comprises;

two motion information units in any one of the N candidate combined motion information unit sets correspond to a same prediction direction;

the third condition comprises;

two motion information units in any one of the N candidate combined motion information unit sets correspond to a same reference frame index;

the fourth condition comprises;

an absolute value of a difference between motion vector horizontal components of two motion information units in any one of the N candidate combined motion information unit sets is less than or equal to a horizontal component threshold, or an absolute value of a difference between motion vector horizontal components of one motion information unit in any one of the N candidate combined motion information unit sets and a pixel sample Z is less than or equal to a horizontal component threshold, wherein the pixel sample Z of the current image block is different from either of the first pixel sample and the second pixel sample; and

the fifth condition comprises;

an absolute value of a difference between motion vector vertical components of two motion information units in any one of the N candidate combined motion information unit sets is less than or equal to a vertical component threshold, or an absolute value of a difference between motion vector vertical components of one motion information unit in any one of the N candidate combined motion information unit sets and a pixel sample Z is less than or equal to a vertical component threshold, wherein the pixel sample Z of the current image block is different from either of the first pixel sample and the second pixel sample.

7. The method according to claim 1, wherein the obtaining motion information of each second pixel sample based on the parsed second bitstream information and the corresponding predicted motion information of each second pixel sample comprises:

obtaining the differential motion information of each second pixel sample based on the parsed second bitstream information; and

adding up the differential motion information of each second pixel sample and the corresponding predicted motion information, to obtain the motion information of each second pixel sample.

8. The method according to claim 1, wherein the motion model is a non-translational motion model, and specifically,

the non-translational motion model is an affine motion model in the following form:

9. The method according to claim 1, wherein the motion model is a non-translational motion model, and specifically,

the non-translational motion model is an affine motion model in the following form:

10. The method according to claim 1, wherein the at least one first pixel sample and the at least one second pixel sample comprise two pixel samples in a top-left pixel sample, a top-right pixel sample, a bottom-left pixel sample, and a central pixel sample a1 of the current image block, wherein

the top-left pixel sample of the current image block is a top-left vertex of the current image block or a pixel block that is at a top-left vertex of the current image block and that is comprised in the current image block;

the bottom-left pixel sample of the current image block is a bottom-left vertex of the current image block or a pixel block that is at a bottom-left vertex of the current image block and that is comprised in the current image block;

the top-right pixel sample of the current image block is a top-right vertex of the current image block or a pixel block that is at a top-right vertex of the current image block and that is comprised in the current image block; and

the central pixel sample a1 of the current image block is a central pixel of the current image block or a pixel block that is at a central pixel of the current image block and that is comprised in the current image block.

11. The method according to claim 10, wherein

a candidate motion information unit set corresponding to the top-left pixel sample of the current image block comprises motion information units of x1 pixel samples, wherein the x1 pixel samples comprise at least one pixel sample that is spatially adjacent to the top-left pixel sample of the current image block and/or at least one pixel sample that is temporally adjacent to the top-left pixel sample of the current image block, and x1 is a positive integer;

and

the x1 pixel samples comprise at least one of a pixel sample that is at a same location as the top-left pixel sample of the current image block and that is in a video frame temporally adjacent to a video frame to which the current image block belongs, a spatially adjacent pixel sample to the left of the current image block, a spatially adjacent pixel sample to the top-left of the current image block, and a spatially adjacent pixel sample to the top of the current image block.

12. The method according to claim 10, wherein

a candidate motion information unit set corresponding to the top-right pixel sample of the current image block comprises motion information units of x2 pixel samples, wherein the x2 pixel samples comprise at least one pixel sample that is spatially adjacent to the top-right pixel sample of the current image block and/or at least one pixel sample that is temporally adjacent to the top-right pixel sample of the current image block, and x2 is a positive integer;

and

the x2 pixel samples comprise at least one of a pixel sample that is at a same location as the top-right pixel sample of the current image block and that is in the video frame temporally adjacent to the video frame to which the current image block belongs, a spatially adjacent pixel sample to the right of the current image block, a spatially adjacent pixel sample to the top-right of the current image block, and a spatially adjacent pixel sample to the top of the current image block.

13. The method according to claim 10, wherein

a candidate motion information unit set corresponding to the bottom-left pixel sample of the current image block comprises motion information units of x3 pixel samples, wherein the x3 pixel samples comprise at least one pixel sample that is spatially adjacent to the bottom-left pixel sample of the current image block and/or at least one pixel sample that is temporally adjacent to the bottom-left pixel sample of the current image block, and x3 is a positive integer;

and

the x3 pixel samples comprise at least one of a pixel sample that is at a same location as the bottom-left pixel sample of the current image block and that is in the video frame temporally adjacent to the video frame to which the current image block belongs, a spatially adjacent pixel sample to the left of the current image block, a spatially adjacent pixel sample to the bottom-left of the current image block, and a spatially adjacent pixel sample to the bottom of the current image block.

14. The method according to claim 10, wherein

a candidate motion information unit set corresponding to the central pixel sample a1 of the current image block comprises motion information units of x5 pixel samples, wherein one of the x5 pixel samples is a pixel sample a2;

and

a location of the central pixel sample a1 in the video frame to which the current image block belongs is the same as a location of the pixel sample a2 in a video frame that is temporally adjacent to the video frame to which the current image block belongs, and x5 is a positive integer.

15. An image prediction method, wherein a current image block comprises at least one first pixel sample and at least one second pixel sample, and the method comprises:

determining candidate motion information unit sets respectively corresponding to each first pixel sample and each second pixel sample, wherein any candidate motion information unit set comprises at least one motion information unit;

determining a combined motion information unit set of the current image block, wherein each motion information unit in the combined motion information unit set is selected from at least some motion information units in the candidate motion information unit sets respectively corresponding to each first pixel sample and each second pixel sample, and motion information of the motion information unit comprises a motion vector with a forward prediction direction and/or a motion vector with a backward prediction direction;

determining, from the combined motion information unit set, motion information units respectively corresponding to each first pixel sample and each second pixel sample;

encoding first bitstream information, wherein the first bitstream information is used to represent the motion information units that respectively correspond to each first pixel sample and each second pixel sample and that are determined from the combined motion information unit set;

using motion information of the motion information unit corresponding to the first pixel sample as motion information of the first pixel sample;

using motion information of the motion information unit corresponding to the second pixel sample as predicted motion information of the second pixel sample;

calculating differential motion information of the second pixel sample, wherein the differential motion information is a difference between the motion information and the predicted motion information;

encoding second bitstream information, wherein the second bitstream information is used to represent differential motion information of each second pixel sample; and

obtaining a predictor of the current image block based on a motion model of the current image block, the motion information of each first pixel sample, and the motion information of each second pixel sample.

16. An image prediction apparatus, wherein a current image block comprises at least one first pixel sample and at least one second pixel sample, and the apparatus comprises:

a processor and a memory coupled to the processor;

the memory is configured to store code or an instruction; and

the processor is configured to invoke the code or the instruction to perform the following operations;

parsing first bitstream information, wherein the first bitstream information is used to indicate motion information units respectively corresponding to each first pixel sample and each second pixel sample;

obtaining motion information of each first pixel sample and predicted motion information of each second pixel sample based on the parsed first bitstream information, wherein the predicted motion information is predicted information of motion information;

parsing second bitstream information, wherein the second bitstream information is used to represent differential motion information of each second pixel sample, and the differential motion information is a difference between motion information and predicted motion information;

obtaining motion information of each second pixel sample based on the parsed second bitstream information and the corresponding predicted motion information of each second pixel sample; and

obtaining a predictor of the current image block based on a motion model of the current image block, the motion information of each first pixel sample, and the motion information of each second pixel sample.

17. The apparatus according to claim 16, wherein the first bitstream information comprises an index, and the index is used to indicate the motion information units respectively corresponding to each first pixel sample and each second pixel sample.

18. The apparatus according to claim 16, wherein the second bitstream information comprises a difference, and the difference is a motion vector difference between a motion vector and a predicted motion vector of any second pixel sample.

19. The apparatus according to claim 16, wherein the processor is specifically configured to:

determine candidate motion information unit sets respectively corresponding to each first pixel sample and each second pixel sample, wherein any candidate motion information unit set comprises at least one motion information unit;

determine a combined motion information unit set of the current image block, wherein each motion information unit in the combined motion information unit set is selected from at least some motion information units in the candidate motion information unit sets respectively corresponding to each first pixel sample and each second pixel sample, and motion information of the motion information unit comprises a motion vector with a forward prediction direction and/or a motion vector with a backward prediction direction;

determine, from the combined motion information unit set based on the parsed first bitstream information, the motion information units respectively corresponding to each first pixel sample and each second pixel sample;

use motion information of the motion information unit corresponding to the first pixel sample as the motion information of the first pixel sample; and

use motion information of the motion information unit corresponding to the second pixel sample as the predicted motion information of the second pixel sample.

20. The apparatus according to claim 19, wherein the processor is specifically configured to:

determine, from N candidate combined motion information unit sets, a combined motion information unit set comprising the motion information units respectively corresponding to each first pixel sample and each second pixel sample, wherein each motion information unit comprised in each of the N candidate combined motion information unit sets is selected from at least some motion information units that meet a constraint in the candidate motion information unit sets respectively corresponding to each first pixel sample and each second pixel sample, N is a positive integer, and the N candidate combined motion information unit sets are different from each other.

21. The apparatus according to claim 20, wherein the N candidate combined motion information unit sets meet at least one of a first condition, a second condition, a third condition, a fourth condition, and a fifth condition, wherein

the first condition comprises:

a motion manner that is of the current image block and that is indicated by a motion information unit in any one of the N candidate combined motion information unit sets is non-translational motion;

the second condition comprises;

two motion information units in any one of the N candidate combined motion information unit sets correspond to a same prediction direction;

the third condition comprises;

two motion information units in any one of the N candidate combined motion information unit sets correspond to a same reference frame index;

the fourth condition comprises;

an absolute value of a difference between motion vector horizontal components of two motion information units in any one of the N candidate combined motion information unit sets is less than or equal to a horizontal component threshold, or an absolute value of a difference between motion vector horizontal components of one motion information unit in any one of the N candidate combined motion information unit sets and a pixel sample Z is less than or equal to a horizontal component threshold, wherein the pixel sample Z of the current image block is different from either of the first pixel sample and the second pixel sample; and

the fifth condition comprises;

an absolute value of a difference between motion vector vertical components of two motion information units in any one of the N candidate combined motion information unit sets is less than or equal to a vertical component threshold, or an absolute value of a difference between motion vector vertical components of any motion information unit in one of the N candidate combined motion information unit sets and a pixel sample Z is less than or equal to a vertical component threshold, wherein the pixel sample Z of the current image block is different from either of the first pixel sample and the second pixel sample.

22. The apparatus according to claim 16, wherein the processor is specifically configured to:

obtain the differential motion information of each second pixel sample based on the parsed second bitstream information; and

add up the differential motion information of each second pixel sample and the corresponding predicted motion information, to obtain the motion information of each second pixel sample.

23. The apparatus according to claim 16, wherein the motion model is a non-translational motion model, and specifically,

the non-translational motion model is an affine motion model in the following form:

24. The apparatus according to claim 16, wherein the motion model is a non-translational motion model, and specifically,

the non-translational motion model is an affine motion model in the following form:

25. The apparatus according to claim 16, wherein the at least one first pixel sample and the at least one second pixel sample comprise two pixel samples in a top-left pixel sample, a top-right pixel sample, a bottom-left pixel sample, and a central pixel sample a1 of the current image block, wherein

the top-left pixel sample of the current image block is a top-left vertex of the current image block or a pixel block that is at a top-left vertex of the current image block and that is comprised in the current image block;

the bottom-left pixel sample of the current image block is a bottom-left vertex of the current image block or a pixel block that is at a bottom-left vertex of the current image block and that is comprised in the current image block;

the top-right pixel sample of the current image block is a top-right vertex of the current image block or a pixel block that is at a top-right vertex of the current image block and that is comprised in the current image block; and

the central pixel sample a1 of the current image block is a central pixel of the current image block or a pixel block that is at a central pixel of the current image block and that is comprised in the current image block.

26. The apparatus according to claim 25, wherein

a candidate motion information unit set corresponding to the top-left pixel sample of the current image block comprises motion information units of x1 pixel samples, wherein the x1 pixel samples comprise at least one pixel sample that is spatially adjacent to the top-left pixel sample of the current image block and/or at least one pixel sample that is temporally adjacent to the top-left pixel sample of the current image block, and x1 is a positive integer;

and

the x1 pixel samples comprise at least one of a pixel sample that is at a same location as the top-left pixel sample of the current image block and that is in a video frame temporally adjacent to a video frame to which the current image block belongs, a spatially adjacent pixel sample to the left of the current image block, a spatially adjacent pixel sample to the top-left of the current image block, and a spatially adjacent pixel sample to the top of the current image block.

27. The apparatus according to claim 25, wherein

a candidate motion information unit set corresponding to the top-right pixel sample of the current image block comprises motion information units of x2 pixel samples, wherein the x2 pixel samples comprise at least one pixel sample that is spatially adjacent to the top-right pixel sample of the current image block and/or at least one pixel sample that is temporally adjacent to the top-right pixel sample of the current image block, and x2 is a positive integer;

and

the x2 pixel samples comprise at least one of a pixel sample that is at a same location as the top-right pixel sample of the current image block and that is in the video frame temporally adjacent to the video frame to which the current image block belongs, a spatially adjacent pixel sample to the right of the current image block, a spatially adjacent pixel sample to the top-right of the current image block, and a spatially adjacent pixel sample to the top of the current image block.

28. The apparatus according to claim 25, wherein

a candidate motion information unit set corresponding to the bottom-left pixel sample of the current image block comprises motion information units of x3 pixel samples, wherein the x3 pixel samples comprise at least one pixel sample that is spatially adjacent to the bottom-left pixel sample of the current image block and/or at least one pixel sample that is temporally adjacent to the bottom-left pixel sample of the current image block, and x3 is a positive integer;

and

the x3 pixel samples comprise at least one of a pixel sample that is at a same location as the bottom-left pixel sample of the current image block and that is in the video frame temporally adjacent to the video frame to which the current image block belongs, a spatially adjacent pixel sample to the left of the current image block, a spatially adjacent pixel sample to the bottom-left of the current image block, and a spatially adjacent pixel sample to the bottom of the current image block.

29. The apparatus according to claim 25, wherein

a candidate motion information unit set corresponding to the central pixel sample a1 of the current image block comprises motion information units of x5 pixel samples, wherein one of the x5 pixel samples is a pixel sample a2;

and

a location of the central pixel sample a1 in the video frame to which the current image block belongs is the same as a location of the pixel sample a2 in a video frame that is adjacent to the video frame to which the current image block belongs, and x5 is a positive integer.

30. An image prediction apparatus, wherein a current image block comprises at least one first pixel sample and at least one second pixel sample, and the apparatus comprises:

a processor and a memory coupled to the processor;

the memory is configured to store code or an instruction; and

the processor is configured to invoke the code or the instruction to perform the following operations;

determining candidate motion information unit sets respectively corresponding to each first pixel sample and each second pixel sample, wherein any candidate motion information unit set comprises at least one motion information unit;

determining a combined motion information unit set of the current image block, wherein each motion information unit in the combined motion information unit set is selected from at least some motion information units in the candidate motion information unit sets respectively corresponding to each first pixel sample and each second pixel sample, and motion information of the motion information unit comprises a motion vector with a forward prediction direction and/or a motion vector with a backward prediction direction;

determining, from the combined motion information unit set, motion information units respectively corresponding to each first pixel sample and each second pixel sample;

encoding first bitstream information, wherein the first bitstream information is used to represent the motion information units that respectively correspond to each first pixel sample and each second pixel sample and that are determined from the combined motion information unit set;

using motion information of the motion information unit corresponding to the first pixel sample as motion information of the first pixel sample;

using motion information of the motion information unit corresponding to the second pixel sample as predicted motion information of the second pixel sample;

calculating differential motion information of the second pixel sample, wherein the differential motion information is a difference between the motion information and the predicted motion information;

encoding second bitstream information, wherein the second bitstream information is used to represent differential motion information of each second pixel sample; and

obtaining a predictor of the current image block based on a motion model of the current image block, the motion information of each first pixel sample, and the motion information of each second pixel sample.