PICTURE PREDICTION METHOD AND RELATED APPARATUS

• US 20170195685A1
• Filed: 03/20/2017
• Published: 07/06/2017
• Est. Priority Date: 10/27/2014
• Status: Active Grant

• Current Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: CHEN, Huanbang, LIANG, Fan, LIN, Sixin
• Granted Patent: US 10,440,380 B2
• Time in Patent Office: Days
• CPC Class Codes: H04N 19/103 Selection of coding mode or of prediction mode
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/147 according to rate distortion criteria rate-distortion as a criterion for motion estimation H04N19/567
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/182 the unit being a pixel
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/537 Motion estimation other than block-based
• CPC Class Codes: H04N 19/567 Motion estimation based on rate distortion criteria

Abstract

A picture prediction method and a related apparatus are disclosed. A picture prediction method includes: determining K1 pixel samples in a picture block x, and determining a candidate motion information unit set corresponding to each pixel sample in the K1 pixel samples, where the candidate motion information unit set corresponding to each pixel sample includes at least one candidate motion information unit; determining a merged motion information unit set i including K1 motion information units, where each motion information unit in the merged motion information unit set i is selected from at least a part of motion information units in candidate motion information unit sets corresponding to different pixel samples in the K1 pixel samples; and predicting a pixel value of the picture block x by using a non-translational motion model and the merged motion information unit set i.

Claims

1. A picture prediction method, comprising:

determining K1 pixel samples in a picture block x, and determining a candidate motion information unit set corresponding to each pixel sample in the K1 pixel samples, wherein the candidate motion information unit set corresponding to each pixel sample comprises at least one candidate motion information unit, and K1 is an integer that is greater than or equal to 2;

determining a merged motion information unit set i comprising K1 motion information units, wherein

each motion information unit in the merged motion information unit set i is selected from at least a part of motion information units in candidate motion information unit sets corresponding to different pixel samples in the K1 pixel samples, and the motion information unit comprises a motion vector whose prediction direction is at least one of;

forward and backward; and

predicting a pixel value of the picture block x by using a non-translational motion model and the merged motion information unit set i.

2. The method according to claim 1, wherein the determining a merged motion information unit set i comprising K1 motion information units comprises:

determining, from N candidate merged motion information unit sets, the merged motion information unit set i comprising the K1 motion information units, wherein each motion information unit comprised in each candidate merged motion information unit set in the N candidate merged motion information unit sets is selected from at least a part of constraint-compliant motion information units in the candidate motion information unit sets corresponding to different pixel samples in the K1 pixel samples, N is a positive integer, the N candidate merged motion information unit sets are different from each other, and each candidate merged motion information unit set in the N candidate merged motion information unit sets comprises the K1 motion information units.

3. The method according to claim 2, wherein the N candidate merged motion information unit sets meet at least one of:

a first condition, a second condition, a third condition, a fourth condition, and a fifth condition, wherein

the first condition comprises that a motion mode of the picture block x indicated by a motion information unit in any candidate merged motion information unit set in the N candidate merged motion information unit sets is non-translational motion;

the second condition comprises that prediction directions of at least two motion information units in any candidate merged motion information unit set in the N candidate merged motion information unit sets are the same;

the third condition comprises that reference frame indexes corresponding to at least two motion information units in any candidate merged motion information unit set in the N candidate merged motion information unit sets are the same;

the fourth condition comprises that an absolute value of a difference between horizontal components of two motion information units in any candidate merged motion information unit set in the N candidate merged motion information unit sets is less than or equal to a horizontal component threshold; and

the fifth condition comprises that an absolute value of a difference between vertical components of two motion information units in any candidate merged motion information unit set in the N candidate merged motion information unit sets is less than or equal to a vertical component threshold.

4. The method according to claim 1, wherein the K1 pixel samples comprise at least two pixel samples in an upper left pixel sample, an upper right pixel sample, a lower left pixel sample, and a central pixel sample a1 of the picture block x, wherein

the upper left pixel sample of the picture block x is one of:

an upper left vertex of the picture block x, and a pixel block in the picture block x comprising an upper left vertex of the picture block x;

the lower left pixel sample of the picture block x is one of;

a lower left vertex of the picture block x, and a pixel block in the picture block x comprising a lower left vertex of the picture block x;

the upper right pixel sample of the picture block x is one of;

an upper right vertex of the picture block x, and a pixel block in the picture block x comprising an upper right vertex of the picture block x; and

the central pixel sample a1 of the picture block x is one of;

a central pixel of the picture block x, and a pixel block in the picture block x comprising a central pixel of the picture block x.

5. The method according to claim 4, wherein

a candidate motion information unit set corresponding to the upper left pixel sample of the picture block x comprises motion information units of x1 pixel samples, wherein

x1 is a positive integer;

the x1 pixel samples comprise at least one of;

at least one pixel sample spatially adjacent to the upper left pixel sample of the picture block x and at least one pixel sample temporally adjacent to the upper left pixel sample of the picture block x; and

the x1 pixel samples comprise at least one of;

a pixel sample that has a same location as the upper left pixel sample of the picture block x, a pixel sample spatially adjacent to a left edge of the picture block x, a pixel sample spatially adjacent to an upper left of the picture block x, and a pixel sample spatially adjacent to an upper edge of the picture block x;

in a video frame temporally adjacent to a video frame to which the picture block x belongs.

6. The method according to claim 4, wherein

a candidate motion information unit set corresponding to the upper right pixel sample of the picture block x comprises motion information units of x2 pixel samples, wherein

x2 is a positive integer;

the x2 pixel samples comprise at least one of;

at least one pixel sample spatially adjacent to the upper right pixel sample of the picture block x and at least one pixel sample temporally adjacent to the upper right pixel sample of the picture block x; and

the x2 pixel samples comprise at least one of;

a pixel sample that has a same location as the upper right pixel sample of the picture block x, a pixel sample spatially adjacent to a right edge of the picture block x, a pixel sample spatially adjacent to an upper right of the picture block x, and a pixel sample spatially adjacent to the upper edge of the picture block x;

in a video frame temporally adjacent to the video frame to which the picture block x belongs.

7. The method according to claim 4, wherein

a candidate motion information unit set corresponding to the lower left pixel sample of the picture block x comprises motion information units of x3 pixel samples, wherein

x3 is a positive integer;

the x3 pixel samples comprise at least one of;

at least one pixel sample spatially adjacent to the lower left pixel sample of the picture block x and at least one pixel sample temporally adjacent to the lower left pixel sample of the picture block x; and

the x3 pixel samples comprise at least one of;

a pixel sample that has a same location as the lower left pixel sample of the picture block x, a pixel sample spatially adjacent to the left edge of the picture block x, a pixel sample spatially adjacent to a lower left of the picture block x, and a pixel sample spatially adjacent to a lower edge of the picture block x;

in a video frame temporally adjacent to the video frame to which the picture block x belongs.

8. The method according to claim 4, wherein

a candidate motion information unit set corresponding to the central pixel sample a1 of the picture block x comprises motion information units of x5 pixel samples, wherein

one pixel sample in the x5 pixel samples is a pixel sample a2,

a location of the central pixel sample a1 in the video frame to which the picture block x belongs is the same as a location of the pixel sample a2 in a video frame adjacent to the video frame to which the picture block x belongs, and

x5 is a positive integer.

9. The method according to claim 1, wherein the predicting a pixel value of the picture block x by using a non-translational motion model and the merged motion information unit set i comprises one of:

when motion vectors whose prediction directions are a first prediction direction in the merged motion information unit set i correspond to different reference frame indexes, performing scaling processing on the merged motion information unit set i, so that the motion vectors whose prediction directions are the first prediction direction in the merged motion information unit set i are scaled down to a same reference frame, and predicting the pixel value of the picture block x by using the non-translational motion model and a scaled merged motion information unit set i, wherein the first prediction direction is forward or backward; and

when motion vectors whose prediction directions are forward in the merged motion information unit set i correspond to different reference frame indexes and motion vectors whose prediction directions are backward in the merged motion information unit set i correspond to different reference frame indexes, performing scaling processing on the merged motion information unit set i, so that the motion vectors whose prediction directions are forward in the merged motion information unit set i are scaled down to a same reference frame and that the motion vectors whose prediction directions are backward in the merged motion information unit set i are scaled down to a same reference frame, and predicting the pixel value of the picture block x by using the non-translational motion model and a scaled merged motion information unit set i.

10. The method according to claim 1, wherein the method further comprises:

determining K2 pixel samples in a picture block y, and determining a candidate motion information unit set corresponding to each pixel sample in the K2 pixel samples, wherein K2 is an integer that is greater than 1, and the picture block y is spatially adjacent to the picture block x, and the candidate motion information unit set corresponding to each pixel sample in the K2 pixel samples comprises at least one candidate motion information unit;

determining a merged motion information unit set j comprising K2 motion information units;

wherein a candidate motion information unit set corresponding to a pixel sample z1 in the K2 pixel samples comprises a motion information unit a2, the motion information unit a2 is obtained based on a motion information unit of a pixel sample z2, the pixel sample z2 is a pixel sample in the picture block x, a distance between the pixel sample z2 and the pixel sample z1 is one of;

less than a threshold and a shortest possible distance, and the K2 motion information units in the merged motion information unit set j are respectively selected from at least a part of constraint-compliant motion information units in the candidate motion information unit set corresponding to each pixel sample in the K2 pixel samples; and

predicting a pixel value of the picture block y by using the non-translational motion model and the merged motion information unit set j.

11. The method according to claim 1, wherein the non-translational motion model is one of:

an affine motion model, a parabolic motion model, a rotational motion model, a perspective motion model, a shearing motion model, a scaling motion model, and a bilinear motion model.

12. The method according to claim 1, wherein the predicting a pixel value of the picture block x by using a non-translational motion model and the merged motion information unit set i comprises one of:

obtaining a motion vector of each pixel in the picture block x through computation by using the non-translational motion model and the merged motion information unit set i, and determining a predicted pixel value of each pixel in the picture block x by using the obtained motion vector of each pixel in the picture block x; and

obtaining a motion vector of each pixel block in the picture block x through computation by using the non-translational motion model and the merged motion information unit set i, and determining a predicted pixel value of each pixel in each pixel block in the picture block x by using the obtained motion vector of each pixel block in the picture block x.

13. The method according to claim 1, wherein the picture prediction method is applied to one of:

a video coding process and a video decoding process.

14. The method according to claim 13, wherein when the picture prediction method is applied to the video decoding process, the determining, from N candidate merged motion information unit sets, the merged motion information unit set i comprising the K1 motion information units, comprises:

determining, from the N candidate merged motion information unit sets, based on an identifier that is of the merged motion information unit set i and is obtained from a video bit stream, the merged motion information unit set i comprising the K1 motion information units.

15. The method according to claim 13, wherein when the picture prediction method is applied to the video coding process, the determining, from N candidate merged motion information unit sets, the merged motion information unit set i comprising the K1 motion information units, comprises:

determining, from the N candidate merged motion information unit sets, according to one of;

distortion and a rate distortion cost, the merged motion information unit set i comprising the K1 motion information units.

16. The method according to claim 13, wherein when the picture prediction method is applied to the video coding process, the method further comprises:

writing an identifier of the merged motion information unit set i into a video bit stream.

17. A picture prediction apparatus, comprising:

a processor; and

a non-transitory computer-readable storage medium coupled to the processor and including computer-executable instructions for execution by the processor to perform operations comprising;

determining K1 pixel samples in a picture block x, and determining a candidate motion information unit set corresponding to each pixel sample in the K1 pixel samples, wherein the candidate motion information unit set corresponding to each pixel sample comprises at least one candidate motion information unit, and K1 is an integer that is greater than or equal to 2;

determining a merged motion information unit set i comprising K1 motion information units, wherein each motion information unit in the merged motion information unit set i is selected from at least a part of motion information units in candidate motion information unit sets corresponding to different pixel samples in the K1 pixel samples, and the motion information unit comprises a motion vector whose prediction direction is at least one of forward and backward; and

predicting a pixel value of the picture block x by using a non-translational motion model and the merged motion information unit set i.

18. The apparatus according to claim 17, wherein the determining a merged motion information unit set i comprising K1 motion information units comprises:

determining, from N candidate merged motion information unit sets, the merged motion information unit set i comprising the K1 motion information units, wherein each motion information unit comprised in each candidate merged motion information unit set in the N candidate merged motion information unit sets is selected from at least a part of constraint-compliant motion information units in the candidate motion information unit sets corresponding to different pixel samples in the K1 pixel samples, N is a positive integer, the N candidate merged motion information unit sets are different from each other, and each candidate merged motion information unit set in the N candidate merged motion information unit sets comprises the K1 motion information units.

19. The apparatus according to claim 18, wherein the N candidate merged motion information unit sets meet at least one of:

a first condition, a second condition, a third condition, a fourth condition, and a fifth condition, wherein

the first condition comprises that a motion mode of the picture block x indicated by a motion information unit in any candidate merged motion information unit set in the N candidate merged motion information unit sets is non-translational motion;

the second condition comprises that prediction directions of at least two motion information units in any candidate merged motion information unit set in the N candidate merged motion information unit sets are the same;

the third condition comprises that reference frame indexes corresponding to at least two motion information units in any candidate merged motion information unit set in the N candidate merged motion information unit sets are the same;

the fourth condition comprises that an absolute value of a difference between horizontal components of two motion information units in any candidate merged motion information unit set in the N candidate merged motion information unit sets is less than or equal to a horizontal component threshold; and

the fifth condition comprises that an absolute value of a difference between vertical components of two motion information units in any candidate merged motion information unit set in the N candidate merged motion information unit sets is less than or equal to a vertical component threshold.

20. The apparatus according to claim 17, wherein the K1 pixel samples comprise at least two pixel samples in an upper left pixel sample, an upper right pixel sample, a lower left pixel sample, and a central pixel sample a1 of the picture block x, wherein

the upper left pixel sample of the picture block x is one of:

an upper left vertex of the picture block x, and a pixel block in the picture block x comprising an upper left vertex of the picture block x;

the lower left pixel sample of the picture block x is one of;

a lower left vertex of the picture block x, and a pixel block in the picture block x comprising a lower left vertex of the picture block x;

the upper right pixel sample of the picture block x is one of;

an upper right vertex of the picture block x, and a pixel block in the picture block x comprising an upper right vertex of the picture block x; and

the central pixel sample a1 of the picture block x is one of;

a central pixel of the picture block x, and a pixel block in the picture block x comprising a central pixel of the picture block x.