MOTION VECTOR REFINEMENT FOR MULTI-REFERENCE PREDICTION

• US 20200137414A1
• Filed: 12/27/2019
• Published: 04/30/2020
• Est. Priority Date: 06/30/2017
• Status: Active Grant

• Current Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: Esenlik, Semih, Kotra, Anand Meher, Zhao, Zhijie
• Granted Patent: US 11,463,726 B2
• Time in Patent Office: Days
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/109 among a plurality of temporal predictive coding modes
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/513 Processing of motion vectors
• CPC Class Codes: H04N 19/523 with sub-pixel accuracy
• CPC Class Codes: H04N 19/533 Motion estimation using multistep search, e.g. 2D-log search or one-at-a-time search [OTS]
• CPC Class Codes: H04N 19/56 Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search
• CPC Class Codes: H04N 19/57 Motion estimation characterised by a search window with variable size or shape
• CPC Class Codes: H04N 19/573 Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures

Abstract

The present invention relates to the performance of motion vector refinement in a search space for multi-reference inter-prediction. Two or more reference pictures are selected, one of those used for motion vector refinement. Based on an initial estimate of a motion vector to the reference picture for motion vector refinement, a search space in this reference image is constructed. Using template matching, the first motion vector is refined. The second motion vector to another reference picture is calculated using its initial estimate, the initial estimate of the first motion vector and the refined first motion vector.

Claims

1. An apparatus for determining a first motion vector in a first reference picture of a video and a second motion vector in a second reference picture of the video, the first and second motion vectors to be applied in inter-prediction of a picture block of the video in a current picture, the apparatus comprising:

a motion vector refinement unit configured to obtain an estimate of the first motion vector and determine the first motion vector by performing a search within a search space specified based on the estimate of the first motion vector, and

a motion vector calculation unit configured to obtain an estimate of the second motion vector and calculate the second motion vector based on the estimate of the second motion vector and based on the first motion vector.

2. The apparatus according to claim 1, wherein the motion vector calculation unit is configured to calculate the second motion vector by adding to the estimate of the second motion vector a difference between the first motion vector and the estimate of the first motion vector or a function of the difference between the first motion vector and the estimate of the first motion vector.

3. The apparatus according to claim 2, wherein the function includes scaling by a scaling factor and/or clipping.

4. The apparatus according to claim 3, wherein the value of the scaling factor depends on a ratio between the respective distances of the first reference picture and the second reference picture to the current picture.

5. The apparatus according to claim 1, further comprising

a reference picture selection unit configured to obtain two reference pictures and select the first reference picture from among the two reference pictures and select the second reference picture from among the two reference pictures.

6. The apparatus according to claim 5, wherein

the reference picture selection unit is configured to select either the first picture or the second picture based on whether the respective first or second picture is referred to in a bit stream also including the coded picture block of the video by an index relating to a predefined list of reference pictures among at least two lists of reference pictures, wherein a list of reference pictures associates an index with the position of a reference picture with respect to the current picture.

7. The apparatus according to claim 6, wherein

the reference picture selection unit is configured to, if the two reference pictures are referred to in the bitstream by an index in the same predefined list of reference pictures, select as the first reference picture the picture with the highest position in said list of reference pictures.

8. The apparatus according to claim 5, wherein the reference picture selection unit is configured to select as the first reference picture the picture which has the lowest temporal layer among the two pictures.

9. The apparatus according to claim 5, wherein the reference picture selection unit is configured to select, as the first reference picture, the picture which has the lowest base quantization value.

10. The apparatus according to claim 5, wherein the reference picture selection unit is configured to select as the first reference picture the picture which has the lowest distance to the current picture.

11. The apparatus according to claim 5, wherein the reference picture selection unit is configured to select the first reference picture and the second reference picture such that the estimate of the first motion vector is smaller in magnitude than the estimate of the second motion vector.

12. The apparatus according to claim 1, further comprising:

a motion vector estimator configured to determine the estimate of the first motion vector and the estimate of the second motion vector by selecting them from a set of motion vector candidates based on similarity of a template with a portion of the picture referred to by the respective motion vector candidate.

13. A video decoder for decoding a plurality of pictures from a bitstream, comprising:

an inter-prediction unit including the apparatus according to claim 1 and a prediction unit configured to determine a prediction block according to a portion of the first reference picture referred to by the first motion vector and a portion of the second reference picture referred to by the second motion vector,

a bitstream parser configured to obtain from the bitstream the estimation of the first motion vector and the estimate of the second motion vector, and

a reconstruction unit configured to reconstruct the current block according to the prediction block.

14. A video encoder for encoding a plurality of pictures into a bitstream, comprising:

an inter-prediction unit including the apparatus according to claim 1 and a prediction unit configured to determine a prediction block according to a portion of the first reference picture referred to by the first motion vector and a portion of the second reference picture referred to by the second motion vector,

a bitstream former configured to include into the bitstream the estimate of the first motion vector and the estimate of the second motion vector,

a reconstruction unit configured to reconstruct the current block according to the prediction block and storing the reconstructed block in a memory.

15. A method for determining a first motion vector in a first reference picture of a video and a second motion vector in a second reference picture of the video, the first and second motion vectors to be applied in inter-prediction of a picture block in a current picture of the video, the method comprising:

obtaining an estimate of the first motion vector,

determining the first motion vector by performing a search within a search space specified based on the estimate of the first motion vector,

obtaining an estimate of the second motion vector, and

calculating the second motion vector based on the estimate of the second motion vector and based on the first motion vector.

16. A non-transitory computer-readable medium carrying executable instructions which, when executed by a processor, cause the method of claim 15 to be performed.