Methods and systems to estimate motion based on reconstructed reference frames at a video decoder

• US 8,917,769 B2
• Filed: 09/25/2009
• Issued: 12/23/2014
• Est. Priority Date: 07/03/2009
• Status: Active Grant

• Current Assignee: Tahoe Research Limited (f/k/a Learndale Limited) (Vector Capital Corporation)
• Original Assignee: Intel Corporation
• Inventors: Xu, Lidong, Chiu, Yi-Jen, Zhang, Wenhao
• Primary Examiner(s): Moore, Jr., Michael J
• Application Number:
  US12/566,823
• Publication Number:
  US 20110002389A1
• Time in Patent Office: 1,915 Days
• Field of Search: None
• US Class Current: 375/240.16
• CPC Class Codes: H04N 19/103 Selection of coding mode or of prediction mode
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/513 Processing of motion vectors
• CPC Class Codes: H04N 19/52 by predictive encoding
• CPC Class Codes: H04N 19/57 Motion estimation characterised by a search window with variable size or shape
• CPC Class Codes: H04N 19/573 Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

Methods and systems to apply motion estimation (ME) based on reconstructed reference pictures in a B frame or in a P frame at a video decoder. For a P frame, projective ME may be performed to obtain a motion vector (MV) for a current input block. In a B frame, both projective ME and mirror ME may be performed to obtain an MV for the current input block. The ME process can be performed on sub-partitions of the input block, which may reduce the prediction error without increasing the amount of MV information in the bitstream. Decoder-side ME can be applied for the prediction of existing inter frame coding modes, and traditional ME or the decoder-side ME can be adaptively selected to predict a coding mode based on a rate distribution optimization (RDO) criterion.

Claims

1. A method, comprising:

specifying, at a video decoder, a search window in a first forward reference frame;

specifying a search path in the search window of the first forward reference frame;

for each motion vector MV0 in the search path, where each MV0 points from a current block to a reference block in the search window, determining a corresponding second motion vector MV1 that points to a reference block in a second reference frame, where the corresponding second motion vector MV1 is a function of MV0;

calculating a metric for each pair of MV0 and MV1 that is found in the search path wherein the metric consists of a sum of absolute differences (SAD) between the reference block in the search window and the reference block in the second reference frame;

selecting the MV0 whose corresponding value for the SAD is closest to zero, where the selected MV0 is used as a motion vector for the current block;

determining a predicted motion for the current block, where the predicted motion is one of

an average, and

a weighted average,

of predicted motion for MV0 and MV1.

2. The method of claim 1, wherein the second reference frame comprises a backward reference frame, and each MV1 is a mirror motion vector.

3. The method of claim 2, wherein the current block is located in a bi-predictive frame.

4. The method of claim 1, wherein the second reference frame comprises a second forward reference frame, and each MV1 is a projective motion vector.

5. The method of claim 4, wherein the current block is located in a predictive frame.

6. The method of claim 1, wherein for each MV0, the corresponding MV1 is determined by a function

7. The method of claim 1, wherein the current block is a sub-block of a larger partitioned block.

8. A non-transitory computer program product including a computer readable medium having computer program logic stored therein, the computer program logic including:

motion estimation logic to

cause a processor to determine a motion vector MV1 for each motion vector MV0 that is found in a search path in a search window of a first forward reference frame, where each MV0 points from a current block to a reference block in the search window, and where the MV1 is a function of the MV0;

further cause the processor to calculate a metric for each pair of MV0 and MV1 wherein the metric consists of a sum of absolute differences (SAD) between the reference block in the search window and the reference block in the second reference frame; and

further cause the processor to select the MV0 whose corresponding value for the metric is closest to zero, where the selected MV0 is used as a motion vector for the current block; and

further cause the processor to determine a predicted motion for the current block, where the predicted motion is one of

an average, and

a weighted average,

of predicted motion for MV0 and MV1.

9. The computer program product of claim 8, wherein said motion estimation logic comprises mirror motion estimation logic, and

wherein the second reference frame comprises a backward reference frame, each MV1 is a mirror motion vector, and the current block is located in a bi-predictive frame.

10. The computer program product of claim 8, wherein said motion estimation logic comprises projective motion estimation logic, and

wherein the second reference frame comprises a second forward reference frame, each MV1 is a projective motion vector, and the current block is located in a predictive frame.

11. The computer program product of claim 8, wherein for each MV0, the corresponding MV1 is determined by a function

12. A system, comprising:

a processor; and

a memory in communication with said processor, said memory for storing a plurality of processing instructions for directing said processor to

specify, at a video decoder, a search window in a first forward reference frame;

specify a search path in a search window of the first forward reference frame;

for each motion vector MV0 in the search path, where each MV0 points from a current block to a reference block in the search window, determine a corresponding second motion vector MV1 that points to a reference block in a second reference frame, where the corresponding second motion vector MV1 is a function of MV0;

calculate a metric for each pair of MV0 and MV1 that is found in the search path wherein said metric consists of a sum of absolute differences (SAD) between the reference block in the search window and the reference block in the second reference frame;

select the MV0 whose corresponding value for the SAD is closest to zero, where the selected MV0 is used as a motion vector for the current block;

determine a predicted motion for the current block, where the predicted motion is one of

an average, and

a weighted average,

of predicted motion for MV0 and MV1.

13. The system of claim 12, wherein the second reference frame comprises a backward reference frame, each MV1 is a minor motion vector, and the current block is located in a bi-predictive frame.

14. The system of claim 12, wherein the second reference frame comprises a second forward reference frame, each MV1 is a projective motion vector, and the current block is located in a predictive frame.