Methods and systems for motion vector derivation at a video decoder

• US 9,654,792 B2
• Filed: 09/25/2009
• Issued: 05/16/2017
• Est. Priority Date: 07/03/2009
• Status: Active Grant

• Current Assignee: Tahoe Research Limited (f/k/a Learndale Limited) (Vector Capital Corporation)
• Original Assignee: Intel Corporation
• Inventors: Xu, Lidong, Chiu, Yi-Jen, Jiang, Hong
• Primary Examiner(s): Phillips, Hassan
• Assistant Examiner(s): Tsegaye, Saba
• Application Number:
  US12/567,540
• Publication Number:
  US 20110002390A1
• Time in Patent Office: 2,790 Days
• Field of Search: 37524016, 37524002, 37524024, 375E07104, 375E07125, 382236, 382232, 382239
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/52 by predictive encoding
• CPC Class Codes: H04N 19/56 Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures

Abstract

Method and apparatus for deriving a motion vector at a video decoder. A block-based motion vector may be produced at the video decoder by utilizing motion estimation among available pixels relative to blocks in one or more reference frames. The available pixels could be, for example, spatially neighboring blocks in the sequential scan coding order of a current frame, blocks in a previously decoded frame, or blocks in a downsampled frame in a lower pyramid when layered coding has been used.

Claims

1. A decoder implemented method, comprising:

decoding a portion of pixels of a first frame of a video, the first frame including a first target block of un-decoded pixels;

selecting a first block of decoded pixels of a first reference frame, the first block of decoded pixels in a position of the first reference frame corresponding to a position of the first target block in the first frame;

identifying a first motion vector of the selected first block of the first reference frame that references a second reference frame of the video, wherein the first reference frame is temporally previous to the first frame and the second reference frame is temporally previous to the first reference frame, and wherein identifying the first motion vector of the selected first block comprises performing a motion search at the decoder, the motion search using the selected first block to search multiple blocks of the second reference frame; and

performing motion compensation for the first target block using the first motion vector.

2. The method of claim 1, further including:

decoding a second portion of pixels of a second frame of the video, the second frame including a second target block of un-decoded pixels;

selecting multiple blocks of decoded pixels of the second frame based on positions of the selected blocks relative to the second target block;

performing a motion search at the decoder to determine second motion vectors of the selected multiple blocks of the second frame that reference a third reference frame of the video;

weighted averaging or median filtering the second motion vectors to determine a third motion vector of the second target block; and

performing motion compensation for the second target block using the third motion vector.

3. The method of claim 1, further including:

decoding a second portion of pixels of a second frame of the video, the second frame including a second target block of un-decoded pixels;

selecting multiple blocks of decoded pixels of the second frame based on positions of the selected blocks relative to the second target block;

performing a motion search at the decoder to determine second motion vectors of the selected multiple blocks of the second frame that reference a third reference frame of the video and third motion vectors of the selected multiple blocks of the second reference frame that reference a fourth reference frame of the video, wherein the third reference frame precedes the second frame, and wherein the fourth reference frame is subsequent to the second frame;

weighted averaging or median filtering the second and third motion vectors to determine a fourth motion vector of the second target block; and

performing motion compensation for the second target block using the fourth motion vector.

4. The method of claim 1, further including:

decoding a portion of pixels of a second frame of the video, the second frame including a second target block of un-decoded pixels;

selecting a block of pixels of a third reference frame of the video that precedes the second frame based on a position of the selected block of the third reference frame relative to the second target block;

performing a motion search at the decoder to determine a third motion vector of the selected block of the third reference frame that references a fourth reference frame of the video that precedes the third reference frame;

selecting a block of pixels of a fifth reference frame of the video that is subsequent to the second frame based on a position of the selected block of the fifth reference frame relative to the second target block;

performing a motion search at the decoder to determine a fourth motion vector of the selected block of the fifth reference frame that references a sixth reference frame of the video that is subsequent to the fifth reference frame; and

performing motion compensation for the second target block based on the third and fourth motion vectors.

5. The method of claim 1, further including:

decoding a portion of pixels of a second frame of the video, the second frame including a second target block of un-decoded pixels;

selecting a block of a downsampled frame that corresponds to the second target block;

performing a motion search at the decoder to determine a second motion vector of the selected block of the downsampled frame that references a down sampled reference frame;

upscaling the second motion vector; and

performing motion compensation for the second target block based on the upscaled second motion vector.

6. The method of claim 1, further including:

decoding a second portion of pixels of a second frame of the video, the second frame including a second target block of un-decoded pixels;

selecting a second block of decoded pixels in a third reference frame, wherein the second block is at least one of above, below, to the left, or to the right with respect to a position of the second target block in the second frame;

performing a motion search at the decoder, using the selected second block, of a fourth reference frame to determine a second motion vector of the selected second block; and

performing motion compensation for the second target block using the second motion vector.

7. A non-transitory computer readable medium encoded with a computer program that includes instructions to cause a processor of a decoder to:

decode a portion of pixels of a first frame of a video, the first frame including a first target block of un-decoded pixels;

select a first block of decoded pixels of a first reference frame, the first block of decoded pixels in a position of the first reference frame corresponding to a position of the first target block in the first frame;

identify a first motion vector of the selected first block of the first reference frame that references a second reference frame of the video, wherein the first reference frame is temporally previous to the first frame and the second reference frame is temporally previous to the first reference frame, and wherein identifying the first motion vector of the selected first block comprises performing a motion search at the decoder, the motion search using the selected first block to search multiple blocks of the second reference frame; and

perform motion compensation for the first target block using the first motion vector.

8. The non-transitory computer readable medium of claim 7, further including instructions to cause the processor to:

decode a second portion of pixels of a second frame of the video, the second frame including a second target block of un-decoded pixels;

select multiple blocks of decoded pixels of the second frame based on positions of the selected blocks relative to the second target block;

perform a motion search at the decoder to determine second motion vectors of the selected multiple blocks of the second frame that reference a third reference frame of the video;

weighted averaging or median filter the second motion vectors to determine a third motion vector of the second target block; and

perform motion compensation for the second target block using the third motion vector.

9. The non-transitory computer readable medium of claim 7, further including instructions to cause the processor to:

decode a second portion of pixels of a second frame of the video, the second frame including a second target block of un-decoded pixels;

select multiple blocks of decoded pixels of the second frame based on positions of the selected blocks relative to the second target block;

perform a motion search at the decoder to determine second motion vectors of the selected multiple blocks of the second frame that reference a third reference frame of the video third motion vectors of the selected multiple blocks of the second reference frame that reference a fourth reference frame of the video, wherein the third reference frame precedes the second frame, and wherein the fourth reference frame is subsequent to the second frame;

weighted averaging or median filtering the second and third motion vectors to determine a fourth motion vector of the second target block; and

perform motion compensation for the second target block using the fourth motion vector.

10. The non-transitory computer readable medium of claim 7, further including instructions to cause the processor to:

decode a portion of pixels of a second frame of the video, the second frame including a second target block of un-decoded pixels;

select a block of pixels of a third reference frame of the video that precedes the second frame based on a position of the selected block of the third reference frame relative to the second target block;

perform a motion search at the decoder to determine a third motion vector of the selected block of the third reference frame that references a fourth reference frame of the video that precedes the third reference frame;

select a block of pixels of a fifth reference frame of the video that is subsequent to the second frame based on a position of the selected block of the fifth reference frame relative to the second target block;

perform a motion search at the decoder to determine a fourth motion vector of the selected block of the fifth reference frame that references a sixth reference frame of the video that is subsequent to the fifth reference frame; and

perform motion compensation for the second target block based on the third and fourth motion vectors.

11. The non-transitory computer readable medium of claim 7, further including instructions to cause the processor to:

decode a portion of pixels of a second frame of the video, the second frame including a second target block of un-decoded pixels;

select a block of a downsampled frame that corresponds to the second target block;

perform a motion search at the decoder to determine a second motion vector of the selected block of the downsampled frame that references a downsampled reference frame;

upscale the second motion vector; and

perform motion compensation for the second target block based on the upscaled second motion vector.

12. The non-transitory computer readable medium of claim 7, further including instructions to cause the processor to:

decode a second portion of pixels of a second frame of the video, the second frame including a second target block of un-decoded pixels;

select a second block of decoded pixels in a third reference frame, wherein the second block is at least one of above, below, to the left, or to the right with respect to a position of the second target block in the second frame;

perform a motion search at the decoder, using the selected second block, of a fourth reference frame to determine a second motion vector of the selected second block; and

perform motion compensation for the second target block using the second motion vector.

13. A decoder apparatus, comprising a processor and memory configured to:

decode a portion of pixels of a first frame of a video, the first frame including a first target block of un-decoded pixels;

select a first block of decoded pixels of a first reference frame, the first block of decoded pixels in a position of the first reference frame corresponding to a position of the first target block in the first frame;

identify a first motion vector of the selected first block of the first reference frame that references a second reference frame of the video, wherein the first reference frame is temporally previous to the first frame and the second reference frame is temporally previous to the first reference frame, and wherein identifying the first motion vector of the selected first block comprises performing a motion search at the decoder apparatus, the motion search using the selected first block to search multiple blocks of the second reference frame; and

perform motion compensation for the first target block using the first motion vector.

14. The apparatus of claim 13, wherein the processor and memory are further configured to:

decode a second portion of pixels of a second frame of the video, the second frame including a second target block of un-decoded pixels;

select multiple blocks of decoded pixels of the second frame based on positions of the selected blocks relative to the second target block;

perform a motion search at the decoder to determine second motion vectors of the selected multiple blocks of the second frame that reference a third reference frame of the video;

weighted average or median filter the second motion vectors to determine a third motion vector of the second target block; and

perform motion compensation for the second target block using the second motion vector.

15. The apparatus of claim 13, wherein the processor and memory are further configured to:

decode a second portion of pixels of a second frame of the video, the second frame including a second target block of un-decoded pixels;

select multiple blocks of decoded pixels of the second frame based on positions of the selected blocks relative to the second target block;

perform a motion search at the decoder to determine second motion vectors of the selected multiple blocks of the second frame that reference a third reference frame of the video third motion vectors of the selected multiple blocks of the second reference frame that reference a fourth reference frame of the video, wherein the third reference frame precedes the second frame, and wherein the fourth reference frame is subsequent to the second frame;

weighted averaging or median filtering the second and third motion vectors to determine a fourth motion vector of the second target block; and

perform motion compensation for the second target block using the fourth motion vector.

16. The apparatus of claim 13, wherein the processor and memory are further configured to:

decode a portion of pixels of a second frame of the video, the second frame including a second target block of un-decoded pixels;

select a block of pixels of a third reference frame of the video that precedes the second frame based on a position of the selected block of the third reference frame relative to the second target block;

perform a motion search at the decoder to determine a third motion vector of the selected block of the third reference frame that references a fourth reference frame of the video that precedes the third reference frame;

select a block of pixels of a fifth reference frame of the video that is subsequent to the second frame based on a position of the selected block of the fifth reference frame relative to the second target block;

perform a motion search at the decoder to determine a fourth motion vector of the selected block of the fifth reference frame that references a sixth reference frame of the video that is subsequent to the fifth reference frame; and

perform motion compensation for the second target block based on the third and fourth motion vectors.

17. The apparatus of claim 13, wherein the processor and memory are further configured to:

decode a portion of pixels of a second frame of the video, the second frame including a second target block of un-decoded pixels;

select a block of a downsampled frame that corresponds to the second target block;

perform a motion search at the decoder to determine a second motion vector of the selected block of the downsampled frame that references a downsampled reference frame;

upscale the second motion vector; and

perform motion compensation for the second target block based on the upscaled second motion vector.

18. The apparatus of claim 13, wherein the processor and memory are further configured to

decode a second portion of pixels of a second frame of the video, the second frame including a second target block of un-decoded pixels;

select a second block of decoded pixels in a third reference frame, wherein the second block is at least one of above, below, to the left, or to the right with respect to a position of the second target block in the second frame;

perform a motion search at the decoder, using the selected second block, of a fourth reference frame to determine a second motion vector of the selected second block; and

perform motion compensation for the second target block using the second motion vector.