System and method for motion compensation using a set of candidate motion vectors obtained from digital video

• US 8,175,163 B2
• Filed: 06/10/2009
• Issued: 05/08/2012
• Est. Priority Date: 06/10/2009
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Chappalli, Mahesh B., Kim, Yeong-Taeg
• Primary Examiner(s): Czekaj, David
• Assistant Examiner(s): Beck, Leron
• Application Number:
  US12/482,295
• Publication Number:
  US 20100316125A1
• Time in Patent Office: 1,063 Days
• Field of Search: 375/240.01, 375/240.16
• US Class Current: 375/240.16
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/553 Motion estimation dealing with occlusions
• CPC Class Codes: H04N 19/573 Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence

Abstract

Embodiments include systems and methods for interpolating frames. In one embodiment, a system and method provides reduced halo and consequently a higher degree of accuracy/fidelity. In one embodiment, a method of video frame interpolation includes receiving first motion data based on a first video frame and a second video frame, receiving second motion data based on the second video frame and a third video frame, and estimating a block of an interpolated frame based on the second motion data and the first motion data. In one embodiment, the estimating comprises selectively using the second motion data based on the first motion data.

Claims

1. A method of video frame interpolation, comprising:

receiving first motion data based on a first video frame and a second video frame;

receiving second motion data based on the second video frame and a third video frame;

estimating a block of an interpolated frame based on the second motion data and the first motion data with the interpolated frame between the second video frame and the third video frame;

selecting a motion vector of the first motion data associated with a portion of the first video frame corresponding to the block to be estimated;

calculating a specified confidence value based on a time associated with the interpolated frame and a time associated with the third video frame; and

wherein;

estimating the block comprises selectively using the second motion data based on the first motion and selecting motion vectors from the second motion data, and estimating the block based on a weighted average of the motion vectors, and the weighted average is weighted;

based on at least one motion vector of the first motion data,

based at least in part on a difference between each of the motion vectors and the motion vector of the first motion data, and

based at least in part on the specified confidence value; and

calculating the specified confidence value includes calculating the specified confidence value based at least in part on (α

−

α

_adj)/α

, wherein α

is the time associated with the third video frame and wherein α

_adj is the time associated with the interpolated frame.

2. The method of claim 1, wherein:

receiving the first motion data includes receiving the second video frame subsequent to the first video frame; and

receiving the second motion data includes receiving the third video frame subsequent to the second video frame whereby the first motion data comprises backward motion estimation data.

3. The method of claim 1, wherein:

receiving the first motion data includes receiving the first video frame subsequent to the second video frame; and

receiving the second motion data includes receiving the second video frame subsequent to the third video frame whereby the first motion data comprises forward motion estimation data.

4. The method of claim 1, wherein estimating the block comprises estimating the block based on the weighted average calculated at least in part as a function of a difference between interpolated values of the block based on the second and third motion data.

5. The method of claim 1, wherein selectively using the second motion data based on the first motion data comprises identifying the block of the interpolated frame as being in one of a covering or uncovering area based at least in part on the first motion data.

6. The method of claim 1, wherein:

selectively using the second motion data based on the first motion data comprises selecting one of the motion vectors from the second motion data based at least in part on the motion vector of the first motion data; and

estimating the block comprises estimating the block based on one of the motion vectors of the second motion data.

7. A system for video frame interpolation, comprising:

a memory configured to store first motion data based on a first video frame and a second video frame and second motion data based on the second video frame and a third video frame; and

at least one processor configured to;

estimate a block of an interpolated frame based on selective use of the second motion data based on the first motion data and the interpolated frame is between the second video frame and the third video frame,

select a motion vector of the first motion data associated with a portion of the first video frame corresponding to the block to be estimated,

select motion vectors from the second motion data, and

estimate the block based on a weighted average of the motion vectors, and the weighted average is weighted;

based on at least one motion vector of the first motion data,

based at least in part on a difference between each of the selected motion vectors and the motion vector of the first motion data, and

based at least in part on a specified confidence value,

calculate the specified confidence value based on a time associated with the interpolated frame and a time associated with the third video frame, and

calculate the specified confidence value based at least in part on (α

−

α

_adj)/α

, wherein a is the time associated with the third video frame and wherein α

_adj is the time associated with the interpolated frame.

8. The system of claim 7, wherein:

the second video frame is subsequent to the first video frame; and

the third video frame is subsequent to the second video frame whereby the first motion data comprises backward motion estimation data.

9. The system of claim 7, wherein:

the first video frame is subsequent to the second video frame; and

the second video frame is subsequent to the third video frame whereby the first motion data comprises forward motion estimation data.

10. The system of claim 7, wherein the weighted average is calculated at least in part as a function of a difference between interpolated values of the block based on the second and third motion data.

11. The system of claim 7, wherein the processor is further configured to selectively use the second motion data based on the first motion data by identifying the block of the interpolated frame as being in one of a covering or uncovering area based at least in part on the first motion data.

12. The system of claim 7, wherein:

the processor is configured to selectively use the second motion data based on the first motion data by selecting one of the motion vectors from the second motion data based at least in part on the motion vector of the first motion data; and

the processor is configured to estimate the block based on one of the motion vectors of the second motion data.

13. The system of claim 7, further comprising at least a television, a set-top box, a digital-video recorder, or a recorded media player.