Increased video playback framerate in low bit-rate video applications

• US 6,594,313 B1
• Filed: 12/23/1998
• Issued: 07/15/2003
• Est. Priority Date: 12/23/1998
• Status: Expired due to Term

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Kasai, Arlene, Hazra, Rajeeb
• Primary Examiner(s): RAO, ANAND SHASHIKANT
• Application Number:
  US09/221,666
• Time in Patent Office: 1,665 Days
• Field of Search: 375/240-2-2, 375/240-5-2
• US Class Current: 375/240.16
• CPC Class Codes: H04N 19/503 involving temporal prediction adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 7/014 involving the use of motion vectors motion estimation and compensation in video coding H04N19/51

Abstract

A method comprising selecting a number of blocks of a frame pair and synthesizing an interpolated frame based on those selected blocks of the frame pair. Additionally, the synthesis of the interpolated frame is aborted upon determining the interpolated frame has an unacceptable quality.

Claims

1. A method comprising:

selecting a number of blocks of a frame pair;

synthesizing an interpolated frame based on the selected blocks of the frame pair; and

aborting the synthesis of the interpolated frame upon determining the interpolated frame has an unacceptable quality.

21. The method of claim 1, wherein determining the interpolated frame has an unacceptable quality further comprises:

determining that a sum of relative portions of covered and uncovered pixels within a block exceeds a first predetermined threshold.

22. The method of claim 21, wherein determining the interpolated frame has an unacceptable quality further comprises:

determining that a sum of relative portions of covered and uncovered pixels within an other block exceeds the first predetermined threshold;

marking the block and the other block as suspect; and

determining the a number of blocks marked as suspect exceeds a second predetermined threshold.

2. A method comprising:

selecting a block size based on a level of activity for a current frame and a previous frame; and

synthesizing an interpolated frame based on the selected block size of the current frame and the previous frame.

3. The method of claim 2 wherein selecting a block size based on a level of activity for a current frame and a previous frame comprises selecting a variable block size within a frame based on the level of activity for the current frame and the previous frame.

4. The method of claim 2 wherein selecting the block size based on the level of activity for the current frame and the previous frame comprises determining proportions of a number of pixels in the current frame belonging to a number of certain classes.

5. The method of claim 4 wherein determining proportions of the number of pixels in the current frame belonging to the number of certain classes comprises:

(a) selecting an initial block size of N×

N, wherein N is a block size variable;

(b) determining relative proportions of pixels for each block that belong to a certain class;

(c) determining the number of blocks that have a single class of pixels in excess of a first predetermined threshold to form a homogenous value;

(d) comparing the homogenous value to a second predetermined threshold;

(e) using block size N×

N wherein N is the block size variable upon determining the homogenous value is greater than the second predetermined threshold; and

(f) dividing the block size variable by two and repeating (b)-(e) until the homogenous value is in excess of the second predetermined threshold or the block size variable is less than a third predetermined threshold.

23. The method of claim 2, wherein selecting a block size further comprises:

dynamically changing the block size on a per-frame basis.

24. The method of claim 23, wherein selecting a block size further comprises:

changing a resolution of a motion field to account for both large and small amounts of motion.

38. The method of claim 23, further comprising:

a video display, wherein the video display receives the current frame and the interpolated frame from the synthesizer to display the current frame and the interpolated frame.

6. A method comprising:

selecting a zero motion vector for a given pixel in an interpolated frame upon determining a current pixel in a current frame corresponding to the given pixel in the interpolated frame is classified as covered or uncovered; and

synthesizing the interpolated frame based on selecting the zero motion vector for the given pixel in the interpolated frame upon determining the current pixel in the current frame corresponding to the given pixel in the interpolated frame is classified as covered or uncovered.

25. The method of claim 6, further comprising:

determining a number of pixels in the current frame to be covered or uncovered.

26. The method of claim 25, further comprising:

low-pass filtering an edge of a covered region.

7. A method comprising:

classifying a number of pixels in a current frame into one of a number of different pixel classifications for synthesis of an interpolated frame; and

aborting of the synthesis of the interpolated frame and repeating a previous frame upon determining the interpolated frame has an unacceptable quality based on the classifying of the number of pixels in the current frame.

8. The method of claim 7 wherein aborting of the synthesis of the interpolated frame and repeating the previous frame upon determining the interpolated frame has the unacceptable quality based on the classifying the number of pixels in the current frame comprises aborting of the synthesis of the interpolated frame and repeating the previous frame upon determining a number of covered and uncovered pixels in the current frame exceeded a predetermined threshold.

27. The method of claim 7, wherein determining the interpolated frame has an unacceptable quality further comprises:

determining that a number of non-stationary blocks in the interpolated frame have been forced to use a zero motion vector.

28. The method of claim 27, wherein determining the interpolated frame has an unacceptable quality further comprises:

determining that the number of non-stationary blocks exceeds a predetermined proportion of all blocks in the interpolated frame.

9. A method comprising:

selecting a best motion vector from a forward motion vector, a backward motion vector, and a zero motion vector for each of a number of blocks for a hypothetical interpolated frame situated temporally in between a current frame and a previous frame;

scaling the best motion vector for each of the number of blocks for the hypothetical interpolated frame for a number of interpolated frames a relative distance of the number of interpolated frames from the current frame; and

synthesizing the number of interpolated frames based on the best motion vector for each block within the number of interpolated frames.

29. The method of claim 9, further comprising:

creating a candidate list for each block in the number of blocks.

30. The method of claim 29, wherein the candidate list includes the forward motion vector, the backward motion vector, and the zero motion vector.

10. A method comprising:

determining a suitable block matching measure for motion compensation, the block matching measure including a block matching error and an overlap; and

synthesizing an interpolated frame based on the suitable block matching measure.

31. The method of claim 10, wherein the block matching error includes a sum of absolute difference.

32. The method of claim 10, wherein the block matching error includes a mean absolute difference.

33. The method of claim 32, further comprising:

calculating a ratio comprising the mean absolute difference and the overlap; and

comparing the ratio to a predetermined threshold.

11. A system comprising:

a block size selector selecting a number of blocks of a frame pair;

a synthesizer synthesizing an interpolated frame based on the selected blocks of the frame pair; and

a mechanism aborting the synthesis of the interpolated frame upon determining the interpolated frame has an unacceptable quality.

12. The system of claim 11 further comprising a video display, wherein the video display receives the frame pair and the interpolated frame from the synthesizer to display the frame pair and the interpolated frame.

34. The system of claim 11, wherein determining the interpolated frame has an unacceptable quality includes determining that a sum of relative portions of covered and uncovered pixels within a block exceeds a predetermined threshold.

13. A system comprising:

a block size selector selecting a block size based on a level of activity for a current frame and a previous frame; and

a synthesizer synthesizing an interpolated frame based on the selected block size of the current frame and the previous frame.

35. The system of claim 13, wherein selecting a block size includes dynamically changing the block size on a per-frame basis.

36. The system of claim 13, wherein selecting a block size includes changing a resolution of a motion field to account for both large and small amounts of motion.

14. A system comprising:

a first mechanism maintaining a number of lists for a number of interpolated blocks of an interpolated frame for determining a best-matched block from a current frame and a previous frame for each interpolated block, wherein each list of the number of lists has a current winning block;

a block size selector selecting the best-matched block for each interpolated block from the current winning block for each list based on an error criterion and an overlap criterion; and

a synthesizer synthesizing the interpolated frame based on the best-matched block for each interpolated block.

15. The system of claim 14 wherein the selection of the best-matched block by the block size selector based on the error criterion and the overlap criterion comprises selection of the best-matched block whose ratio of a block matching error to a corresponding overlap is smallest.

16. The system of claim 15 further comprising:

a second mechanism substituting a zero motion vector for a best motion vector for creation of each interpolated block of the interpolated frame upon determining the corresponding overlap is less than a first predetermined threshold; and

a third mechanism aborting the synthesis of the interpolated frame and repeating a previous frame upon determining a number of interpolated blocks whose corresponding overlap is less than a first predetermined threshold is greater than a second predetermined threshold.

17. A system comprising:

a mechanism selecting a zero motion vector for a given pixel in an interpolated frame upon determining a current pixel in a current frame corresponding to the given pixel in the interpolated frame is classified as covered or uncovered; and

a synthesizer synthesizing the interpolated frame based on the selection of the zero motion vector for the given pixel in the interpolated frame upon determining the current pixel in the current frame corresponding to the given pixel in the interpolated frame is classified as covered or uncovered.

37. The system of claim 17, further comprising:

a mechanism to calculate the zero motion vector capable of being coupled to the mechanism selecting a zero motion vector.

18. A system comprising:

a first mechanism classifying a number of pixels in a current frame into one of a number of different pixel classifications for synthesis of an interpolated frame; and

a second mechanism aborting the synthesis of the interpolated frame and repeating a previous frame upon determining the interpolated frame has an unacceptable quality based on the classification of the number of pixels in the current frame.

39. The system of claim 18, wherein determining the interpolated frame has an unacceptable quality includes marking blocks having a sum of relative portions of covered and uncovered pixels exceeding a predetermined threshold as suspect.

40. The method of claim 39, wherein the number of different pixel classifications includes covered, uncovered, moving, and stationary.

19. A system comprising:

a block size selector selecting a best motion vector for each of a number of blocks for a hypothetical interpolated frame situated temporally in between a current frame and a previous frame;

a mechanism scaling the best motion vector for each of the number of blocks for the hypothetical interpolated frame for a number of interpolated frames a relative distance of the number of interpolated frames from the current frame; and

a synthesizer synthesizing the number of interpolated frames based on the best motion vector for each block within the number of interpolated frames.

41. The system of claim 19, further comprising:

a classification map having a state for each one of a plurality of pixels, the classification map being accessible by the block size selector.

42. The system of claim 41, wherein the classification map has a number of different pixel classifications including covered, uncovered, moving, and stationary.

20. An article comprising a computer readable medium having a computer program stored thereon for execution on a computer with instructions to:

select a block size based on a level of activity for a current frame and a previous frame; and

synthesize an interpolated frame based on the selected block size of the current frame and the previous frame.