Method and apparatus for motion compensated frame rate up conversion for block-based low bit rate video

• US 20050265451A1
• Filed: 05/04/2005
• Published: 12/01/2005
• Est. Priority Date: 05/04/2004
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Shi, Fang, Raveendran, Vijayalakshmi R.
• Granted Patent: US 8,369,405 B2
• Time in Patent Office: Days
• US Class Current: 375/240.150
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51
• CPC Class Codes: H04N 7/014 involving the use of motion vectors motion estimation and compensation in video coding H04N19/51

Abstract

A method and apparatus for video frame interpolation using a current video frame, at least one previous video frame, and a set of transmitted motion vectors is described. A first set of motion vectors is created as a function of the set of transmitted motion vectors. An intermediated video frame is identified, where the intermediate video frame having a plurality of non-overlapping blocks. Each non-overlapping block is assigned with at least one motion vector chosen from the first set of motion vectors to create a set of assigned motion vectors. Then, a second set of motion vectors is created as a function of the assigned set of motion vectors. A video frame is generated using the second set of motion vectors.

Claims

1. At least one processor configured to implement a method for video frame interpolation using a current video frame, at least one previous video frame, and a set of transmitted motion vectors, the method comprising:

creating a first set of motion vectors as a function of the set of transmitted motion vectors;

identifying an intermediate video frame, the intermediate video frame having a plurality of non-overlapping blocks;

assigning each non-overlapping block with at least one motion vector chosen from the first set of motion vectors to create a set of assigned motion vectors;

creating a second set of motion vectors as a function of the assigned set of motion vectors; and

,

generating a video frame using the second set of motion vectors.

2. The at least one processor of claim 1, wherein creating the first set of motion vectors as a function of the set of transmitted motion vectors comprises:

partitioning the set of transmitted motion vectors into a first portion of motion vectors and a second portion of motion vectors; and

,

modifying the first portion of motion vectors as a function of the second portion of motion vectors.

3. The at least one processor of claim 1, wherein assigning each non-overlapping block with at least one motion vector comprises:

determining a seeded motion vector; and

,

performing a motion vector search based on the seeded motion vector.

4. The at least one processor of claim 3, wherein determining the seeded motion vector comprises locating a motion vector passing through the center of the non-overlapping block.

5. The at least one processor of claim 3, wherein performing the motion vector search based on the seeded motion vector comprises performing a bi-directional motion vector search.

6. The at least one processor of claim 5, wherein performing the bi-directional motion vector search comprises:

constructing a previous video frame search block in the at least one previous video frame;

assigning a first search range to a portion of the current video frame; and

,

performing a search in the first search range for a first matching block to the previous video frame search block based on a first predetermined criteria.

7. The at least one processor of claim 6, wherein performing the bi-directional motion vector search comprises:

constructing a current video frame search block in the current video frame;

assigning a second search range to a portion of the at least one previous video frame; and

,

performing a search in the second search range for a second matching block to the current video frame search block based on a second predetermined criteria.

8. The at least one processor of claim 6, wherein performing the bi-directional motion vector search comprises locating a first motion vector based on the first matching block.

15. The at least one processor of claim 1, wherein the set of transmitted motion vectors comprise a plurality of current frame motion vectors and a plurality of previous frame motion vectors.

9. The at least one processor of claim 9, wherein the first predetermined criteria is based on a distortion metric.

10. The at least one processor of claim 9, wherein the first predetermined criteria is based on minimizing the distortion metric.

11. The at least one processor of claim 9, wherein the distortion metric is based on a sum of absolute differences between the first matching block and the previous video frame search block.

12. The at least one processor of claim 9, wherein the distortion metric is based on a sum of squared differences between the first matching block and the previous video frame search block.

13. The at least one processor of claim 9, wherein the distortion metric is based on a human visual system-based metric.

14. The at least one processor of claim 9, wherein the distortion metric is based on a statistical.

16. A method for video frame interpolation using a current video frame, at least one previous video frame, and a set of transmitted motion vectors, the method comprising:

creating a first set of motion vectors as a function of the set of transmitted motion vectors;

identifying an intermediate video frame, the intermediate video frame having a plurality of non-overlapping blocks;

assigning each non-overlapping block with at least one motion vector chosen from the first set of motion vectors to create a set of assigned motion vectors;

creating a second set of motion vectors as a function of the assigned set of motion vectors; and

,

generating a video frame using the second set of motion vectors.

17. The method for video frame interpolation of claim 16, wherein creating the first set of motion vectors as a function of the set of transmitted motion vectors comprises:

partitioning the set of transmitted motion vectors into a first portion of motion vectors and a second portion of motion vectors; and

,

modifying the first portion of motion vectors as a function of the second portion of motion vectors.

18. The method for video frame interpolation of claim 16, wherein assigning each non-overlapping block with at least one motion vector comprises:

determining a seeded motion vector; and

,

performing a motion vector search based on the seeded motion vector.

19. The method for video frame interpolation of claim 18, wherein determining the seeded motion vector comprises locating a motion vector passing through the center of the non-overlapping block.

20. The method for video frame interpolation of claim 18, wherein performing the motion vector search based on the seeded motion vector comprises performing a bi-directional motion vector search.

21. The method for video frame interpolation of claim 20, wherein performing the bidirectional motion vector search comprises:

constructing a previous video frame search block in the at least one previous video frame;

assigning a first search range to a portion of the current video frame; and

,

performing a search in the first search range for a first matching block to the previous video frame search block based on a first predetermined criteria.

22. The method for video frame interpolation of claim 21, wherein performing the bidirectional motion vector search comprises:

constructing a current video frame search block in the current video frame;

assigning a second search range to a portion of the at least one previous video frame; and

,

performing a search in the second search range for a second matching block to the current video frame search block based on a second predetermined criteria.

23. The method for video frame interpolation of claim 21, wherein performing the bi-directional motion vector search comprises locating a first motion vector based on the first matching block.

24. The method for video frame interpolation of claim 21, wherein the first predetermined criteria is based on a distortion metric.

25. The method for video frame interpolation of claim 24, wherein the first predetermined criteria is based on minimizing the distortion metric.

26. The method for video frame interpolation of claim 24, wherein the distortion metric is based on a sum of absolute differences between the first matching block and the previous video frame search block.

27. The method for video frame interpolation of claim 24, wherein the distortion metric is based on a sum of squared differences between the first matching block and the previous video frame search block.

28. The method for video frame interpolation of claim 24, wherein the distortion metric is based on a human visual system-based metric.

29. The method for creating the interpolated video frame of claim 24, wherein the distortion metric is based on a statistical.

30. The method for creating the interpolated video frame of claim 16, wherein the set of transmitted motion vectors comprise a plurality of current frame motion vectors and a plurality of previous frame motion vectors.

31. A computer-readable medium having instructions stored thereon, the stored instructions, when executed by a processor, cause the processor to perform a method for video frame interpolation using a current video frame, at least one previous video frame, and a set of transmitted motion vectors, the method comprising:

creating a first set of motion vectors as a function of the set of transmitted motion vectors;

identifying an intermediate video frame, the intermediate video frame having a plurality of non-overlapping blocks;

assigning each non-overlapping block with at least one motion vector chosen from the first set of motion vectors to create a set of assigned motion vectors;

creating a second set of motion vectors as a function of the assigned set of motion vectors; and

,

generating a video frame using the second set of motion vectors.

32. The computer-readable medium of claim 31, wherein creating the first set of motion vectors as a function of the set of transmitted motion vectors comprises:

partitioning the set of transmitted motion vectors into a first portion of motion vectors and a second portion of motion vectors; and

,

modifying the first portion of motion vectors as a function of the second portion of motion vectors.

33. The computer-readable medium of claim 31, wherein assigning each non-overlapping block with at least one motion vector comprises:

determining a seeded motion vector; and

,

performing a motion vector search based on the seeded motion vector.

34. The computer-readable medium of claim 33, wherein determining the seeded motion vector comprises locating a motion vector passing through the center of the non-overlapping block.

35. The computer-readable medium of claim 33, wherein performing the motion vector search based on the seeded motion vector comprises performing a bi-directional motion vector search.

36. The computer-readable medium of claim 35, wherein performing the bi-directional motion vector search comprises:

constructing a previous video frame search block in the at least one previous video frame;

assigning a first search range to a portion of the current video frame; and

,

performing a search in the first search range for a first matching block to the previous video frame search block based on a first predetermined criteria.

37. The computer-readable medium of claim 36, wherein performing the bi-directional motion vector search comprises:

constructing a current video frame search block in the current video frame;

assigning a second search range to a portion of the at least one previous video frame; and

,

performing a search in the second search range for a second matching block to the current video frame search block based on a second predetermined criteria.

38. The computer-readable medium of claim 36, wherein performing the bi-directional motion vector search comprises locating a first motion vector based on the first matching block.

39. The computer-readable medium of claim 36, wherein the first predetermined criteria is based on a distortion metric.

40. The computer-readable medium of claim 39, wherein the first predetermined criteria is based on minimizing the distortion metric.

41. The computer-readable medium of claim 39, wherein the distortion metric is based on a sum of absolute differences between the first matching block and the previous video frame search block.

42. The computer-readable medium of claim 39, wherein the distortion metric is based on a sum of squared differences between the first matching block and the previous video frame search block.

43. The computer-readable medium of claim 39, wherein the distortion metric is based on a human visual system-based metric.

44. The computer-readable medium of claim 39, wherein the distortion metric is based on a statistical.

45. The computer-readable medium of claim 31, wherein the set of transmitted motion vectors comprise a plurality of current frame motion vectors and a plurality of previous frame motion vectors.

46. An apparatus for video frame interpolation using a current video frame, at least one previous video frame, and a set of transmitted motion vectors, comprising:

means for creating a first set of motion vectors as a function of the set of transmitted motion vectors;

means for identifying an intermediate video frame, the intermediate video frame having a plurality of non-overlapping blocks;

means for assigning each non-overlapping block with at least one motion vector chosen from the first set of motion vectors to create a set of assigned motion vectors;

means for creating a second set of motion vectors as a function of the assigned set of motion vectors; and

,

means for generating a video frame using the second set of motion vectors.

47. The apparatus for video frame interpolation of claim 46, wherein the means for creating the first set of motion vectors as a function of the set of transmitted motion vectors comprises:

means for partitioning the set of transmitted motion vectors into a first portion of motion vectors and a second portion of motion vectors; and

,

means for modifying the first portion of motion vectors as a function of the second portion of motion vectors.

48. The apparatus for video frame interpolation of claim 46, wherein the means for assigning each non-overlapping block with at least one motion vector comprises:

means for determining a seeded motion vector; and

,

means for performing a motion vector search based on the seeded motion vector.

49. The apparatus for video frame interpolation of claim 48, wherein the means for determining the seeded motion vector comprises means for locating a motion vector passing through the center of the non-overlapping block.

50. The apparatus for video frame interpolation of claim 48, wherein the means for performing the motion vector search based on the seeded motion vector comprises means for performing a bi-directional motion vector search.

51. The apparatus for video frame interpolation of claim 50, wherein the means for performing the bi-directional motion vector search comprises:

means for constructing a previous video frame search block in the at least one previous video frame;

means for assigning a first search range to a portion of the current video frame; and

,

means for performing a search in the first search range for a first matching block to the previous video frame search block based on a first predetermined criteria.

52. The apparatus for video frame interpolation of claim 51, wherein the means for performing the bi-directional motion vector search comprises:

means for constructing a current video frame search block in the current video frame;

means for assigning a second search range to a portion of the at least one previous video frame; and

,

means for performing a search in the second search range for a second matching block to the current video frame search block based on a second predetermined criteria.

53. The apparatus for video frame interpolation of claim 51, wherein the means for performing the bi-directional motion vector search comprises means for locating a first motion vector based on the first matching block.

60. The apparatus for creating the interpolated video frame of claim 46, wherein the set of transmitted motion vectors comprise a plurality of current frame motion vectors and a plurality of previous frame motion vectors.

54. The apparatus for video frame interpolation of claim 54, wherein the first predetermined criteria is based on a distortion metric.

55. The apparatus for video frame interpolation of claim 54, wherein the first predetermined criteria is based on minimizing the distortion metric.

56. The apparatus for video frame interpolation of claim 54, wherein the distortion metric is based on a sum of absolute differences between the first matching block and the previous video frame search block.

57. The apparatus for video frame interpolation of claim 54, wherein the distortion metric is based on a sum of squared differences between the first matching block and the previous video frame search block.

58. The apparatus for video frame interpolation of claim 54, wherein the distortion metric is based on a human visual system-based metric.

59. The apparatus for creating the interpolated video frame of claim 54, wherein the distortion metric is based on a statistical.