Method and apparatus for motion compensated frame rate up conversion for block-based low bit rate video

• US 8,369,405 B2
• Filed: 05/04/2005
• Issued: 02/05/2013
• Est. Priority Date: 05/04/2004
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Shi, Fang, Raveendran, Vijayalakshmi R.
• Primary Examiner(s): WONG, ALLEN C
• Application Number:
  US11/122,678
• Publication Number:
  US 20050265451A1
• Time in Patent Office: 2,834 Days
• Field of Search: 375/240, 375/240.01, 375/240.12, 375/240.15, 375/240.16, 382/240
• US Class Current: 375/240.16
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51
• CPC Class Codes: H04N 7/014 involving the use of motion vectors motion estimation and compensation in video coding H04N19/51

Abstract

A method and apparatus for video frame interpolation using a current video frame, at least one previous video frame, and a set of transmitted motion vectors is described. A first set of motion vectors is created as a function of the set of transmitted motion vectors. An intermediated video frame is identified, where the intermediate video frame having a plurality of non-overlapping blocks. Each non-overlapping block is assigned with at least one motion vector chosen from the first set of motion vectors to create a set of assigned motion vectors. Then, a second set of motion vectors is created as a function of the assigned set of motion vectors. A video frame is generated using the second set of motion vectors.

Claims

1. At least one processor configured to implement a method in a decoder for video frame interpolation using a current video frame, at least one previous video frame, and a set of transmitted motion vectors, the method comprising:

creating a first set of motion vectors as a function of the set of transmitted motion vectors;

identifying an intermediate video frame, the entire intermediate video frame having a plurality of non-overlapping blocks;

assigning each non-overlapping block with at least one motion vector chosen from the first set of motion vectors to create a set of assigned motion vectors, the assigning comprising;

determining a seeded motion vector passing through the non-overlapping block; and

performing a motion vector search based on the seeded motion vector;

creating a second set of motion vectors as a function of the assigned set of motion vectors; and

,

generating an interpolated video frame using the second set of motion vectors.

2. The at least one processor of claim 1, wherein creating the first set of motion vectors as a function of the set of transmitted motion vectors comprises:

partitioning the set of transmitted motion vectors into a first portion of motion vectors and a second portion of motion vectors; and

,

modifying the first portion of motion vectors as a function of the second portion of motion vectors.

3. The at least one processor of claim 1, wherein determining the seeded motion vector comprises locating a motion vector passing through the center of the non-overlapping block.

4. The at least one processor of claim 1, wherein performing the motion vector search based on the seeded motion vector comprises performing a bi-directional motion vector search.

5. The at least one processor of claim 4, wherein performing the bi-directional motion vector search comprises:

constructing a previous video frame search block in the at least one previous video frame;

assigning a first search range to a portion of the current video frame; and

,

performing a search in the first search range for a first matching block to the previous video frame search block based on a first predetermined criteria.

6. The at least one processor of claim 5, wherein performing the bi-directional motion vector search comprises:

constructing a current video frame search block in the current video frame;

assigning a second search range to a portion of the at least one previous video frame; and

,

performing a search in the second search range for a second matching block to the current video frame search block based on a second predetermined criteria.

7. The at least one processor of claim 5, wherein performing the bi-directional motion vector search comprises locating a first motion vector based on the first matching block.

8. The at least one processor of claim 5, wherein the first predetermined criteria is based on a distortion metric.

9. The at least one processor of claim 8, wherein the first predetermined criteria is based on minimizing the distortion metric.

10. The at least one processor of claim 8, wherein the distortion metric is based on a sum of absolute differences between the first matching block and the previous video frame search block.

11. The at least one processor of claim 8, wherein the distortion metric is based on a sum of squared differences between the first matching block and the previous video frame search block.

12. The at least one processor of claim 8, wherein the distortion metric is based on a human visual system-based metric.

13. The at least one processor of claim 8, wherein the distortion metric is based on a statistical.

14. The at least one processor of claim 1, wherein the set of transmitted motion vectors comprise a plurality of current frame motion vectors and a plurality of previous frame motion vectors.

15. A method, implemented in a decoder, for video frame interpolation using a current video frame, at least one previous video frame, and a set of transmitted motion vectors, the method comprising:

creating a first set of motion vectors as a function of the set of transmitted motion vectors;

identifying an intermediate video frame, the entire intermediate video frame having a plurality of non-overlapping blocks;

assigning each non-overlapping block with at least one motion vector chosen from the first set of motion vectors to create a set of assigned motion vectors, the assigning comprising;

determining a seeded motion vector passing through the non-overlapping block; and

performing a motion vector search based on the seeded motion vector;

creating a second set of motion vectors as a function of the assigned set of motion vectors; and

,

generating an interpolated video frame using the second set of motion vectors.

16. The method for video frame interpolation of claim 15, wherein creating the first set of motion vectors as a function of the set of transmitted motion vectors comprises:

partitioning the set of transmitted motion vectors into a first portion of motion vectors and a second portion of motion vectors; and

,

modifying the first portion of motion vectors as a function of the second portion of motion vectors.

17. The method for video frame interpolation of claim 15, wherein determining the seeded motion vector comprises locating a motion vector passing through the center of the non-overlapping block.

18. The method for video frame interpolation of claim 15, wherein performing the motion vector search based on the seeded motion vector comprises performing a bi-directional motion vector search.

19. The method for video frame interpolation of claim 18, wherein performing the bidirectional motion vector search comprises:

constructing a previous video frame search block in the at least one previous video frame;

assigning a first search range to a portion of the current video frame; and

,

performing a search in the first search range for a first matching block to the previous video frame search block based on a first predetermined criteria.

20. The method for video frame interpolation of claim 19, wherein performing the bidirectional motion vector search comprises:

constructing a current video frame search block in the current video frame;

assigning a second search range to a portion of the at least one previous video frame; and

,

performing a search in the second search range for a second matching block to the current video frame search block based on a second predetermined criteria.

21. The method for video frame interpolation of claim 19, wherein performing the bi-directional motion vector search comprises locating a first motion vector based on the first matching block.

22. The method for video frame interpolation of claim 19, wherein the first predetermined criteria is based on a distortion metric.

23. The method for video frame interpolation of claim 22, wherein the first predetermined criteria is based on minimizing the distortion metric.

24. The method for video frame interpolation of claim 22, wherein the distortion metric is based on a sum of absolute differences between the first matching block and the previous video frame search block.

25. The method for video frame interpolation of claim 22, wherein the distortion metric is based on a sum of squared differences between the first matching block and the previous video frame search block.

26. The method for video frame interpolation of claim 22, wherein the distortion metric is based on a human visual system-based metric.

27. The method for creating the interpolated video frame of claim 22, wherein the distortion metric is based on a statistical.

28. The method for creating the interpolated video frame of claim 15, wherein the set of transmitted motion vectors comprise a plurality of current frame motion vectors and a plurality of previous frame motion vectors.

29. A non-transitory computer-readable medium having computer executable instructions stored thereon, the stored instructions, when executed by a processor, cause the processor to perform a method for video frame interpolation using a current video frame, at least one previous video frame, and a set of transmitted motion vectors, the method comprising:

creating a first set of motion vectors as a function of the set of transmitted motion vectors;

identifying an intermediate video frame, the intermediate video frame having a plurality of non-overlapping blocks;

assigning each non-overlapping block with at least one motion vector chosen from the first set of motion vectors to create a set of assigned motion vectors, the assigning comprising;

determining a seeded motion vector passing through the non-overlapping block; and

performing a motion vector search based on the seeded motion vector;

creating a second set of motion vectors as a function of the assigned set of motion vectors; and

,

generating an interpolated video frame using the second set of motion vectors.

30. The non-transitory computer-readable medium of claim 29, wherein creating the first set of motion vectors as a function of the set of transmitted motion vectors comprises:

partitioning the set of transmitted motion vectors into a first portion of motion vectors and a second portion of motion vectors; and

,

modifying the first portion of motion vectors as a function of the second portion of motion vectors.

31. The non-transitory computer-readable medium of claim 29, wherein determining the seeded motion vector comprises locating a motion vector passing through the center of the non-overlapping block.

32. The non-transitory computer-readable medium of claim 29, wherein performing the motion vector search based on the seeded motion vector comprises performing a bi-directional motion vector search.

33. The non-transitory computer-readable medium of claim 32, wherein performing the bi-directional motion vector search comprises:

constructing a previous video frame search block in the at least one previous video frame;

assigning a first search range to a portion of the current video frame; and

,

performing a search in the first search range for a first matching block to the previous video frame search block based on a first predetermined criteria.

34. The non-transitory computer-readable medium of claim 33, wherein performing the bi-directional motion vector search comprises:

constructing a current video frame search block in the current video frame;

assigning a second search range to a portion of the at least one previous video frame; and

,

performing a search in the second search range for a second matching block to the current video frame search block based on a second predetermined criteria.

35. The non-transitory computer-readable medium of claim 33, wherein performing the bi-directional motion vector search comprises locating a first motion vector based on the first matching block.

36. The non-transitory computer-readable medium of claim 33, wherein the first predetermined criteria is based on a distortion metric.

37. The non-transitory computer-readable medium of claim 36, wherein the first predetermined criteria is based on minimizing the distortion metric.

38. The non-transitory computer-readable medium of claim 36, wherein the distortion metric is based on a sum of absolute differences between the first matching block and the previous video frame search block.

39. The non-transitory computer-readable medium of claim 36, wherein the distortion metric is based on a sum of squared differences between the first matching block and the previous video frame search block.

40. The non-transitory computer-readable medium of claim 36, wherein the distortion metric is based on a human visual system-based metric.

41. The non-transitory computer-readable medium of claim 36, wherein the distortion metric is based on a statistical.

42. The non-transitory computer-readable medium of claim 29, wherein the set of transmitted motion vectors comprise a plurality of current frame motion vectors and a plurality of previous frame motion vectors.

43. An apparatus for video frame interpolation using a current video frame, at least one previous video frame, and a set of transmitted motion vectors, comprising:

means for creating a first set of motion vectors as a function of the set of transmitted motion vectors;

means for identifying an intermediate video frame, the entire intermediate video frame having a plurality of non-overlapping blocks;

means for assigning each non-overlapping block with at least one motion vector chosen from the first set of motion vectors to create a set of assigned motion vectors, the assigning means comprising;

means for determining a seeded motion vector passing through the non-overlapping block; and

means for performing a motion vector search based on the seeded motion vector;

means for creating a second set of motion vectors as a function of the assigned set of motion vectors; and

,

means for generating an interpolated video frame using the second set of motion vectors.

44. The apparatus for video frame interpolation of claim 43, wherein the means for creating the first set of motion vectors as a function of the set of transmitted motion vectors comprises:

means for partitioning the set of transmitted motion vectors into a first portion of motion vectors and a second portion of motion vectors; and

,

means for modifying the first portion of motion vectors as a function of the second portion of motion vectors.

45. The apparatus for video frame interpolation of claim 43, wherein the means for determining the seeded motion vector comprises means for locating a motion vector passing through the center of the non-overlapping block.

46. The apparatus for video frame interpolation of claim 43, wherein the means for performing the motion vector search based on the seeded motion vector comprises means for performing a bi-directional motion vector search.

47. The apparatus for video frame interpolation of claim 46, wherein the means for performing the bi-directional motion vector search comprises:

means for constructing a previous video frame search block in the at least one previous video frame;

means for assigning a first search range to a portion of the current video frame; and

,

means for performing a search in the first search range for a first matching block to the previous video frame search block based on a first predetermined criteria.

48. The apparatus for video frame interpolation of claim 47, wherein the means for performing the bi-directional motion vector search comprises:

means for constructing a current video frame search block in the current video frame;

means for assigning a second search range to a portion of the at least one previous video frame; and

,

means for performing a search in the second search range for a second matching block to the current video frame search block based on a second predetermined criteria.

49. The apparatus for video frame interpolation of claim 47, wherein the means for performing the bi-directional motion vector search comprises means for locating a first motion vector based on the first matching block.

50. The apparatus for video frame interpolation of claim 47, wherein the first predetermined criteria is based on a distortion metric.

51. The apparatus for video frame interpolation of claim 50, wherein the first predetermined criteria is based on minimizing the distortion metric.

52. The apparatus for video frame interpolation of claim 50, wherein the distortion metric is based on a sum of absolute differences between the first matching block and the previous video frame search block.

53. The apparatus for video frame interpolation of claim 50, wherein the distortion metric is based on a sum of squared differences between the first matching block and the previous video frame search block.

54. The apparatus for video frame interpolation of claim 50, wherein the distortion metric is based on a human visual system-based metric.

55. The apparatus for creating the interpolated video frame of claim 50, wherein the distortion metric is based on a statistical.

56. The apparatus for creating the interpolated video frame of claim 43, wherein the set of transmitted motion vectors comprise a plurality of current frame motion vectors and a plurality of previous frame motion vectors.