APPARATUS AND METHOD OF ENHANCED FRAME INTERPOLATION IN VIDEO COMPRESSION

• US 20070230578A1
• Filed: 11/10/2006
• Published: 10/04/2007
• Est. Priority Date: 04/04/2006
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Shi, Fang, Raveendran, Vijayalakshmi R.
• Granted Patent: US 8,634,463 B2
• Time in Patent Office: Days
• US Class Current: 375/240.160
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/31 in the temporal domain
• CPC Class Codes: H04N 19/573 Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence

Abstract

Aspects include systems and methods of improving processing in an encoder in a multimedia transmission system. Other aspects include method and systems of processing in a decoder in a multimedia transmission system. For example, one aspect comprises a method and apparatus of reduced encoding size in an encoder by encoding in terms of an interpolated frame. Other aspects include decoders configured to decode video signals encoded with reference to an interpolated frame.

Claims

1. A method of processing multimedia data, the method comprising:

generating at least one interpolated frame;

identifying a plurality of reference frames comprising said interpolated frame; and

encoding at least a portion of a frame based at least on the plurality of reference frames.

2. The method of claim 1, wherein the plurality of reference frames comprises at least one list of reference frames, and wherein said at least one interpolated frame is at a specified position in said list.

3. The method of claim 2, wherein said specified position is at an end of the at least one list of the reference frames.

4. The method of claim 1, wherein encoding comprises predicting the portion based at least in part on said interpolated frame.

5. The method of claim 4, wherein predicting comprises generating at least one of a motion vector or a residual based on said interpolated frame.

6. The method of claim 1, wherein encoding comprises generating encoding data in accordance with a standard compliant bit stream syntax.

7. The method of claim 6, wherein the standard compliant bitstream syntax comprises an H.264 compliant bitstream syntax.

8. The method of claim 1, wherein encoding comprises generating encoding data having a field that is indicative of either an interpolated or non-interpolated reference frame.

9. The method of claim 1, wherein encoding comprises generating encoding data indicative of an encoding mode and a residual pattern, wherein said encoding data is further indicative of whether the portion is encoded based on the interpolated frame.

10. The method of claim 9, wherein the plurality of reference frames comprise a sequence of reference frames, wherein the interpolated frame is at a specified position in said sequence, and wherein said encoding data is indicative of the specified position in said sequence.

11. The method of claim 1, wherein generating comprises generating the interpolated frame based on at least one predictive frame.

12. The method of claim 1, wherein generating comprises generating an upconversion frame.

13. The method of claim 1, wherein generating comprises generating the interpolated frame at substantially the time associated with the encoding of the portion of the frame.

14. An apparatus for processing multimedia data, comprising:

a generator configured to generate at least one interpolated frame;

an identifier configured to identify a plurality of reference frames comprising said interpolated frame; and

an encoder configured to encode at least a portion of a frame based at least on the plurality of reference frames.

15. The apparatus of claim 14, wherein the plurality of reference frames comprise at least one list of reference frames, and wherein said at least one interpolated frame is at a specified position in said list.

16. The apparatus of claim 15, wherein said specified position is at an end of the at least one list of the reference frames.

17. The apparatus of claim 14, wherein said encoder is configured to predict the portion based at least in part on said interpolated frame.

18. The apparatus of claim 17, wherein said encoder is configured to generate at least one of a motion vector or a residual based on said interpolated frame.

19. The apparatus of claim 14, wherein said encoder is configured to generate encoding data in accordance with a standard compliant bit stream syntax.

20. The apparatus of claim 19, wherein the standard compliant bitstream syntax comprises a H.264 compliant bitstream syntax.

21. The apparatus of claim 14, wherein said encoder is configured to generate encoding data having a field that is indicative of either an interpolated or non-interpolated reference frame.

22. The apparatus of claim 14, wherein said encoder is configured to generate encoding data indicative of an encoding mode and a residual pattern, wherein said encoding data is further indicative of whether the portion is encoded based on the interpolated frame.

23. The apparatus of claim 22, wherein the plurality of reference frames comprise a sequence of reference frames, wherein the interpolated frame is at a specified position in said sequence, and wherein said encoding data is indicative of the specified position in said sequence.

24. The apparatus of claim 14, wherein said generator is configured to generate the interpolated frame based on at least one predictive frame.

25. The apparatus of claim 14, wherein said generator is configured to generate an upconversion frame.

26. The apparatus of claim 14, wherein said generator is configured to generate the interpolated frame at substantially the time associated with the encoding of the portion of the frame.

27. An apparatus for processing multimedia data, comprising:

means for generating at least one interpolated frame;

means for identifying a plurality of reference frames comprising said interpolated frame; and

means for encoding at least a portion of a frame based at least on the plurality of reference frames.

28. The apparatus of claim 27, wherein the plurality of reference frames comprise at least one list of reference frames, and wherein said at least one interpolated frame is at a specified position in said list.

29. The apparatus of claim 28, wherein said specified position is at an end of the at least one list of the reference frames.

30. The apparatus of claim 27, wherein said encoding means is configured to predict the portion based at least in part on said interpolated frame.

31. The apparatus of claim 30, wherein said encoding means is configured to generate at least one of a motion vector or a residual based on said interpolated frame.

32. The apparatus of claim 27, wherein said encoding means is configured to generate encoding data in accordance with a standard compliant bit stream syntax.

33. The apparatus of claim 32, wherein the standard compliant bitstream syntax comprises a H.264 compliant bitstream syntax.

34. The apparatus of claim 27, wherein said encoding means is configured to generate encoding data having a field that is indicative of either an interpolated or non-interpolated reference frame.

35. The apparatus of claim 27, wherein said encoding is configured to generate encoding data indicative of an encoding mode and a residual pattern, wherein said encoding data is further indicative of whether the portion is encoded based on the interpolated frame.

36. The apparatus of claim 35, wherein the plurality of reference frames comprise a sequence of reference frames, wherein the interpolated frame is at a specified position in said sequence, and wherein said encoding data is indicative of the specified position in said sequence.

37. The apparatus of claim 27, wherein said generating means is configured to generate the interpolated frame based on at least one predictive frame.

38. The apparatus of claim 27, wherein said generating means is configured to generate an upconversion frame.

39. The apparatus of claim 27, wherein said generating means is configured to generate the interpolated frame at substantially the time associated with the encoding of the portion of the frame.

40. A multimedia data processor comprising a configuration to:

generate at least one interpolated frame;

identify a plurality of reference frames comprising said interpolated frame; and

encode at least a portion of a frame based at least on the plurality of reference frames.

41. A machine readable medium comprising instructions for processing multimedia data, wherein the instructions upon execution cause a machine to:

generate at least one interpolated frame;

identify a plurality of reference frames comprising said interpolated frame; and

encode at least a portion of a frame based at least on the plurality of reference frames.

42. A method of processing multimedia data, the method comprising:

receiving encoded data indicative of an interpolated reference frame for at least a portion of a frame;

generating at least a portion of said interpolated reference frame; and

decoding at least a portion of the frame based at least in part on the generated portion of the interpolated reference frame.

43. The method of claim 42, wherein receiving comprises identifying an encoded data field that is indicative of an interpolated reference frame.

44. The method of claim 42, wherein generating comprises generating at least a portion of a frame rate upconversion frame.

45. The method of claim 42, wherein generating comprises generating based on at least one predictive frame.

46. An apparatus for processing multimedia data, comprising:

a receiver configured to receive encoded data indicative of an interpolated reference frame for at least a portion of a frame;

a generator configured to generate at least a portion of said interpolated reference frame; and

a decoder configured to decode at least a portion of the frame based at least in part on the generated portion of the interpolated reference frame.

47. The apparatus of claim 46, wherein said receiver is further configured to identify an encoded data field that is indicative of an interpolated reference frame.

48. The apparatus of claim 46, wherein said generator is configured to generate at least a portion of a frame rate upconversion frame.

49. The apparatus of claim 46, wherein said generator is configured to generate said interpolated frame based on at least one predictive frame.

51. The apparatus of claim 46, wherein said receiving means is further configured to identify an encoded data field that is indicative of an interpolated reference frame.

52. The apparatus of claim 46, wherein said generating means is configured to generate at least a portion of a frame rate upconversion frame.

53. The apparatus of claim 46, wherein said generating means is configured to generate said interpolated frame based on at least one predictive frame.

50. An apparatus for processing multimedia data, comprising:

means for receiving encoded data indicative of an interpolated reference frame for at least a portion of a frame;

means for generating at least a portion of said interpolated reference frame; and

means for decoding at least a portion of the frame based at least in part on the generated portion of the interpolated reference frame.

54. A multimedia data processor comprising a configuration to:

receive encoded data indicative of an interpolated reference frame for at least a portion of a frame;

generate at least a portion of said interpolated reference frame; and

decode at least a portion of the frame based at least in part on the generated portion of the interpolated reference frame.

55. A machine readable medium comprising instructions for processing multimedia data, wherein the instructions upon execution cause a machine to:

receive encoded data indicative of an interpolated reference frame for at least a portion of a frame;

generate at least a portion of said interpolated reference frame; and

decode at least a portion of the frame based at least in part on the generated portion of the interpolated reference frame.