Methods and apparatus using virtual reference pictures

• US 8,363,724 B2
• Filed: 07/10/2007
• Issued: 01/29/2013
• Est. Priority Date: 07/11/2006
• Status: Active Grant

• Current Assignee: Interdigital Vc Holdings, Inc. (InterDigital, Inc.)
• Original Assignee: Thomson Licensing (Vantiva SA)
• Inventors: Su, Yeping, Yin, Peng, Pandit, Purvin Bibhas, Gomila, Cristina
• Primary Examiner(s): MILLS, DONALD L
• Application Number:
  US12/309,066
• Publication Number:
  US 20110243224A1
• Time in Patent Office: 2,030 Days
• Field of Search: None
• US Class Current: 375/240.15
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/423 characterised by memory arrangements H04N19/433 takes precedence
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence

Abstract

There are provided methods and apparatus using virtual reference pictures. An apparatus includes an encoder for encoding at least one picture, using at least one virtual reference picture, to form a resultant bitstream. The at least one virtual reference picture is different than a previously decoded picture.

Claims

1. An apparatus, comprising:

an encoder for encoding at least one picture, using at least one virtual reference picture, to form a resultant bitstream, wherein the at least one virtual reference picture is different than a previously decoded picture, and the at least one virtual reference picture is stored in a decoded picture buffer that also stores non-virtual reference pictures.

2. The apparatus of claim 1, wherein the at least one picture corresponds to at least one of at least two views of multi-view video content.

3. The apparatus of claim 1, wherein the at least one virtual reference picture is formed from at least one reference picture that has undergone a signal processing transformation.

4. The apparatus of claim 3, wherein said encoder signals warping parameters of the signal processing transformation in at least one high level syntax included in the resultant bitstream.

5. The apparatus of claim 1, wherein the at least one virtual reference picture is formed from at least one filtered reference picture.

6. The apparatus of claim 5, wherein said encoder signals filter coefficients used to filter the at least one filtered reference picture in at least one high level syntax included in the resultant bitstream.

7. The apparatus of claim 1, wherein the at least one picture corresponds to multi-view video content, and the at least one virtual reference picture is formed using at least one view interpolated picture.

8. The apparatus of claim 7, wherein said encoder signals parameters for the at least one view interpolated picture in at least one high level syntax element included in the resultant bitstream.

9. The apparatus of claim 1, wherein said encoder signals an existence of the at least one virtual reference picture in the resultant bitstream.

10. The apparatus of claim 1, wherein said encoder signals an existence of the at least one virtual reference picture in a high-level syntax in the resultant bitstream.

11. The apparatus of claim 1, wherein at least one of a method and a parameter for creating the at least one virtual reference picture is signaled in at least one high level syntax element included in the resultant bitstream.

12. The apparatus of claim 1, wherein said encoder utilizes pre-specified insertion and deletion operations for the at least one virtual reference picture with respect to the decoded picture buffer.

13. The apparatus of claim 1, wherein the at least one virtual reference picture is stored in a short-term memory portion of the decoded picture buffer.

14. The apparatus of claim 1, wherein the at least one virtual reference picture is stored in a long-term memory portion of the decoded picture buffer.

15. The apparatus of claim 1, wherein said encoder directly refers to the at least one virtual reference picture in a prediction process for the at least one picture, using an index of the at least one virtual reference picture.

16. The apparatus of claim 15, wherein a default reference list construction process is performed to include the at least one virtual reference picture.

17. The apparatus of claim 16, wherein at least one reference picture list reordering command is defined to include the at least one virtual reference picture in the default reference list construction.

18. The apparatus of claim 16, wherein a pre-specified reference list reordering command is combined with a high level syntax to include the at least one virtual reference picture in the default reference list construction.

19. The apparatus of claim 1, wherein each of the at least one virtual reference picture is respectively referred to by a reference picture index corresponding to a reference picture list.

76. The apparatus of claim 1, wherein dedicated memory slots are used in the decoded picture buffer to store virtual reference pictures including the virtual picture, and wherein each of the virtual reference pictures are identified by a virtual reference picture identifier that is unique for each of the virtual reference pictures.

80. The apparatus of claim 13, wherein the at least one virtual reference picture and any other virtual reference pictures are stored in the short-term memory portion of the decoded picture buffer using unused frame numbers and unused picture order counts to differentiate the virtual reference pictures from the non-virtual reference pictures.

84. The apparatus of claim 14, wherein the at least one virtual reference picture and any other virtual reference pictures are stored in the long-term memory portion of the decoded picture buffer using unused long term identifiers to differentiate the virtual reference pictures from the non-virtual reference pictures.

20. In a video encoder, a method, comprising:

encoding at least one picture, using at least one virtual reference picture, to form a resultant bitstream, wherein the at least one virtual reference picture is different than a previously decoded picture, and the at least one virtual reference picture is stored in a decoded picture buffer that also stores non-virtual reference pictures.

21. The method of claim 20, wherein the at least one picture corresponds to at least one of at least two views of multi-view video content.

22. The method of claim 20, wherein the at least one virtual reference picture is formed from at least one reference picture that has undergone a signal processing transformation.

23. The method of claim 22, wherein said encoding step comprises signaling warping parameters of the signal processing transformation in at least one high level syntax included in the resultant bitstream.

24. The method of claim 20, wherein the at least one virtual reference picture is formed from at least one filtered reference picture.

25. The method of claim 24, wherein said encoding step comprises signaling filter coefficients used to filter the at least one filtered reference picture in at least one high level syntax included in the resultant bitstream.

26. The method of claim 20, wherein the at least one picture corresponds to multi-view video content, and the at least one virtual reference picture is formed using at least one view interpolated picture.

27. The method of claim 26, wherein said encoding step comprises signaling parameters for the at least one view interpolated picture in at least one high level syntax element included in the resultant bitstream.

28. The method of claim 20, wherein said encoding step comprises signaling an existence of the at least one virtual reference picture in the resultant bitstream.

29. The method of claim 20, wherein said encoding step comprises signaling an existence of the at least one virtual reference picture in a high-level syntax in the resultant bitstream.

30. The method of claim 20, wherein said encoding step comprises signaling at least one of a method and a parameter for creating the at least one virtual reference picture in at least one high level syntax element included in the resultant bitstream.

31. The method claim 20, further comprising utilizing pre-specified insertion and deletion operations for the at least one virtual reference picture with respect to the decoded picture buffer.

32. The method of claim 20, further comprising storing the at least one virtual reference picture in a short-term memory portion of the decoded picture buffer.

33. The method of claim 20, further comprising storing the at least one virtual reference picture in a long-term memory portion of the decoded picture buffer.

34. The method of claim 20, wherein said encoding step directly refers to the at least one virtual reference picture in a prediction process for the at least one picture, using an index of the at least one virtual reference picture.

35. The method of claim 34, further comprising performing a default reference list construction process that includes the at least one virtual reference picture.

36. The method of claim 35, further comprising defining at least one reference picture list reordering command to include the at least one virtual reference picture in the default reference list construction process.

37. The method of claim 35, wherein said step of performing the default reference list construction process comprises combining a pre-specified reference list reordering command with a high level syntax to include the at least one virtual reference picture in the default reference list construction.

38. The method of claim 20, wherein each of the at least one virtual reference picture is respectively referred to by a reference picture index corresponding to a reference picture list.

77. The method of claim 20, wherein dedicated memory slots are used in the decoded picture buffer to store virtual reference pictures including the virtual picture, and wherein each of the virtual reference pictures are identified by a virtual reference picture identifier that is unique for each of the virtual reference pictures.

81. The method of claim 32, wherein the at least one virtual reference picture and any other virtual reference pictures are stored in the short-term memory portion of the decoded picture buffer using unused frame numbers and unused picture order counts to differentiate the virtual reference pictures from the non-virtual reference pictures.

85. The method of claim 33, wherein the at least one virtual reference picture and any other virtual reference pictures are stored in the long-term memory portion of the decoded picture buffer using unused long term identifiers to differentiate the virtual reference pictures from the non-virtual reference pictures.

39. An apparatus, comprising:

a decoder for decoding at least one picture, using at least one virtual reference picture, from a bitstream, wherein the at least one virtual reference picture is different than a previously decoded picture, and the at least one virtual reference picture is stored in a decoded picture buffer that also stores non-virtual reference pictures.

40. The apparatus of claim 39, wherein the at least one picture corresponds to at least one of at least two views of multi-view video content.

41. The apparatus of claim 39, wherein the at least one virtual reference picture is formed from at least one reference picture that has undergone a signal processing transformation.

42. The apparatus of claim 41, wherein said decoder determines warping parameters of the signal processing transformation from at least one high level syntax included in the bitstream.

43. The apparatus of claim 39, wherein the at least one virtual reference picture is formed from at least one filtered reference picture.

44. The apparatus of claim 43, wherein said decoder determines filter coefficients used to filter the at least one filtered reference picture from at least one high level syntax included in the bitstream.

45. The apparatus of claim 39, wherein the at least one picture corresponds to multi-view video content, and the at least one virtual reference picture is formed using at least one view interpolated picture.

46. The apparatus of claim 45, wherein said decoder determines parameters for the at least one view interpolated picture from at least one high level syntax element included in the resultant bitstream.

47. The apparatus of claim 39, wherein said decoder determines an existence of the at least one virtual reference picture in the bitstream.

48. The apparatus of claim 39, wherein said decoder determines an existence of the at least one virtual reference picture from a high-level syntax in the bitstream.

49. The apparatus of claim 39, wherein at least one of a method and a parameter for creating the at least one virtual reference picture is determined from at least one high level syntax element included in the bitstream.

50. The apparatus of claim 39, wherein said decoder utilizes pre-specified insertion and deletion operations for the at least one virtual reference picture with respect to the decoded picture buffer.

51. The apparatus of claim 39, wherein the at least one virtual reference picture is stored in a short-term memory portion of the decoded picture buffer.

52. The apparatus of claim 39, wherein the at least one virtual reference picture is stored in a long-term memory portion of the decoded picture buffer.

53. The apparatus of claim 39, wherein said decoder directly refers to the at least one virtual reference picture in a prediction process for the at least one picture, using an index of the at least one virtual reference picture.

54. The apparatus of claim 53, wherein a default reference list construction process is performed to include the at least one virtual reference picture.

55. The apparatus of claim 54, wherein at least one reference picture list reordering command is defined to include the at least one virtual reference picture in the default reference list construction.

56. The apparatus of claim 39, wherein each of the at least one virtual reference picture is respectively referred to by a reference picture index corresponding to a reference picture list.

78. The apparatus of claim 39, wherein dedicated memory slots are used in the decoded picture buffer to store virtual reference pictures including the virtual picture, and wherein each of the virtual reference pictures are identified by a virtual reference picture identifier that is unique for each of the virtual reference pictures.

82. The apparatus of claim 51, wherein the at least one virtual reference picture and any other virtual reference pictures are stored in the short-term memory portion of the decoded picture buffer using unused frame numbers and unused picture order counts to differentiate the virtual reference pictures from the non-virtual reference pictures.

86. The apparatus of claim 52, wherein the at least one virtual reference picture and any other virtual reference pictures are stored in the long-term memory portion of the decoded picture buffer using unused long term identifiers to differentiate the virtual reference pictures from the non-virtual reference pictures.

57. In a video decoder, a method, comprising:

decoding at least one picture, using at least one virtual reference picture, from a bitstream, wherein the at least one virtual reference picture is different than a previously decoded picture, and the at least one virtual reference picture is stored in a decoded picture buffer that also stores non-virtual reference pictures.

58. The method of claim 57, wherein the at least one picture corresponds to at least one of at least two views of multi-view video content.

59. The method of claim 57, wherein the at least one virtual reference picture is formed from at least one reference picture that has undergone a signal processing transformation.

60. The method of claim 59, wherein said decoding step comprises determining warping parameters of the signal processing transformation from at least one high level syntax included in the bitstream.

61. The method of claim 57, wherein the at least one virtual reference picture is formed from at least one filtered reference picture.

62. The method of claim 61, wherein said decoding step comprises determining filter coefficients used to filter the at least one filtered reference picture from at least one high level syntax included in the bitstream.

63. The method of claim 57, wherein the at least one picture corresponds to multi-view video content, and the at least one virtual reference picture is formed using at least one view interpolated picture.

64. The method of claim 63, wherein said decoding step comprises determining parameters for the at least one view interpolated picture from at least one high level syntax element included in the resultant bitstream.

65. The method of claim 57, wherein said decoding step comprises determining an existence of the at least one virtual reference picture in the bitstream.

66. The method of claim 57, wherein said decoding step comprises determining an existence of the at least one virtual reference picture from a high-level syntax in the bitstream.

67. The method of claim 57, wherein said decoding step comprises determining at least one of a method and a parameter for creating the at least one virtual reference picture from at least one high level syntax element included in the bitstream.

68. The method of claim 57, further comprising utilizing pre-specified insertion and deletion operations for the at least one virtual reference picture with respect to the decoded picture buffer.

69. The method of claim 57, further comprising storing the at least one virtual reference picture in a short-term memory portion of the decoded picture buffer.

70. The method of claim 57, further comprising storing the at least one virtual reference picture in a long-term memory portion of the decoded picture buffer.

71. The method of claim 57, wherein said encoding step directly refers to the at least one virtual reference picture in a prediction process for the at least one picture, using an index of the at least one virtual reference picture.

72. The method of claim 71, further comprising performing a default reference list construction process that includes the at least one virtual reference picture.

73. The method of claim 72, further comprising defining at least one reference picture list reordering command to include the at least one virtual reference picture in the default reference list construction process.

74. The method of claim 57, wherein each of the at least one virtual reference picture is respectively referred to by a reference picture index corresponding to a reference picture list.

79. The method of claim 57, wherein dedicated memory slots are used in the decoded picture buffer to store virtual reference pictures including the virtual picture, and wherein each of the virtual reference pictures are identified by a virtual reference picture identifier that is unique for each of the virtual reference pictures.

83. The method of claim 69, wherein the at least one virtual reference picture and any other virtual reference pictures are stored in the short-term memory portion of the decoded picture buffer using unused frame numbers and unused picture order counts to differentiate the virtual reference pictures from the non-virtual reference pictures.

87. The method of claim 70, wherein the at least one virtual reference picture and any other virtual reference pictures are stored in the long-term memory portion of the decoded picture buffer using unused long term identifiers to differentiate the virtual reference pictures from the non-virtual reference pictures.

75. A non-transitory computer readable storage media having video signal data encoded thereupon, comprising:

at least one picture, encoded using at least one virtual reference picture, to form a resultant bitstream, wherein the at least one virtual reference picture is different than a previously decoded picture, and the at least one virtual reference picture is stored in a decoded picture buffer that also stores non-virtual reference pictures.