Video Rate Control for Video Coding Standards

• US 20100080292A1
• Filed: 12/11/2007
• Published: 04/01/2010
• Est. Priority Date: 12/12/2006
• Status: Active Grant

• Current Assignee: Ecole de technologie superieure, Masimo Corporation
• Original Assignee: Vantrix Corporation
• Inventors: Coulombe, Stéphane
• Granted Patent: US 8,401,076 B2
• Time in Patent Office: Days
• US Class Current: 375/240.120
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/149 by estimating the code amount by means of a model, e.g. mathematical model or statistical model
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/50 using predictive coding H04N19/61 takes precedence
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

A method and device for improving rate controlling in video coding of sequences including a series of Inter frames separated by Intra frames, when a decoding delay is considered, comprise for each Inter frame of the series: computing a target frame size, computing a maximum buffer level related to a position of each Inter frame relative to a previous Intra frame and an upcoming Intra frame, and optimizing a transmission buffer level in response to the computed target frame size and the computed maximum buffer level.

Claims

1. A method for rate controlling in video coding comprising:

specifying a maximum bitrate and an average bitrate;

calculating an Inter frame size for meeting with the specified maximum bitrate, the Inter frame size being related to a position of the Inter frame relative to a position of an Intra frame; and

adjusting the calculated Inter frame size for meeting with both the specified maximum bitrate and the average bitrate.

2. A method as defined in claim 1, further comprising specifying a size of the Intra frame.

3. A method as defined in claim 2, further comprising estimating quantization parameters of the Intra frame for encoding the Intra frame for meeting with the specified Intra frame size.

4. A method as defined in claim 2, wherein calculating the Inter frame size for meeting with the specified maximum bitrate comprises calculating an ideal Inter frame size based on the specified maximum bitrate.

5. A method as defined in claim 4, wherein calculating the ideal Inter frame size comprises using the following equation, when the position of the Inter frame indicates that an Intra frame has been encoded during a last-occurred time period or is going to be encoded during an upcoming time period:

6. A method as defined in claim 4, wherein calculating the ideal Inter frame size further comprises using the following equation, when the position of the Inter frame indicates that no Intra frame has been encoded during a last-occurred time period or is going to be encoded during an upcoming time period:

7. A method as defined in claim 5, wherein calculating the Inter frame size for meeting with the specified maximum bitrate further comprises taking into consideration past encoded frames.

8. A method as defined in claim 7, wherein calculating the Inter frame size B_PI which takes into consideration past encoded frames comprises using the following equations, when an Intra frame has been encoded within a last-occurred time period:

9. A method as defined in claim 8, wherein calculating the Inter frame size B_P1 which takes into consideration past encoded frames further comprises using the following equations, when an Intra frame needs to be encoded in an upcoming time period:

10. A method as defined in claim 7, wherein calculating the Inter frame size which takes into consideration past encoded frames comprises taking into consideration past Intra frames.

11. A method as defined in claim 9, wherein calculating the Inter frame size for meeting with the specified maximum bitrate further comprises limiting the Inter frame size to a maximum size allowed by the specified maximum bitrate.

12. A method as defined in claim 11, wherein limiting the Inter frame size B_P, comprises using the following equations:

13. A method as defined in claim 6, wherein calculating the Inter frame size for meeting the specified maximum bitrate further comprises limiting the Inter frame size to a maximum size allowed by the specified maximum bitrate by using the following equation:

B_P=max(0,min(B_P,tmp2))

with B_P being the Inter frame size and

14. A method as defined in claim 1, wherein adjusting the calculated Inter frame size for meeting with both the specified maximum and average bitrates comprises calculating an ideal Inter frame size based on the specified average bitrate.

15. A method as defined in claim 14, wherein calculating the ideal Inter frame size based on the specified average bitrate comprises using the following equation, when the position of the Inter frame indicates that an intra frame has been encoded during a last-occurred time period or is going to be encoded during an upcoming time period:

16. A method as defined in claim 14, wherein calculating the ideal Inter frame size based on the specified average bitrate further comprises using the following equation, when the position of the Inter frame indicates that no Intra frame has been encoded during a last-occurred time period or is going to be encoded during an upcoming time period:

17. A method as defined in claim 15, wherein adjusting the calculated Inter frame size for meeting with both the specified maximum and average bitrates comprises taking into consideration past encoded frames.

18. A method as defined in claim 17, wherein adjusting the calculated Inter frame size for meeting with both the specified maximum and average bitrates further comprises comparing the calculated Inter frame size with a corresponding encoded frame size.

19. A method as defined in claim 18, wherein comparing the calculated Inter frame size with a corresponding encoded frame size comprises storing results from the comparison in a variable.

20. A method as defined in claim 19, wherein storing the results from the comparison in a variable comprising updating the variable using the following equation:

B_extra=B_extra+B_target−

B_encoded

where B_extra is the variable, B_target is the calculated Inter frame size and B_encoded is the corresponding encoded frame size.

21. A method as defined in claim 20, wherein updating the variable B_extra comprises distributing extra bits, usable for encoding Inter future frames, over a next Inter frame size, when the variable B_extra is positive.

22. A method as defined in claim 20, wherein updating the variable comprises reducing the calculated Inter frame size for a next Inter frame size when the variable B_extra is negative.

23. A method as defined in claim 20, wherein updating the variable comprises dropping an encoded Inter frame when the variable B_extra is below a given threshold.

24. A method as defined in claim 14, wherein adjusting the calculated Inter frame size for meeting both the specified maximum and average bitrates further comprises limiting the Inter frame size to a maximum size allowed by the specified maximum bitrate.

25. A method as defined in claim 1, further comprising verifying that an encoded frame size is inferior to the adjusted Inter frame size prior to transmission.

26. A method as defined in claim 1, further comprising memorizing an encoder state variables before encoding a frame and restoring the state variables after an encoded frame has been dropped.

27. A method as defined in claim 1, further comprising dropping encoded frames which have a size leading to exceed the specified maximum bitrate.

28. A method as defined in claim 27, further comprising dropping encoded frames which have a size leading to exceed the specified average bitrate.

29. A method as defined in claim 28, further comprising re-encoding the encoded frames having a size leading to exceed the specified average and maximum bitrates with different encoding parameters.

30. A method for improving rate controlling in video coding of sequences including a series of Inter frames separated by Intra frames, when a decoding delay is considered, the method comprising:

for each Inter frame of the series, computing a target frame size;

for each Inter frame of the series, computing a maximum buffer level related to a position of each Inter frame relative to a previous Intra frame and an upcoming Intra frame; and

optimizing a transmission buffer level in response to the computed target frame size and the computed maximum buffer level.

31. A method as defined in claim 30, further comprising dropping frames when an encoded frame exceeds the maximum buffer level.

32. A method as defined in claim 30, further comprising memorizing encoder state variables before encoding a frame.

33. A method as defined in claim 32, further comprising restoring the encoder state variables after an encoded frame has been dropped prior to encoding a next frame.

34. A method as defined in claim 30, further comprising ensuring that a transmitted frame size is inferior to a given maximum limit.

35. A method as defined in claim 34, wherein the maximum limit is a maximum bitrate.

36. A method as defined in claim 30, wherein computing the target frame size for each Inter frame of the series comprises computing the following equations:

37. A method as defined in claim 36, wherein computing the target frame size for each Inter frame of the series further comprises:

determining a number of bits unused in encoding an Intra frame; and

distributing the number of unused bits over the target frame sizes of N Inter frames following the Intra frame.

38. A method as defined in claim 37, wherein determining the number of unused bits in encoding the Intra frame comprises calculating a difference between the encoded Intra frame size and a corresponding Intra frame target size.

39. A method as defined in claim 38, wherein distributing the number of unused bits over the target frame sizes of the N Inter frames comprises optimizing the transmission buffer level over a number L of Inter frames, where 0≦

N≦

L.

40. A method as defined in claim 37, wherein distributing the number of unused bits over the target frame sizes of the N Inter frames comprises distributing substantially equally the number of unused bits over the target frame sizes of the N Inter frame sizes.

41. A method as defined in claim 30, wherein computing the target frame size for each Inter frame of the series further comprises computing the following equations, when distributing substantially equally a number of unused bits over the target frame sizes of N Inter frames:

42. A method as defined in claim 30, wherein computing, for each Inter frame of the series, the maximum buffer level related to the position of each Inter frame relative to the previous and upcoming Intra frames ensures that the transmission buffer can accommodate the upcoming Intra frame.

43. A method as defined in claim 42, wherein computing, for each Inter frame of the series, the maximum buffer level comprises computing a maximum buffer level close to a maximum buffer size when an Inter frame to be encoded is near the previous Intra frame and close to zero when the Inter frame is near the upcoming Intra frame.

45. A method as defined in claim 30, wherein optimizing the transmission buffer level in response to the calculated target frame size and the maximum buffer level comprises computing a difference between the computed target frame size and the computed maximum buffer level.

46. A method as defined in claim 45, wherein computing the difference between the computed target frame size and the computed maximum buffer level comprises computing a margin for encoding a frame.

47. A method as defined in claim 46, wherein computing the margin for encoding a frame comprises computing the following equation:

Δ

(t)=BufferLevel_MAX(t)−

BufferLevel(t)

where Δ

(t) is the margin, BufferLevel_MAX(t) is the computed maximum buffer level at time t and BufferLevel(t) is the transmission buffer level at time t.

48. A method as defined in claim 47, further comprising updating each Inter frame target size.

49. A method as defined in claim 48, wherein updating each Inter frame target size comprises calculating a number of bits unused in encoding a previous frame.

50. A method as defined in claim 49, wherein calculating the number of unused bits in the encoding of the previous frame comprises calculating the following equation:

E(t)=BufferLevel_MAX(t)−

BufferLevel_actual(t)−

Δ

(t)

where BufferLevel_actual(t) is the transmission buffer level after actual encoding of a frame.

44. A method as defined 30, wherein computing the maximum buffer level BufferLevel_MAX(t) for each Inter frame of the series comprises using the following equations:

51. A device for rate controlling in video coding, the device comprising:

means for calculating an Inter frame size for meeting with a specified maximum bitrate, the Inter frame size being related to a position of the Inter frame relative to a position of an Intra frame; and

means for adjusting the calculated Inter frame size for meeting with a specified average bitrate, in addition to the specified maximum bitrate.

52. A device for rate controlling in video coding, the device comprising:

a first calculator of an Inter frame size for meeting with a specified maximum bitrate, the Inter frame size being related to a position of the Inter frame relative to a position of an Intra frame; and

a second calculator of the Inter frame size calculated by the first calculator for meeting with a specified average bitrate, in addition to the specified maximum bitrate.

53. A device as defined in claim 52, further comprising a predictor of an Intra frame size and of quantizing parameters of the Intra frame.

54. A device as defined in claim 52, further comprising a validator that an encoded frame size is inferior to the adjusted Inter frame size, prior to transmission.

55. A device as defined in claim 52, further comprising a circular buffer for containing transmitted frames.

56. A device as defined in claim 52, further comprising an encoder having state variables which are memorized before encoding a frame and restored after an encoded frame has been dropped.

57. A device for improving rate control in video coding of sequences including a series of Inter frames separated by Intra frames, when a decoding delay is allowed, the device comprising:

for each Inter frame, means for computing a target frame size;

for each Inter frame, means for computing a maximum buffer level related to a position of each Inter frame relative to a previous Intra frame and an upcoming Intra frame; and

means for optimizing a transmission buffer level in response to the computed target frame size and the computed maximum buffer level.

58. A device for improving rate control in video coding of sequences including a series of Inter frames separated by Intra frames, when a decoding delay is allowed, the device comprising:

a calculator of a target frame size, for each Inter frame;

a calculator of a maximum buffer level related to a position of each Inter frame relative to a previous Intra frame and an upcoming Intra frame, for each Inter frame; and

an optimizer of a transmission buffer level in response to the computed target frame size and the computed maximum buffer level.

59. A device as defined in claim 58, further comprising an encoder having state variables which are memorized before encoding a frame and restored after an encoded frame has been dropped.