Dynamic bit rate adaptation over bandwidth varying connection

• US 9,137,551 B2
• Filed: 08/16/2011
• Issued: 09/15/2015
• Est. Priority Date: 08/16/2011
• Status: Active Grant

• Current Assignee: Masimo Corporation
• Original Assignee: Vantrix Corporation
• Inventors: Ct, Sbastien, Krause, Jean-Nol
• Primary Examiner(s): Philippe, Gims
• Assistant Examiner(s): Pezzner, Benjamin
• Application Number:
  US13/210,652
• Publication Number:
  US 20130044801A1
• Time in Patent Office: 1,491 Days
• Field of Search: 375/240.01, 375/E7.159, 348/419.1
• US Class Current: 1/1
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04N 19/146 Data rate or code amount at the encoder output
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 21/23406 involving management of server-side video buffer
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/64322 IP
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments

Abstract

Audio and video streams of a media file are transcoded just-in-time by keeping the transcoding process synchronized to a wall clock. The transcoding is initially configured slightly faster than real-time. The transcoded frames are pushed from the transcoder through a variable bandwidth connection as soon as they are transcoded. Transcoding is then monitored periodically to assess the bandwidth that was available during the transmission of a transcoded buffer. Two time intervals are measured: a buffer interval which is the difference in the transcoded buffer timestamps of the previous two transmissions, and a clock interval corresponding to the real time transmission time of the last transcoded buffer. If a significant difference is observed, the transcoder bit rate is adjusted by a factor that is derived from that difference.

Claims

1. A method for encoding a media file or part thereof for transmission in real time over a varying bandwidth link to a mobile unit, comprising:

employing at least one processor of a transcoder for;

(i) encoding one or more fragments of the media file into transcoded media fragments to form a stream fragment;

(ii) sending the stream fragment to a transmission buffer of the transcoder;

(iii) determining an estimated transmit time of a previously formed stream fragment, the estimated transmit time being a time interval required for said previously formed stream fragment to be conveyed from the transmission buffer and consumed by the varying bandwidth link;

(iv) determining an estimated display time of said previously formed stream fragment, the estimated display time being determined before said previously formed stream fragment is conveyed from the transmission buffer, the estimated display time being derived from timestamps embedded in two previously formed stream fragments; and

(v) adjusting a current encoding rate of the transcoded media fragments as a function of the estimated transmit time and the estimated display time.

2. The method of claim 1, wherein the step (i) comprises:

encoding at least a first fragment of the media file into corresponding first transcoded media fragment at an initial encoding rate; and

setting the current encoding rate equal to the initial encoding rate;

and the step (v) comprises encoding subsequent fragments of the media file into corresponding transcoded media fragments at the adjusted current encoding rate.

3. The method of claim 1, wherein the function is a product of the current encoding rate and an adjustment factor which depends on the estimated display time and the estimated transmit time.

4. The method of claim 3, wherein the adjustment factor is the estimated display time divided by the estimated transmit time.

5. The method of claim 1, wherein the step (v) further comprises adjusting the current encoding rate as a function of the current encoding rate.

6. The method of claim 1, further comprising transmitting the transcoded media fragments to the mobile unit for display.

7. The method of claim 1, wherein the step (iii) comprises adjusting the current encoding rate from time to time.

8. The method of claim 7, wherein the step (iii) further comprises estimating the estimated transmit time as a current clock interval between a current time and a previous time of a last encoding.

9. The method of claim 8, wherein the step (v) comprises adjusting the current encoding rate after a current clock interval of N seconds has elapsed.

10. The method of claim 9, wherein N is in a range between 2 and 5 seconds.

11. The method of claim 8, wherein the step (v) further comprises:

representing the estimated display time as a current buffer interval;

comparing the current clock interval with the current buffer interval; and

adjusting the current encoding rate as a function of the current clock interval and the current buffer interval.

12. The method of claim 11, wherein the two previously formed stream fragments comprise last two stream fragments.

13. The method of claim 11, wherein the adjusting comprises adjusting the current encoding rate provided the current clock interval differs from the current buffer interval by a predetermined amount.

14. The method of claim 11, wherein the adjusting comprises:

increasing the current encoding rate by a first incremental value computed from a first relative difference between the current buffer interval and the current clock interval, provided the first relative difference exceeds a first predetermined threshold TH1; and

decreasing the current encoding rate by a second incremental value computed from a second relative difference between the current clock interval and the current buffer interval, provided the second relative difference exceeds a second predetermined threshold TH2.

15. The method of claim 14, wherein TH1 is at least 0.05, and TH2 is at least 0.02.

16. The method of claim 14, wherein

the first incremental value is equal to the first relative difference divided by a first attenuation factor AF1, and

the second incremental value is equal to the second relative difference divided by a second attenuation factor AF2, the first and second attenuation factors AF1 and AF2 being chosen so as to prevent excessive or rapid over-adjusting of the current encoding rate and to ensure stability of the adjusting step.

17. The method of claim 1, wherein the step (v) comprises adjusting the current encoding rate only if a pause in the transmission of the transcoded media fragments is not detected.

18. The method of claim 17, wherein the step (v) further comprises detecting the pause, comprising:

computing a clock interval average as an average of past current clock intervals since the encoding of the media file has begun, a current clock interval being measured between a current time and a previous time of a last encoding; and

determining that the pause is detected provided the current clock interval exceeds the clock interval average by a predetermined margin XC.

19. The method of claim 18, further comprising:

determining that the pause is detected provided;

at least two current clock intervals have occurred; and

a number of times the pause has already been detected is less than a maximum number (MP) of detected pauses.

20. The method of claim 19, further comprising determining that a pause is detected provided the current clock interval exceeds the clock interval average multiplied by a factor M.

21. The method of claim 20, wherein MP is 3, XC is 1, and M is 4.

22. The method of claim 18, further comprising determining that a pause is detected provided the current clock interval exceeds the clock interval average multiplied by a factor M.

23. The method of claim 18, further comprising replacing the current clock interval with the average clock interval provided the pause is detected.

24. A media processing system for encoding a media file or part thereof for transmission in real time over a varying bandwidth link to a mobile unit, comprising:

a processor and a non-transitory computer readable storage medium having computer readable instructions stored thereon for execution by the processor, forming;

a transcoder configured to transcode one ore more fragments of the media file into transcoded media fragments to form a stream fragment;

a transmission buffer of the transcoder configured to convey the stream fragment to the varying bandwidth link; and

a dynamic bit rate adapter configured to adjust a current encoding bit rate of the transcoder as a function of

(i) an estimated transmit time of a previously formed stream fragment, the estimated transmit time being a time interval required for said previously formed media fragment to be conveyed from the transmission buffer and consumed by the varying bandwidth link, and

(ii) an estimated display time of said previously formed stream fragment, the estimated display time being determined before said previously formed stream fragment is conveyed from the transmission buffer, the estimated display time being derived from timestamps embedded in two previously formed stream fragments.

25. The media processing system of claim 24, wherein the dynamic bit rate adapter comprises:

a wall clock module for tracking a current time; and

a quality of service (QoS) adjuster configured to process the current time and the timestamps into a QoS value to be sent to the transcoder for adjusting the encoding bit rate of the transcoder from time to time.

26. The media processing system of claim 24, wherein the transmission buffer is adapted to forward transcoded media fragments to a network, and to send timestamps associated with at least some of the transcoded media fragments to the dynamic bit rate adapter.

27. The media processing system of claim 26, wherein the dynamic bit rate adapter is further configured to adjust the current encoding rate as a function of the current encoding rate.

28. The media processing system of claim 24, wherein the dynamic bit rate adapter further comprises a pause detector configured to detect a cessation of transmission of the transcoded media fragments and to inhibit the dynamic bit rate adapter from adjusting the current encoding rate provided the cessation has been detected.

29. The media processing system of claim 24, wherein the dynamic bit rate adapter is further configured to estimate the estimated transmit time as a current clock interval between a current time and a previous time of a last encoding.

30. The media processing system of claim 29, wherein the dynamic bit rate adapter is further configured to:

represent the estimated display time as a current buffer interval;

compare the current clock interval with the current buffer interval; and

adjust the current encoding rate as a function of the current clock interval and the current buffer interval.

31. The media processing system of claim 30, wherein the dynamic bit rate adapter is further configured to adjust the current encoding rate provided the current clock interval differs from the current buffer interval by a predetermined amount.

32. A method for encoding a media file or part thereof for transmission in real time over a varying bandwidth link to a mobile unit, comprising:

employing at least one processor of a transcoder for;

(a) encoding at least a first fragment of the media file into a transcoded media fragment at a current encoding rate equal to an initial encoding rate to form a stream fragment, sending the stream fragment to a transmission buffer of the transcoder, and transmitting the stream fragment;

(b) reevaluating the current encoding rate into a new encoding rate based on an estimated transmit time of a previously transmitted stream fragment, the estimated transmit time being a time interval required for said stream fragment to be conveyed from the transmission buffer and consumed by the varying bandwidth link, and further based on an estimated display time of said previously formed stream fragment, the estimated display time being determined before said previously formed stream fragment is conveyed from the transmission buffer, the estimated display time being derived from timestamps embedded in at least some of the transcoded media fragments of the last stream and a previous stream fragment;

(c) setting the current encoding rate to be equal to the new encoding rate;

(d) encoding another fragment of the media file into a transcoded media fragment at the current encoding rate; and

(e) repeating the steps (b) to (d) until the media file or the part thereof has been completely transcoded.

33. The method of claim 32, wherein the step (b) comprises estimating the estimated transmit time as a current clock interval between a current time and a previous time of a last reevaluating.

34. The method of claim 33, wherein the step (b) further comprises:

representing the estimated display time as a current buffer interval;

comparing the current clock interval with the current buffer interval; and

modifying the current encoding rate provided the current clock interval differs from the current buffer interval.

35. The method of claim 34, wherein the modifying comprises modifying the current encoding rate as a function of the current clock interval and the current buffer interval.

36. The method of claim 33, wherein the step (b) comprises reevaluating the current encoding rate only after a current clock interval of N seconds has elapsed.

37. The method of claim 32, wherein the step (b) further comprises computing the new encoding rate as a function of the current encoding rate.

38. The method of claim 37, wherein the function is a product of the current encoding rate and an adjustment factor depending on the estimated display time and the estimated transmit time.

39. The method of claim 32, wherein the step (b) further comprises detecting a cessation of transmission of the transcoded media fragments, and suppressing an adjusting of the current encoding rate provided the cessation has been detected.