Adaptive bitrate management for streaming media over packet networks

• US 8,255,551 B2
• Filed: 07/29/2011
• Issued: 08/28/2012
• Est. Priority Date: 07/10/2007
• Status: Active Grant

• Current Assignee: OptiMorphix, Inc.
• Original Assignee: Bytemobile Inc. (Cloud Software Group)
• Inventors: Melnyk, Miguel A., Stavrakos, Nicholas J., Penner, Andrew, Tidemann, Jeremy, Breg, Fabian
• Primary Examiner(s): Chan, Wing
• Assistant Examiner(s): MIAH, RAZU A
• Application Number:
  US13/194,761
• Publication Number:
  US 20110283012A1
• Time in Patent Office: 396 Days
• Field of Search: 709230-231, 370/506
• US Class Current: 709/230
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/193 at the transport layer, e.g. TCP related
• CPC Class Codes: H04L 47/2416 Real-time traffic
• CPC Class Codes: H04L 47/25 with rate being modified by the source upon detecting a change of network conditions
• CPC Class Codes: H04L 47/283 in response to processing delays, e.g. caused by jitter or round trip time [RTT]
• CPC Class Codes: H04L 47/32 by discarding or delaying data units, e.g. packets or frames
• CPC Class Codes: H04L 47/38 by adapting coding or compression rate
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 69/163 In-band adaptation of TCP data exchange; In-band control procedures
• CPC Class Codes: H04N 21/23406 involving management of server-side video buffer
• CPC Class Codes: H04N 21/234381 by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
• CPC Class Codes: H04N 21/2368 Multiplexing of audio and video streams
• CPC Class Codes: H04N 21/2385 Channel allocation H04N21/266 takes precedence; Bandwidth allocation H04N21/24 takes precedence ; allocation of channels according to the instantaneous demands of the users in time-division multiplex systems H04J3/1682; admission control, resource allocation in open networks H04L12/5692; arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896; negotiating bandwidth in wireless networks H04W28/16
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/6377 directed to server one-way streaming services wherein the source is controlled by the destination H04L65/613
• CPC Class Codes: H04N 21/658 Transmission by the client directed to the server
• CPC Class Codes: H04N 21/6582 : Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number arrangements where receivers interact with the broadcast H04H20/38
• CPC Class Codes: H04N 21/6583 : Acknowledgement

Abstract

A method including providing pseudo-streaming media data to a terminal; receiving a transport control protocol (TCP) acknowledgement from the terminal; estimating one or more network conditions of a network based at least in part on the TCP acknowledgement; determining an optimal session bitrate based on the estimated one or more network conditions; and providing pseudo-streaming media data to the terminal based on the optimal session bitrate.

Claims

1. A method comprising:

providing pseudo-streaming media data to a terminal, wherein the provided pseudo streaming media data includes a sequence number and corresponds to a timestamp;

causing a table to store an association between the sequence number and the timestamp;

receiving a transport control protocol (TCP) acknowledgement from the terminal, wherein the received TCP acknowledgement includes the sequence number and corresponds to a certain time;

acquiring the timestamp from the table using the sequence number of the TCP acknowledgement;

estimating one or more network conditions of a network using a comparison between the certain time and the acquired timestamp;

establishing stability criterion based on the estimated one or more network conditions;

determining the stability of the network;

determining an optimal session bitrate based on the estimated one or more network conditions;

providing the optimal session bitrate based on the stability determination; and

providing pseudo-streaming media data to the terminal according to the optimal session bitrate.

2. The method of claim 1, wherein establishing stability criterion further comprises comparing the media time in transit and a round trip time estimate for assisting with the stability determination.

3. The method of claim 1, wherein establishing stability criterion further comprises comparing a bitrate received with a current bitrate session.

4. The method of claim 1, further comprising maintaining or incrementally increasing a current bitrate when the stability of the network is considered normal.

5. The method of claim 1, further comprising adjusting a current bitrate when the stability of the network is not normal.

6. A method comprising:

providing pseudo-streaming media data to a terminal, wherein the provided pseudo streaming media data includes a sequence number and corresponds to a timestamp;

causing a table to store an association between the sequence number and the timestamp;

receiving a transport control protocol (TCP) acknowledgement from the terminal, wherein the received TCP acknowledgement includes the sequence number and corresponds to a certain time;

acquiring the timestamp from the table using the sequence number of the TCP acknowledgement;

estimating one or more network conditions of a network using a comparison between the certain time and the acquired timestamp;

determining an optimal session bitrate based on the estimated one or more network conditions;

allocating the optimal session bitrate between audio media data and video media data to produce an optimal audio bitrate and an optimal video bitrate;

encoding audio and video media data according to the optimal audio bitrate and the optimal video bitrate; and

providing pseudo-streaming media data to the terminal according to the optimal session bitrate.

7. The method of claim 6, wherein providing media data to the terminal includes providing the encoded audio media data and the encoded video media data based on the optimal audio bitrate and the optimal video bitrate.

8. A method comprising:

receiving a transport control protocol (TCP) acknowledgement from a terminal based on the terminal receiving pseudo-streaming media data, wherein the received TCP acknowledgement includes a sequence number and corresponds to a certain time;

acquiring a timestamp from a table using the sequence number of the TCP acknowledgement wherein the table includes the sequence number and the associated timestamp for the pseudo-streaming media data provided to the terminal;

estimating one or more network conditions of a network using a comparison between the certain time and the acquired timestamp;

determining the stability of the network based on the estimations, wherein determining the stability of the network is based on establishing stability criterion, and establishing stability criterion further comprises comparing the media time in transit and a round trip time estimate for assisting with the stability determination;

controlling a bitrate based on the determination; and

providing the bitrate to an encoder for transmitting pseudo-streaming media data according to the provided bitrate.

9. The method of claim 8, wherein establishing stability criterion further comprises comparing a bitrate received with a current bitrate session.

10. The method of claim 8, wherein controlling the bitrate includes maintaining or incrementally increasing a current bitrate when the stability of the network is considered normal.

11. The method of claim 8, wherein controlling the bitrate includes adjusting a current bitrate when the stability of the network is considered not normal.

12. A method comprising:

receiving an optimal session bitrate based on information provided by a transport control protocol (TCP) acknowledgement;

allocating the optimal session bitrate between audio and video media to produce an optimal audio bitrate and an optimal video bitrate, wherein allocating the optimal session bitrate between audio and video media involves allocating a higher bitrate for either the audio media or the video media over the other;

encoding audio and video media data according to the optimal audio bitrate and the optimal video bitrate;

multiplexing the encoded audio and video media data; and

providing the multiplexed audio and video data for transmittal to a terminal.

13. The method of claim 12, further comprising dropping frames of the encoded video data.

14. A system comprising:

a terminal, having a media player, configured to receive pseudo-streaming media data and provide a transport control protocol (TCP) acknowledgement based on the received pseudo-streaming media data, wherein the TCP acknowledgement includes a sequence number and corresponds to a certain time; and

an adaptive bitrate manager configured to;

receive the TCP acknowledgement,

acquire a timestamp from a table using the sequence number of the TCP acknowledgement, wherein the table includes the sequence number and the associated timestamp for the pseudo-streaming media data provided to the terminal,

estimate one or more network conditions using a comparison between the certain time and the acquired timestamp,

determine an optimal session bitrate based on the estimated one or more network conditions, and

provide pseudo-streaming media data to the terminal based on the optimal session bitrate;

wherein the adaptive bitrate manager further comprises;

an adaptive bitrate controller for receiving the receiver report and calculating an optimal session bitrate, and

an encoder for obtaining the optimal session bitrate, allocating the optimal session bitrate between audio and video media to produce an optimal audio bitrate and an optimal video bitrate, encoding audio and video data according to the optimal audio bitrate and the optimal video bitrate, and providing the encoded audio and video data for transmittal to the terminal.

15. A non-transitory computer readable storage medium storing instructions that, when executed by a computer, cause the computer to perform a method for processing a transport control protocol (TCP) acknowledgement, the method comprising:

providing pseudo-streaming media data to a terminal, wherein the provided pseudo-streaming media data includes a sequence number and corresponds to a timestamp;

causing a table to store an association between the sequence number and the timestamp;

receiving the transport control protocol (TCP) acknowledgement from the terminal, wherein the received TCP acknowledgement includes the sequence number and corresponds to a certain time;

acquiring the timestamp from the table using the sequence number of the TCP acknowledgement;

estimating one or more network conditions of a network using a comparison between the certain time and the acquired timestamp;

determining an optimal session bitrate based on the estimated one or more network conditions;

allocating the optimal session bitrate between audio media data and video media data to produce an optimal audio bitrate and an optimal video bitrate;

encoding audio and video media data according to the optimal audio bitrate and the optimal video bitrate; and

providing pseudo-streaming media data to the terminal according to the optimal session bitrate.

16. A non-transitory computer readable storage medium storing instructions that, when executed by a computer, cause the computer to perform a method for processing a transport control protocol (TCP) acknowledgement, the method comprising:

receiving a transport control protocol (TCP) acknowledgement from a terminal based on the terminal receiving pseudo-streaming media data, wherein the received TCP acknowledgement includes a sequence number and corresponds to a certain time;

acquiring a timestamp from a table using the sequence number of the TCP acknowledgement, wherein the table includes the sequence number and the associated timestamp for the pseudo-streaming media data provided to the terminal;

estimating one or more network conditions of a network using a comparison between the certain time and the acquired timestamp;

determining the stability of the network based on the estimations, wherein determining the stability of the network is based on establishing stability criterion, and establishing stability criterion further comprises comparing the media time in transit and a round trip time estimate for assisting with the stability determination;

controlling a bitrate based on the determination; and

providing the bitrate to an encoder for transmitting pseudo-streaming media data according to the provided bitrate.

17. A non-transitory computer readable storage medium storing instructions that, when executed by a computer, cause the computer to perform a method for processing an optimal session bitrate, the method comprising:

receiving the optimal session bitrate based on information provided by a transport control protocol (TCP) acknowledgement;

allocating the optimal session bitrate between audio and video media to produce an optimal audio bitrate and an optimal video bitrate, wherein allocating the optimal session bitrate between audio and video media involves allocating a higher bitrate for either the audio media or the video media over the other;

encoding audio and video media data according to the optimal audio bitrate and the optimal video bitrate;

multiplexing the encoded audio and video media data; and

providing the multiplexed audio and video data for transmittal to a terminal.

18. A terminal comprising:

a buffer that receives pseudo-streaming media data packets transmitted by an adaptive bitrate manager over a network; and

a media player that receives the pseudo-streaming media data packets and provides a transport control protocol (TCP) acknowledgement to the adaptive bitrate manager, where the adaptive bitrate manager is configured to;

receive the transport control protocol (TCP) acknowledgement, wherein the received TCP acknowledgement includes a sequence number and corresponds to a certain time,

acquire a timestamp from a table using the sequence number of the TCP acknowledgement, wherein the table includes the sequence number and the associated timestamp for the pseudo-streaming media data provided to the terminal,

estimate one or more network conditions using a comparison between the certain time and the acquired timestamp,

determine an optimal session bitrate based on the estimated one or more network conditions, and

provide pseudo-streaming media data to the buffer according to the optimal session bitrate;

wherein the adaptive bitrate manager further comprises;

an adaptive bitrate controller for receiving the receiver report and calculating an optimal session bitrate, and

an encoder for obtaining the optimal session bitrate, allocating the optimal session bitrate between audio and video media to produce an optimal audio bitrate and an optimal video bitrate, encoding audio and video data according to the optimal audio bitrate and the optimal video bitrate, and providing the encoded audio and video data for transmittal to the terminal.

19. A non-transitory computer readable storage medium storing instructions that, when executed by a computer, cause the computer to perform a method for processing a transport control protocol (TCP) acknowledgement, the method comprising:

providing pseudo-streaming media data to a terminal, wherein the provided pseudo-streaming media data includes a sequence number and corresponds to a timestamp;

causing a table to store an association between the sequence number and the timestamp;

receiving the transport control protocol (TCP) acknowledgement from the terminal, wherein the received TCP acknowledgement includes the sequence number and corresponds to a certain time;

acquiring the timestamp from the table using the sequence number of the TCP acknowledgement;

estimating one or more network conditions of a network using a comparison between the certain time and the acquired timestamp;

establishing stability criterion based on the estimated one or more network conditions;

determining the stability of the network;

determining an optimal session bitrate based on the estimated one or more network conditions;

providing the optimal session bitrate based on the stability determination; and

providing pseudo-streaming media data to the terminal according to the optimal session bitrate.