Adaptive bitrate management for streaming media over packet networks

• US 8,621,061 B2
• Filed: 07/24/2012
• Issued: 12/31/2013
• Est. Priority Date: 07/10/2007
• Status: Active Grant

• Current Assignee: OptiMorphix, Inc.
• Original Assignee: Citrix Systems, Inc. (Cloud Software Group)
• Inventors: Melnyk, Miguel A., Stavrakos, Nicholas J., Breg, Fabian, Penner, Andrew
• Primary Examiner(s): Chan, Wing F
• Assistant Examiner(s): MIAH, RAZU A
• Application Number:
  US13/557,086
• Publication Number:
  US 20120290739A1
• Time in Patent Office: 525 Days
• Field of Search: 709/233, 370/312
• US Class Current: 709/223
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/2416 Real-time traffic
• CPC Class Codes: H04L 47/25 with rate being modified by the source upon detecting a change of network conditions
• CPC Class Codes: H04L 47/263 Rate modification at the source after receiving feedback
• CPC Class Codes: H04L 47/283 in response to processing delays, e.g. caused by jitter or round trip time [RTT]
• CPC Class Codes: H04L 47/32 by discarding or delaying data units, e.g. packets or frames
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/197 including determination of the initial value of an encoding parameter H04N19/56 takes precedence
• CPC Class Codes: H04N 21/23406 involving management of server-side video buffer
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/234381 by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/6377 directed to server one-way streaming services wherein the source is controlled by the destination H04L65/613
• CPC Class Codes: H04N 21/6437 Real-time Transport Protocol [RTP]
• CPC Class Codes: H04N 21/658 Transmission by the client directed to the server
• CPC Class Codes: H04N 21/6582 Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number arrangements where receivers interact with the broadcast H04H20/38

Abstract

A method including receiving a receiver report from a terminal; estimating one or more network conditions of a media network based at least in part on the receiver report; determining an optimal session bitrate based on the estimated one or more network conditions; and providing media data to the terminal based on the optimal session bitrate.

Claims

1. A method comprising:

receiving media data that includes audio media data and video media data;

receiving a receiver report from a terminal;

estimating one or more network conditions of a media network using the receiver report;

determining an optimal session bitrate using the estimated one or more network conditions;

allocating the optimal session bitrate between audio media data and video media data to produce an optimal audio bitrate and an optimal video bitrate;

encoding the audio media data using the optimal audio bitrate;

encoding the video media data using the optimal video bitrate; and

providing the encoded audio media data and the encoded video media data for transmission to the terminal.

2. The method of claim 1, wherein the receiver report is selected from a group including a real-time transport control protocol (RTCP) receiver report, and a real-time transport control protocol extended reports (RTCP XR) receiver report.

3. The method of claim 1, further comprising dropping frames of the encoded video data.

4. The method of claim 1, wherein allocating the optimal session bitrate between the audio media data and the video media data is based on one or more metrics selected from a group including a predetermined allocation, a user preference, an optimal performance data, privileging one type of media data over the other, and an amount of audio and video media data to be provided.

5. The method of claim 1, wherein determining the optimal session bitrate further comprises:

establishing stability criterion based on the estimated one or more network conditions;

determining the stability of the media network; and

providing the optimal session bitrate based on the stability determination.

6. The method of claim 5, wherein establishing stability criterion further comprises:

comparing a media time in transit and a round trip time estimate for assisting with the stability determination.

7. The method of claim 5, wherein establishing stability criterion further comprises:

comparing a bitrate received with a current bitrate session.

8. The method of claim 5, further comprising:

if the stability of the media network is considered normal, maintaining or incrementally increasing a current bitrate; and

if the stability of the media network is not normal, adjusting the current bitrate.

9. A method comprising:

receiving media data that includes audio media data and video media data;

receiving an optimal session bitrate;

allocating the optimal session bitrate between the audio media data and the video media data to produce an optimal audio bitrate and an optimal video bitrate;

encoding the audio media data using the optimal audio bitrate;

encoding the video media data using the optimal video bitrate;

dropping frames of the encoded video data; and

providing the encoded audio media data and remaining encoded video media data for transmittal to a terminal.

10. The method of claim 9, wherein allocating the optimal session bitrate between the audio media data and the video media data is based on one or more metrics selected from a group including a predetermined allocation, a user preference, an optimal performance data, privileging one type of media data over the other, and an amount of audio and video media data to be provided.

11. A terminal including one or more processors and a memory, the terminal comprising:

a buffer configured to receive media data packets transmitted by an adaptive bitrate manager over a media network; and

a media player configured to receive media data packets and provide a receiver report to the adaptive bitrate manager configured to;

receive media data that includes audio media data and video media data;

receive the receiver report,

estimate one or more network conditions of the media network using-the receiver report,

determine an optimal audio bitrate and an optimal video bitrate using estimated one or more network conditions,

encode the audio media data using the optimal audio bitrate,

encode the video media data using the optimal video bitrate, and

provide the encoded audio media data and the encoded video media data for transmission as media data packets.

12. A system comprising:

an adaptive bitrate manager configured to estimate one or more network conditions of a media network between the adaptive bitrate manager and a terminal, to determine an optimal session bitrate using the estimated one or more network conditions, and to acquire media data that includes audio media data and video media data, the adaptive bitrate manager further comprising;

a bitrate splitter configured to allocate the optimal session bitrate between audio media data and video media data to produce an optimal audio bitrate and an optimal video bitrate, and

one or more encoders configured to;

encode the audio media data using the optimal audio bitrate,

encode the video media data using the optimal video bitrate, and

provide the encoded audio media data and the encoded video media data for transmission to the terminal.

13. The system of claim 12, wherein the adaptive bitrate manager further comprises:

an adaptive bitrate controller configured to determine the optimal session bitrate.

14. The system of claim 12, wherein the adaptive bitrate manager includes one or more processors.

15. A non-transitory computer readable storage medium storing instructions that, when executed by one or more computers, cause the one or more computers to perform a method for processing a receiver report, the method comprising:

receiving media data that includes audio media data and video media data;

receiving the receiver report from a terminal;

estimating one or more network conditions of a media network using the receiver report;

determining an optimal session bitrate using the estimated one or more network conditions;

allocating the optimal session bitrate between audio media data and video media data to produce an optimal audio bitrate and an optimal video bitrate;

encoding the audio media data using the optimal audio bitrate;

encoding the video media data using the optimal video bitrate; and

providing the encoded audio media data and the encoded video media data for transmission to the terminal.

17. The non-transitory computer readable storage medium of claim 15, wherein allocating the optimal session bitrate between the audio media data and the video media data is based on one or more metrics selected from a group including a predetermined allocation, a user preference, an optimal performance data, privileging one type of media data over the other, and an amount of audio and video media data to be provided.

18. The non-transitory computer readable storage medium of claim 15, wherein determining the optimal session bitrate further comprises:

establishing stability criterion based on the estimated one or more network conditions;

determining the stability of the media network; and

providing the optimal session bitrate based on the stability determination.

19. The non-transitory computer readable storage medium of claim 18, wherein establishing stability criterion further comprises one or more of the following operations:

comparing a media time in transit and a round trip time estimate for assisting with the stability determination;

or

comparing a bitrate received with a current bitrate session.

16. A non-transitory computer readable storage medium storing instructions that, when executed by one or more computers, cause the one or more computers to perform a method for processing an optimal session bitrate, the method comprising:

receiving media data that includes audio media data and video media data;

receiving the optimal session bitrate;

allocating the optimal session bitrate between the audio media data and the video media data to produce an optimal audio bitrate and an optimal video bitrate;

encoding the audio media data using the optimal audio bitrate;

encoding the video media data using the optimal audio bitrate and the optimal video bitrate dropping frames of the encoded video data; and

providing the encoded audio media data and remaining encoded video media data for transmittal to a terminal.

20. The non-transitory computer readable storage medium of claim 16, wherein allocating the optimal session bitrate between the audio media data and the video media data is based on one or more metrics selected from a group including a predetermined allocation, a user preference, an optimal performance data, privileging one type of media data over the other, and an amount of audio and video media data to be provided.