Reduction of latency in video distribution networks using adaptive bit rates

• US 9,204,203 B2
• Filed: 04/03/2012
• Issued: 12/01/2015
• Est. Priority Date: 04/07/2011
• Status: Active Grant

• Current Assignee: ActiveVideo Networks, Inc. (CommScope Holding Company, Inc.)
• Original Assignee: ActiveVideo Networks, Inc. (CommScope Holding Company, Inc.)
• Inventors: Brockmann, Ronald A., Dev, Anuj, Hiddink, Gerrit, Dahlby, Joshua, Pavlovskaia, Lena Y.
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): AYNALEM, NATHNAEL B
• Application Number:
  US13/438,617
• Publication Number:
  US 20120257671A1
• Time in Patent Office: 1,337 Days
• Field of Search: 375/240
• US Class Current: 1/1
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/164 Feedback from the receiver or from the transmission channel
• CPC Class Codes: H04N 21/64753 directed to the client
• CPC Class Codes: H04N 21/64769 for rate control flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/6587 Control parameters, e.g. trick play commands, viewpoint selection

Abstract

Systems and methods are provided for reducing and controlling playback latency in an unmanaged, buffered data network. A delay cost function is determined, the function representing the effect of playback latency on end user experience. An encoder transmits audiovisual data through the network to a client device. Network latency is measured, and the delay cost function is evaluated to establish an encoding bitrate for the encoder. The encoding of the audiovisual data is altered in response to dynamic network conditions, thereby controlling end-to-end playback latency of the system, which is represented by the playout length of data buffered between the encoder and the client device.

Claims

1. A method of controlling playback latency associated with transmission of source audiovisual data through an unmanaged, buffered data network, the method comprising:

at a server system coupled to a client device through the data network;

encoding a first portion of the source audiovisual data into a first portion of transmission audiovisual data at a first encoding bitrate;

buffering the first portion of transmission audiovisual data;

transmitting the first portion of transmission audiovisual data to the client device through the data network;

calculating a delay cost function based on a delay associated with the buffering and with network delay, raised to a power, scaled by a scaling factor, and added to a minimum cost to transmit data;

altering an encoding bitrate from the first encoding bitrate to a second encoding bitrate based on the calculated delay cost function;

encoding a second portion of the source audiovisual data into a second portion of transmission audiovisual data at the second encoding bitrate;

buffering the second portion of transmission audiovisual data; and

transmitting the second portion of transmission audiovisual data to the client device through the data network.

2. The method of claim 1, wherein encoding the first and second portions of the source audiovisual data is performed according to an MPEG standard.

3. The method of claim 1, wherein the client device comprises at least one of a television set top box, a television, a personal computer, a tablet computer, a smartphone, and an optical disc player.

4. The method of claim 1, wherein the unmanaged, buffered data network includes at least one of a cable data network, a broadcast wireless data network, a point-to-point wireless data network, a satellite network, and a portion of the Internet.

5. The method of claim 1, wherein the unmanaged, buffered data network is coupled to a managed data network that is capable of providing interactive television signals.

6. The method of claim 1, wherein transmitting the first and second portions of transmission audiovisual data includes dividing the data of each video frame into at least one frame portion, the method further comprising:

waiting to receive, from the client device, for each frame portion, an acknowledgement that the frame portion has been received by the client device, wherein the delay cost function is calculated using a length of time between the completion of the encoding of a frame portion and receipt from the client device of the acknowledgement of the frame portion.

7. The method of claim 1, wherein transmitting the first and second portions of transmission audiovisual data is performed in accordance with the Transmission Control Protocol (hereinafter “

TCP”

), further comprising calculating an estimated available bandwidth as

estimate=8*mss*cwnd/rtt,

where “

mss”

is a TCP Maximum Segment Size, “

cwnd”

is a TCP Congestion Window Size, and “

rtt”

is a round trip time associated with the data network; and

establishing the second encoding bitrate as being equal to the value of the ratio of “

estimate”

to the delay cost function.

8. The method of claim 1, wherein transmitting the first and second portions of transmission audiovisual data is performed in accordance with the Transmission Control Protocol (hereinafter “

TCP”

), further comprising calculating an estimated available bandwidth as

estimate=8*mss*cwnd/rtt,

where “

mss”

is a TCP Maximum Segment Size, “

cwnd”

is a TCP Congestion Window Size, and “

rtt”

is a round trip time associated with the data network; and

establishing the second encoding bitrate as being equal to the value of the ratio of “

estimate”

to the delay cost function;

wherein the source audiovisual data includes source audio data having an audio bitrate and source video frames having a video encoding bitrate, and the altering includes altering the video encoding bitrate so that the sum of the audio bitrate and the video encoding bitrate is no greater than the second encoding bitrate.

9. The method of claim 1, wherein the source audiovisual data are generated by an interactive software application according to encoding bitrates including the first and second encoding bitrates, the method further comprising notifying the application of the altering.

10. The method of claim 9, wherein, when the second encoding bitrate is lower than a given threshold defined by the application, the application alters the source audiovisual data by not generating a transparent screen object.

11. The method of claim 9, wherein the source audiovisual data include a graphical user interface, and wherein when the second encoding bitrate is lower than a given threshold defined by the application, the application alters the source audiovisual data by postponing the generation of a dynamic screen region in the graphical user interface.

12. The method of claim 1, further comprising pausing the encoder when the value of the delay cost function falls below a given threshold.

13. A non-transitory computer-usable data storage medium on which is stored computer program code for instructing a server system, that comprises at least one computing processor, to execute a method of controlling playback latency associated with transmission of source audiovisual data through an unmanaged, buffered data network to a client device, the program code comprising:

program code for encoding a first portion of the source audiovisual data into a first portion of transmission audiovisual data at a first encoding bitrate;

program code for buffering the first portion of transmission audiovisual data;

program code for transmitting the first portion of transmission audiovisual data to the client device through the data network;

program code for calculating a delay cost function based on a delay associated with the buffering and with network delay, raised to a power, scaled by a scaling factor, and added to a minimum cost to transmit data a network latency associated with the data;

program code for altering an encoding bitrate from the first encoding bitrate to a second encoding bitrate based on the calculated delay cost function;

program code for encoding a second portion of the source audiovisual data into a second portion of transmission audiovisual data at the second encoding bitrate;

program code for buffering the second portion of transmission audiovisual data; and

program code for transmitting the second portion of transmission audiovisual data to the client device through the data network.

14. The non-transitory computer-usable data storage medium of claim 13, wherein encoding the first and second portions of the source audiovisual data is performed according to an MPEG standard.

15. The non-transitory computer-usable data storage medium of claim 13, wherein the client device comprises at least one of a television set top box, a television, a personal computer, a tablet computer, a smartphone, and an optical disc player.

16. The non-transitory computer-usable data storage medium of claim 13, wherein the unmanaged, buffered data network includes at least one of a cable data network, a broadcast wireless data network, a point-to-point wireless data network, a satellite network, and a portion of the Internet.

17. The non-transitory computer-usable data storage medium of claim 13, wherein the unmanaged, buffered data network is coupled to a managed data network that is capable of providing interactive television signals.

18. The non-transitory computer-usable data storage medium of claim 13, wherein the program code for transmitting the first and second portions of transmission audiovisual data includes program code for dividing the data of each video frame into at least one frame portion, the medium further comprising:

program code for waiting to receive, from the client device, for each frame portion, an acknowledgement that the frame portion has been received by the client device, wherein the delay cost function is calculated using a length of time between the completion of the encoding of a frame portion and receipt from the client device of the acknowledgement of the frame portion.

19. The non-transitory computer-usable data storage medium of claim 13, wherein transmitting the first and second portions of transmission audiovisual data is performed in accordance with the Transmission Control Protocol (hereinafter “

TCP”

), further comprising program code for calculating an estimated available bandwidth as

estimate=8*mss*cwnd/rtt,

where “

mss”

is a TCP Maximum Segment Size, “

cwnd”

is a TCP Congestion Window Size, and “

rtt”

is a round trip time associated with the data network; and

for establishing the second encoding bitrate as being equal to the value of the ratio of “

estimate”

to the delay cost function, and wherein altering the encoding of the source audiovisual data includes encoding the source audiovisual data using the established encoding bitrate.

20. The non-transitory computer-usable data storage medium of claim 13, wherein transmitting the first and second portions of transmission audiovisual data is performed in accordance with the Transmission Control Protocol (hereinafter “

TCP”

), further comprising program code for calculating an estimated available bandwidth as

estimate=8*mss*cwnd/rtt,

where “

mss”

is a TCP Maximum Segment Size, “

cwnd”

is a TCP Congestion Window Size, and “

rtt”

is a round trip time associated with the data network; and

for establishing the second encoding bitrate as being equal to the value of the ratio of “

estimate”

to the delay cost function;

wherein the source audiovisual data includes source audio data having an audio bitrate and source video frames having a video encoding bitrate, and the altering includes altering the video encoding bitrate so that the sum of the audio bitrate and the video encoding bitrate is no greater than the second encoding bitrate.

21. The non-transitory computer-usable data storage medium of claim 13, wherein the source audiovisual data are generated by an interactive software application according to encoding bitrates including the first and second encoding bitrates, the medium further comprising program code for notifying the application of the altering.

22. The non-transitory computer-usable data storage medium of claim 21, wherein, when the second encoding bitrate is lower than a given threshold defined by the application, the application alters the source audiovisual data by not generating a transparent screen object.

23. The non-transitory computer-usable data storage medium of claim 21, wherein the source audiovisual data include a graphical user interface, and wherein when the second encoding bitrate is lower than a given threshold defined by the application, the application alters the source audiovisual data by postponing the generation of a dynamic screen region in the graphical user interface.

24. The non-transitory computer-usable data storage medium of claim 13, further comprising program code for pausing the encoder when the value of the delay cost function falls below a given threshold.

25. A server system for controlling playback latency associated with transmission of source audiovisual data through an unmanaged, buffered data network to a client device, the server system comprising:

an interactive application, executing on one or more computing devices, the interactive application being controlled by commands received from the client device to generate source audiovisual data;

an encoder for encoding the source audiovisual data into transmission audiovisual data according to an encoding bitrate;

a transmission buffer for buffering the transmission audiovisual data;

a transmitter for transmitting the transmission audio data through the data network; and

a delay cost module for calculating a delay cost function based on a delay associated with the buffering and with network delay, raised to a power, scaled by a scaling factor, and added to a minimum cost to transmit data, the delay cost module calculating the encoding bitrate based on the delay cost function and providing the encoding bitrate to the encoder.

26. The server system of claim 25, wherein the client device comprises at least one of a television set top box, a television, a personal computer, a tablet computer, a smartphone, and an optical disc player.

27. The server system of claim 25, wherein the unmanaged, buffered data network includes at least one of a cable data network, a broadcast wireless data network, a point-to-point wireless data network, a satellite network, and a portion of the Internet.

28. The server system of claim 25, wherein the unmanaged, buffered data network is coupled to a managed data network that is capable of providing interactive television signals.