Integrated adaptive jitter buffer

• US 8,279,884 B1
• Filed: 06/18/2007
• Issued: 10/02/2012
• Est. Priority Date: 11/21/2006
• Status: Active Grant

• Current Assignee: PICO Mobile Networks Incorporated
• Original Assignee: PICO Mobile Networks Incorporated
• Inventors: Narasimha, Madihally J., Chang, Lu
• Primary Examiner(s): Ton, Dang
• Assistant Examiner(s): Kaur, Pamit
• Application Number:
  US11/764,790
• Time in Patent Office: 1,933 Days
• Field of Search: 370/412, 370/351, 370/230
• US Class Current: 370/412
• CPC Class Codes: H04L 47/22 Traffic shaping
• CPC Class Codes: H04L 47/283 in response to processing delays, e.g. caused by jitter or round trip time [RTT]
• CPC Class Codes: H04L 47/365 Dynamic adaptation of the packet size

Abstract

The present invention is a method of correcting packet discontinuities using the steps of:

• (A) generating a continuous real time data stream from input of media content from a media source comprising packets transmitted by way of a computer packet network to a specific receiving device to establish a transmission portion of an end to end communication,
• (B) a jitter buffer receiving real time data stream packets from the packet network and temporarily storing at least some of them in the jitter buffer,
• (C) the jitter buffer operating on multiple fixed length packets to output a first output of a predetermined sequence of said fixed length packets, preferably substantially as they were originally transmitted,
• (D) a control unit receiving the first output and changing the length of one or more of fixed length packets of the first output to form a second output in response to a detected delay or other discontinuity in the packet sequence,
• (E) a playout buffer receiving the second output and operating on the stream of original and varied length packets to deliver them to a digital to analog converter (DAC), and
• (F) transmission of analog output of the DAC to interface devices such as displays, speakers, and mechanical devices for intelligible playout of the media content for appreciation by a human interacting with the interface devices.

Claims

1. A system comprising:

an input/output interface configured to receive a sequence of packets from a media source over a packet network;

a control device configured to

store the sequence of packets in a jitter buffer,

determine a first set of statistics of the jitter buffer prior to storing the sequence of packets in the jitter buffer, and

generate a first signal output including the sequence of packets based on the first set of statistics;

a decoder configured to

receive the first signal output, and

decode packet data of the first signal output to generate a second signal output, the second signal output including a silence period based on the first signal output, the silence period being between a leading waveform and a trailing waveform;

a signal processing device configured to

receive the second signal output,

sample at least one of the leading waveform and the trailing waveform to generate an expansion block,

fill the silence period with the expansion block to generate an expanded output, and

sample at least one of the leading waveform and the trailing waveform to generate a compressed block and a compressed output based on the second signal output; and

a transmitter configured to transmit one of the expanded output and the compressed output from a playout buffer.

2. The system of claim 1, wherein:

the control device is configured to detect absence of a packet in the sequence of packets prior to playout of packets stored by a playout device and generate a discontinuity message based on the absence of the packet; and

the signal processing device is configured to generate the expansion block based on the discontinuity message.

3. The system of claim 1, wherein the control device is configured to:

detect clock skew associated with the sequence of packets based on the first set of statistics and prior to playout of packets stored in a playout buffer by a playout device;

generate one of a compression message and an expansion message based on the clock skew; and

generate the first signal output to include one of the compression message and the expansion message.

4. The system of claim 3, wherein the signal processing device is configured to one of (i) generate the expansion block based on the expansion message and (ii) generate the compression block based on the compression message.

5. The system of claim 1, wherein the expanded output and the compressed output comprise only one of audio packets and video packets.

6. The system of claim 1, wherein the control device is configured to:

calculate a first difference between a predetermined buffer delay and an average buffer delay;

determine a current fill level of the jitter buffer;

expand playout of decoded waveforms when the difference exceeds a first maximum value and the current fill level is less than a second maximum level; and

compress playout of decoded waveforms when the difference is less than the first maximum value and the current fill level exceeds the second maximum value.

7. The system of claim 1, wherein:

the control device is configured to generate the first signal output to include one of an expansion message and a compression message based on the first set of statistics; and

the signal processing device is configured to one of expand and compress the second signal output based on one of the expansion message and the compression message.

8. The system of claim 1, wherein:

the signal processing device is configured to generate one of the expansion block and the compression block based on the first set of statistics; and

the first set of statistics comprises at least one of

a jitter buffer delay of the jitter buffer, and

a current fill level of the jitter buffer.

9. The system of claim 1, wherein the control device is configured to:

determine a first filtered jitter buffer delay;

determine a first difference during a first period between the first filtered jitter buffer delay and a predetermined jitter buffer delay;

generate an expansion message when at least one of

the difference is less than a first threshold, and

a current fill level of the jitter buffer is less than a second threshold; and

generate a compression message when at least one of

the difference is greater than a third threshold, and

the current fill level is greater than a fourth threshold.

10. The system of claim 9, wherein the signal processing device is configured to:

determine a second filtered jitter buffer delay;

determine a second difference during a second period between the second filtered jitter buffer delay and the predetermined jitter buffer delay; and

one of expands and compresses the second signal output based on the second filtered jitter buffer delay and the second difference.

11. The system of claim 10, wherein:

the control device is configured to determine the first filtered buffer delay prior to the sequence of packets being stored in the jitter buffer; and

the signal processing device is configured to determine the second filtered jitter buffer delay subsequent to the sequence of packets being stored in the jitter buffer.

12. The system of claim 1, further comprising a playout device configured to store the expanded output and the compressed output in a playout buffer,

wherein the transmitter is configured to transmit one of the expanded output and the compressed output from the playout buffer.

13. The system of claim 1, further comprising a playout device that:

receives the first signal output and sends the first signal output to the decoder;

receives the second signal output from the decoder;

sends the second signal output to the signal processing device to generate one of the expansion block and the compression block;

receives processed signals from the signal processing device based on one of the expansion block and the compression block; and

sends the processed signals to a playout buffer.

14. The system of claim 1, wherein:

the control device is configured to generate the first signal output including the sequence of packets and one of a compression message and an expansion message based on the first set of statistics; and

the signal processing device is configured to at least one of (i) conceal discontinuities of the second signal output or (ii) expand or compress the second signal output, based on the one of the compression message and the expansion message and a second set of statistics of the jitter buffer.

15. The system of claim 14, wherein the second set of statistics is determined subsequent to storing the sequence of packets in the jitter buffer.

16. The system of claim 14, wherein the second signal output includes the sequence of packets.

17. The system of claim 14, wherein the signal processing device is configured to store the one of the compression message and the expansion message with the sequence of packets in the jitter buffer.

18. The system of claim 1, wherein:

the control device is configured to operate in a first mode and in a second mode;

the control device estimates a first network delay based on a second network delay determined prior to the first network delay, a first instantaneous network delay, and a second instantaneous network delay during the first mode; and

the control device estimates the first network delay based on the second network delay, the first instantaneous network delay, and an averaging time parameter during the second mode.

19. The system of claim 18, wherein:

the control device is configured to determine a variance of the first network delay;

the control device is configured to determine a delay of the jitter buffer based on the first network delay and the variance of the first network delay; and

the first set of statistics includes the delay of the jitter buffer.

20. The system of claim 1, wherein the control device:

determines a desired delay of the jitter buffer based on a measured network delay and the first set of statistics;

determines an actual delay of the jitter buffer based on a fill level of the jitter buffer when the sequence of packets is received by the input/output interface;

generates an error signal based on the desired delay and the actual delay; and

generates the first signal output to include one of a compression message and an expansion message based on the error signal.

21. A method comprising:

receiving a sequence of packets from a media source over a network;

storing the sequence of packets in a jitter buffer;

determining a first set of statistics of the jitter buffer prior to the storing of the sequence of packets in the jitter buffer;

generating a first signal output including the sequence of packets based on the first set of statistics;

decoding packet data of the first signal output using a decoder to generate a second signal output, the second signal output comprising a silence period based on the first signal output, the silence period being between a leading waveform and a trailing waveform;

receiving the second signal output;

performing one of

sampling at least one of the leading waveform and the trailing waveform to generate an expansion block and filling the silence period with the expansion block to generate an expanded output, and

sampling at least one of the leading waveform and the trailing waveform to generate a compressed block and to generate a compressed output; and

transmitting one of the expanded output and the compressed output from a playout buffer.

22. The method of claim 21, further comprising:

detecting absence of a packet in the sequence of packets prior to playout of packets stored by a playout device;

generating a discontinuity message based on the absence of the packet; and

generating the expansion block based on the discontinuity message.

23. The method of claim 21, further comprising:

detecting clock skew associated with the sequence of packets based on the first set of statistics and prior to playout of packets stored by a playout device;

generating one of a compression message and an expansion message based on the clock skew; and

generating the first signal output to include one of the compression message and the expansion message.

24. The method of claim 21, further comprising one of (i) generating the expansion block based on the expansion message and (ii) generating the compression block based on the compression message.