Multimedia signal latency management by skipping

• US 8,626,499 B2
• Filed: 07/20/2010
• Issued: 01/07/2014
• Est. Priority Date: 07/21/2009
• Status: Expired due to Fees

• Current Assignee: Polycom Incorporated (HP Inc.)
• Original Assignee: ViVu, Inc. (HP Inc.)
• Inventors: Yellamraju, SivaKiran Venkata, Katasani, Simha Sundeep Reddy
• Primary Examiner(s): Harrell, Robert B
• Application Number:
  US12/840,293
• Publication Number:
  US 20110022705A1
• Time in Patent Office: 1,267 Days
• Field of Search: 709/224
• US Class Current: 704/224
• CPC Class Codes: H04L 65/1069 Session establishment or de-establishment
• CPC Class Codes: H04L 65/60 Network streaming of media packets
• CPC Class Codes: H04L 65/756 adapting media to device capabilities
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 65/80 Responding to QoS

Abstract

A method or system for enabling client devices connected to a network to subscribe to a one or more multimedia signals available in a selection of bandwidths, based on an analysis of capabilities of the client, the quality of the connection, and the real-time changes in the ability of the client to process the signal.

Claims

1. A method executing on hardware of a client device coupled to a network for managing the impact of signal latency on the reception of a multimedia signal by the client device, the method comprising:

calculating, on the client device, a first moving average of the signal latency over a first period of time;

calculating, on the client device, a minimum of a first moving average of the latency over a session;

measuring, on the client device, a change in signal latency over some configured period of time;

buffering, on the client device, a part of the signal having a length corresponding to a selected period of time prior to presenting it;

skipping playback, on the client device, of a defined period of time of the signal if the change in signal latency exceeds some configured value;

skipping playback, on the client device, of the defined period of time of the signal if the signal latency exceeds a first configured percentage of the measurement of the minimum of the first moving average; and

skipping playback, on the client device, of the defined period of time of the signal if the signal latency exceeds a second configured percentage of the first moving average of the signal latency a third configured period of time after the last time a part of the signal was skipped.

2. The method in claim 1, where the defined period of time for which playback is skipped is smaller than the buffer length.

3. The method in claim 2, where the defined period of time for which playback is skipped is less than 0.5 seconds.

4. The method in claim 1, where the first period of time over which the first moving average of the signal latency is calculated is shorter than the size of the buffer but longer than the second configured period of time.

5. The method in claim 4, where the first period of time over which the first moving average of the signal latency is calculated is at least 3 seconds but not more than 10 seconds, and the second period of time is at least 30 seconds but not more than 2 minutes.

6. The method in claim 1, where method further comprises:

calculating, on the client device, a second moving average over a second period of time, wherein the second period of time is longer than the first period of time;

shorter than the expected length of a session; and

skipping, on the client device, the defined period of time of the signal if the first moving average exceeds the second moving average at least the second period of time after the last time a part of the signal was skipped.

7. The method in claim 1, where the determining of the measurement of the minimum signal latency further comprises initializing the minimum signal latency as a current latency measured some configured time into the current session, thereafter for the duration of the session comparing a current value of the minimum signal latency to a current latency value, and setting the minimum signal latency value to the current latency value if the current latency value is less that the minimum signal latency value.

8. The method in claim 1, where the selected period of time corresponding to the length of the buffered signal is larger than the expected signal latency but smaller than the expected length of a session.

9. The method in claim 1, wherein buffering a part of the signal having a length corresponding to a selected period of time comprises buffering no less than 15 seconds and no more than 1 minute of the signal.

10. The method in claim 1, where the change in signal latency is calculated by comparing the latency measured at the first configured period of time prior to the most recent measurement.

11. The method in claim 1, where the first configured percentage is at least 50% and no more than 200%.

12. The method in claim 1, where the second configured percentage is at least 25% and no more than 100%.

13. The method in claim 1, wherein the third configured period of time is at least the length of time of the first period of time.

14. A method executing on hardware of a client device for maintaining the output quality of a media stream received and presented by the client device, the method comprising:

initializing a signal buffer on the client device to hold at least 30 seconds of a media signal at the start of a session;

measuring, on the client device, the latency of the signal processing relative to the realtime position in the signal buffer;

calculating, on the client device, the derivative of the latency over time;

calculating, on the client device, a first moving average of the latency over the first 30 second time period;

defining, on the client device, a first threshold associated with the first moving average, greater than the first moving average by some fixed percentage; and

determining, on the client device, if the absolute value of the signal latency is above the first threshold, and if so commanding the device performing the signal processing to perform a series of skips of the signal over a first period until the derivative of the signal latency is no longer exceeds the first threshold.

15. The method in claim 14, further comprising:

calculating, on the client device, a second moving average of the derivative of the latency over the most recent 5 second time period;

calculating, on the client device, a third moving average of the derivative of the latency over the most recent 30 second time period; and

determining, on the client device, if the second moving average exceeds the third moving average, and if so commanding the device performing the signal processing to perform a series of skips of the signal over a first period until the third moving average exceeds the second moving average.

16. The method in claim 15, further comprising:

defining, on the client device, a first value, associated with expected number of skips over a first period of time greater than the largest moving average but less than 10 minutes;

defining, on the client device, a second value associated with the first value, greater than the first value by some fixed percentage;

defining, on the client device, a third value associated with the first value, less than the first value by some fixed percentage;

determining, on the client device, if the number of skips over the last first period of time exceeds the second value, and if so command the device performing the signal processing to select a signal of lower quality if available, or stop playing otherwise; and

determining, on the client device, if the number of skips over the last first period of time is less than the third value, and if so command the device performing the signal processing to select a signal of greater quality if available.

17. The method in claim 15, further comprising:

defining, on the client device, a fourth value associated with the maximum number of skips allowed over some period of time, between the first value and the second value;

defining, on the client device, a fifth value associated with the minimum number of skips allowed over some period of time, between the first value and the third value;

defining, on the client device, a sixth value associated with the initial time between signal skips;

determining, on the client device, if the number of skips over the last first period of time exceeds the fourth value, and if so command the device performing the signal processing to skip a larger portion of the signal over a period larger than the sixth value, both increased by a fixed percentage, until the number of skips over the last period of time is below the fourth value; and

determining, on the client device, if the number of skips over the last first period of time is less than the fifth value, and if so command the device performing the signal processing to skip a smaller portion of the signal over a period less than the sixth value, both decreased by a fixed percentage, until the number of skips over the last period of time is above the fifth value.

18. The method in claim 15, further comprising:

determining, on the client device, if the derivative of the signal latency exceeds 150% of the first moving average, and if so commanding the device performing the signal processing to perform a series of skips of the signal over a first period until the derivative of the signal latency no longer exceeds 150% of the first moving average.

19. A computer readable medium having stored thereon instructions that, when executed by a computer, causes the computer to perform a method for maintaining the output quality of a media stream, the medium comprising:

programming instructions for initializing a signal buffer to hold at least 30 seconds of a media signal at the start of a session;

programming instructions for measuring the latency of the signal processing relative to the real-time position in the signal buffer;

programming instructions for calculating the derivative of the latency over time;

programming instructions for calculating a first moving average of the latency over the first 30 second time period;

programming instructions for defining a first threshold associated with the first moving average, greater than the first moving average by some fixed percentage; and

programming instructions for determining if the absolute value of the signal latency is above the first threshold, and if so commanding the device performing the signal processing to perform a series of skips of the signal over a first period until the derivative of the signal latency is no longer exceeds the first threshold.

20. The computer readable media in claim 19, further comprising:

programming instructions for calculating a second moving average of the derivative of the latency over the most recent 5 second time period;

programming instructions for calculating a third moving average of the derivative of the latency over the most recent 30 second time period; and

programming instructions for determining if the second moving average exceeds the third moving average, and if so commanding the device performing the signal processing to perform a series of skips of the signal over a first period until the third moving average exceeds the second moving average.

21. The computer readable media in claim 19, further comprising:

programming instructions for defining a first value, associated with expected number of skips over a first period of time greater than the largest moving average but less than 10 minutes;

programming instructions for defining a second value associated with the first value, greater than the first value by some fixed percentage;

programming instructions for defining a third value associated with the first value, less than the first value by some fixed percentage;

programming instructions for determining if the number of skips over the last first period of time exceeds the second value, and if so command the device performing the signal processing to select a signal of lower quality if available, or stop playing otherwise; and

programming instructions for determining if the number of skips over the last first period of time is less than the third value, and if so command the device performing the signal processing to select a signal of greater quality if available.

22. The computer readable media in claim 19, further comprising:

programming instructions for defining a fourth value associated with the maximum number of skips allowed over some period of time, between the first value and the second value;

programming instructions for defining a fifth value associated with the minimum number of skips allowed over some period of time, between the first value and the third value;

programming instructions for defining a sixth value associated with the initial time between signal skips;

programming instructions for determining if the number of skips over the last first period of time exceeds the fourth value, and if so command the device performing the signal processing to skip a larger portion of the signal over a period larger than the sixth value, both increased by a fixed percentage, until the number of skips over the last period of time is below the fourth value; and

programming instructions for determining if the number of skips over the last first period of time is less than the fifth value, and if so command the device performing the signal processing to skip a smaller portion of the signal over a period less than the sixth value, both decreased by a fixed percentage, until the number of skips over the last period of time is above the fifth value.

23. The computer readable media of claim 19, further comprising:

computer instructions for determining if the derivative of the signal latency exceeds 150% of the first moving average, and if so commanding the device performing the signal processing to perform a series of skips of the signal over a first period until the derivative of the signal latency no longer exceeds 150% of the first moving average.