Method and apparatus for measuring stream availability, quality and performance

• US 20040136327A1
• Filed: 02/11/2003
• Published: 07/15/2004
• Est. Priority Date: 02/11/2002
• Status: Active Grant

• Current Assignee: Akamai Technologies, Inc.
• Original Assignee: Akamai Technologies, Inc.
• Inventors: Sitaraman, Ramesh K., Barton, Reid W.
• Granted Patent: US 7,010,598 B2
• Time in Patent Office: Days
• US Class Current: 370/252
• CPC Class Codes: H04L 41/046 comprising network management agents or mobile agents therefor
• CPC Class Codes: H04L 41/5009 Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF]
• CPC Class Codes: H04L 41/5032 Generating service level reports
• CPC Class Codes: H04L 43/00 Arrangements for monitoring or testing data switching networks
• CPC Class Codes: H04L 43/062 related to network traffic
• CPC Class Codes: H04L 43/0805 by checking availability
• CPC Class Codes: H04L 43/0817 by checking functioning
• CPC Class Codes: H04L 43/0829 Packet loss
• CPC Class Codes: H04L 43/0847 Transmission error
• CPC Class Codes: H04L 43/0888 Throughput
• CPC Class Codes: H04L 43/0894 Packet rate
• CPC Class Codes: H04L 43/106 using time related information in packets, e.g. by adding timestamps
• CPC Class Codes: H04L 43/12 Network monitoring probes
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/80 Responding to QoS

Abstract

A streaming measurement agent designed to experience, measure, and report on a media stream as an actual end user would experience the stream. Preferably, agent resides transparently within a streaming media player itself so that it can monitor stream packet flows within the player as the measured streams are being played. In an illustrative embodiment, the agent comprises a performance monitoring module (PMM), which is software that resides in an interface between an existing core module and a renderer of a media player. The agent PMM intercepts each useful packet as it goes from the core to the renderer and, as a result, it is able to compute quality metrics about the playback. The agent functions “transparently” to the media player by presenting the core with an application programming interface (API) that emulates the API that the renderer normally presents to the core. Thus, when the core believes it is calling the renderer, it is actually calling the agent PMM, which can then receive all the packets from the core and process them. After computing relevant performance metrics using the packets it receives, the agent PMM calls the renderer. A set of performance agents can be managed by a service provider to enable a content provider to determine how a stream is perceived by end users.

Claims

1. A method for measuring the quality of a stream of data packets received by a media player, the media player having a first component that receives the data packets, and a second component that receives the data packets from the first component and renders the data contained therein, comprising;

intercepting a set of useful data packets of a given stream as each useful data packet of the set exits the first component and before such useful data packet enters the second component; and

capturing a first stream quality metric from the intercepted useful data packets.

2. The method as described in claim 1 wherein the first stream quality metric is a statistic computed from the intercepted useful data packets.

3. The method as described in claim 2 wherein the first stream quality metric is actual playback bandwidth.

4. The method as described in claim 2 wherein the first stream quality metric is useful packet rate.

5. The method as described in claim 1 further including the step of generating a second stream quality metric from a media player callback function.

6. The method as described in claim 5 wherein the second stream quality metric is selected from a set of metrics that include:

startup time, startup time components, and interruption and rebuffer statistics.

7. The method as described in claim 1 further including the step of outputting the stream quality metric.

8. Apparatus for use in monitoring stream performance in a streaming media player having a first component that receives the data packets, and a second component that receives the data packets from the first component and renders the data contained therein, comprising:

code for intercepting each useful packet of a given stream as the useful packet exits the first component and before the useful packet enters the second component; and

code for capturing a given metric from the intercepted useful packets, wherein the given metric is a statistic computed from the intercepted useful data packets.

9. The apparatus as described in claim 8 further including code for reading a test configuration, wherein each test configuration includes at least one test suite.

10. The apparatus as described in claim 9 wherein each test suite includes a set of parameters selected from a set that includes:

a start time, and at least one test.

11. The apparatus as described in claim 10 wherein the test includes a set of parameters selected from a set that includes:

a URL, a maximum startup time, a maximum stream time, and a maximum play time.

12. The apparatus as described in claim 9 further including code for scheduling the test suite.

13. The apparatus as described in claim 8 further including code for outputting the given stream quality metric.

14. Apparatus, comprising:

a core;

a renderer; and

a first set of code located intermediate the core and the renderer for generating a first metric as a stream of useful data packets are passed between the core and the renderer, wherein the first metric is a statistic computed from the intercepted useful data packets.

15. The apparatus as described in claim 14 further including a second set of code for receiving a test suite identifying at least one stream quality test, for scheduling the test suite, and for outputting the first metric captured by the first set of code.

16. The apparatus as described in claim 14 wherein the first set of code generates a second metric using a player callback function.

17. The apparatus as described in claim 14 wherein each useful data packet has a given set of properties, wherein the properties are that the packet has not been thinned by a server, that the packet was not lost in an unrecoverable manner during transport, and that the packet arrived at the player in time to be useful for rendering.

18. A stream monitoring system, comprising:

a first sub-system for provisioning a given stream quality test;

a set of monitoring agents positioned at given network locations for executing the given stream quality test, each of the monitoring agents associated with a streaming media player having a core and a renderer, each agent comprising;

code for intercepting each useful packet of a given stream as the useful packet exits the core and before the useful packet enters the renderer;

code for capturing a given stream quality metric from the intercepted packets; and

code for outputting data associated with the given stream quality metric;

a second sub-system for collecting the data from the set of monitoring agents.

19. A stream monitoring method, comprising:

delivering a media stream to each of a set of agents, each agent having a core and a renderer;

at each agent, intercepting useful packets of the stream as those packets cross an interface between the core and the renderer;

capturing a given stream quality metric from the intercepted useful packets; and

collecting data from each of the agents.

20. The stream monitoring method as described in claim 19 further including the step of:

enabling a view of the collected data.