Method and apparatus for measuring stream availability, quality and performance

• US 7,725,576 B2
• Filed: 03/06/2006
• Issued: 05/25/2010
• Est. Priority Date: 02/11/2002
• Status: Active Grant

• Current Assignee: Akamai Technologies, Inc.
• Original Assignee: Akamai Technologies, Inc.
• Inventors: Barton, Reid W., Sitaraman, Ramesh K.
• Primary Examiner(s): Nguyen; Quang N.
• Assistant Examiner(s): Yi; David
• Application Number:
  US11/368,835
• Publication Number:
  US 20060179154A1
• Time in Patent Office: 1,541 Days
• Field of Search: 709201-203, 709/204, 709217-219, 709/224, 709/231, 709/246
• US Class Current: 709/224
• CPC Class Codes: H04L 41/046 comprising network management agents or mobile agents therefor
• CPC Class Codes: H04L 41/5009 Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF]
• CPC Class Codes: H04L 41/5032 Generating service level reports
• CPC Class Codes: H04L 43/00 Arrangements for monitoring or testing data switching networks
• CPC Class Codes: H04L 43/062 related to network traffic
• CPC Class Codes: H04L 43/0805 by checking availability
• CPC Class Codes: H04L 43/0817 by checking functioning
• CPC Class Codes: H04L 43/0829 Packet loss
• CPC Class Codes: H04L 43/0847 Transmission error
• CPC Class Codes: H04L 43/0888 Throughput
• CPC Class Codes: H04L 43/0894 Packet rate
• CPC Class Codes: H04L 43/106 using time related information in packets, e.g. by adding timestamps
• CPC Class Codes: H04L 43/12 Network monitoring probes
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/80 Responding to QoS

Abstract

A streaming measurement agent designed to experience, measure, and report on a media stream as an actual end user would experience the stream. Preferably, agent resides transparently within a streaming media player itself so that it can monitor stream packet flows within the player as the measured streams are being played. In an illustrative embodiment, the agent comprises a performance monitoring module (PMM), which is software that resides in an interface between an existing core module and a renderer of a media player. The agent PMM intercepts each useful packet as it goes from the core to the renderer and, as a result, it is able to compute quality metrics about the playback. The agent functions “transparently” to the media player by presenting the core with an application programming interface (API) that emulates the API that the renderer normally presents to the core. Thus, when the core believes it is calling the renderer, it is actually calling the agent PMM, which can then receive all the packets from the core and process them. After computing relevant performance metrics using the packets it receives, the agent PMM calls the renderer. A set of performance agents can be managed by a service provider to enable a content provider to determine how a stream is perceived by end users.

Claims

1. A stream monitoring system, comprising:

one or more stream monitoring computers including a first sub-system for provisioning a stream quality test;

a set of monitoring agents positioned at given network locations for executing the stream quality test, each of the monitoring agents associated with a streaming media player having a core and a renderer, each agent comprising;

code executed in association with the streaming media player for intercepting each useful packet of a given stream as the useful packet exits the core and before the useful packet enters the renderer, wherein each useful packet has a given set of properties, wherein the properties are that the packet has not been thinned by a server, that the packet was not lost in an unrecoverable manner during transport, and that the packet arrived at the renderer in time to be useful for rendering;

code for capturing a given stream quality metric from the intercepted useful packets; and

code for outputting data associated with the given stream quality metric;

the one or more stream monitoring computers also including a second sub-system for collecting the data from the set of monitoring agents.

2. The system as described in claim 1 wherein the stream quality metric is actual playback bandwidth.

3. The system as described in claim 1 wherein the stream quality metric is useful packet rate.

4. The system as described in claim 1 wherein the code for outputting data also generates a metric using a player callback function.

5. The system as described in claim 4 wherein the metric generated using the player callback function is selected from a set of metrics that include:

startup time, startup time components, and interruption and rebuffer statistics.

6. The system as described in claim 1 wherein each monitoring agent further includes code for receiving a test suite identifying at least one stream quality test, for scheduling the test suite, and for outputting the first metric captured by the first set of code.

7. The system as described in claim 6 wherein the test suite includes a set of parameters selected from a set that includes:

a start time, and at least one test.

8. The system as described in claim 7 wherein the stream quality test includes a set of parameters selected from a set that includes:

a URL, a maximum startup time, a maximum stream time, and a maximum play time.

9. A stream monitoring method, comprising:

delivering a media stream to each of a set of media players, each media player having a core and a renderer;

at each media player, intercepting useful packets of the stream as those packets cross an interface between the core and the renderer, wherein each useful packet has a given set of properties, wherein the properties are that the packet has not been thinned by a server, that the packet was not lost in an unrecoverable manner during transport, and that the packet arrived at the renderer in time to be useful for rendering;

capturing a given stream quality metric from the intercepted useful packets;

outputting data associated with the given stream quality metric; and

collecting data from each of the media players at a set of one or more stream monitoring computers.

10. The stream monitoring method as described in claim 9 further including the step of:

enabling a view of the collected data.

11. The stream monitoring method as described in claim 9 wherein the media stream is delivered over a content delivery network operated by a service provider.