Real-time network monitoring and subscriber identification with an on-demand appliance

• US 8,792,347 B2
• Filed: 05/31/2013
• Issued: 07/29/2014
• Est. Priority Date: 06/01/2012
• Status: Active Grant

• Current Assignee: Otello Corp. ASA
• Original Assignee: Opera Software Ireland Limited (Otello Corp. ASA)
• Inventors: Swenson, Erik R., Bhandari, Nitin
• Primary Examiner(s): PHAN, MAN U
• Application Number:
  US13/907,847
• Publication Number:
  US 20130322242A1
• Time in Patent Office: 424 Days
• Field of Search: 370230-232, 370237-359, 370401-465, 709223-229, 709232-241
• US Class Current: 370/232
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 43/0882 Utilisation of link capacity
• CPC Class Codes: H04L 47/11 Identifying congestion
• CPC Class Codes: H04L 47/12 Avoiding congestion; Recovering from congestion
• CPC Class Codes: H04L 47/2441 relying on flow classification, e.g. using integrated services [IntServ]
• CPC Class Codes: H04L 47/823 Prediction of resource usage
• CPC Class Codes: H04L 47/83 based on usage prediction
• CPC Class Codes: H04N 21/64738 Monitoring network characteristics, e.g. bandwidth, congestion level data switched network analysis H04L41/14; monitoring functioning in data switched networks H04L43/0817; flow control in packet networks H04L47/10
• CPC Class Codes: H04W 24/00 Supervisory, monitoring or testing arrangements
• CPC Class Codes: H04W 24/08 Testing, supervising or monitoring using real traffic
• CPC Class Codes: H04W 24/10 Scheduling measurement reports ; Arrangements for measurement reports
• CPC Class Codes: H04W 28/06 Optimizing the usage of the radio link, e.g. header compression, information sizing , discarding information system modifying transmission characteristic according to link quality by modifying frame length H04L1/0007; dynamic adaptation of the packet size for flow control or congestion control H04L47/365

Abstract

A system and a method are disclosed for selectively monitoring traffic in a service provider network. The system receives a notice for a beginning of a network data flow, which responds to a request from a user device for content at an origin server. The system then determines whether to monitor the data flow from the origin server to the user device. If so determined, the system collects statistic information of the data flow and stores the statistic information to a flow record in a database. The system also maps the flow record to a subscriber of the service provider network by analyzing the statistic information of the data flow and estimates bandwidth provided to the data flow by the service provider'"'"'s network based on the analysis of the statistic information of the data flow.

Claims

1. A method for selectively monitoring traffic in a service provider network, the method comprising:

receiving a notice for a beginning of a network data flow between an origin server and a user device, the data flow for delivering media content from the origin server to the user device in response to a request for the media content from the user device;

determining whether to monitor the data flow between the origin server and the user device;

responsive to a determination to monitor the data flow, collecting statistic information corresponding to the data flow;

storing the statistic information collected for the data flow to a flow record in a database;

mapping the flow record in the database to a subscriber of the service provider network based on an analysis of the statistic information collected for the data flow, where mapped flow records are aggregated for the subscriber;

determining historical bandwidth provided to the subscriber based on the mapped flow records aggregated for the subscriber; and

estimating bandwidth to be consumed on the service provider network by the data flow between the user device and the origin server based on the statistic information collected for the data flow and the historical bandwidth provided to the subscriber.

2. The method of claim 1, wherein determining whether to monitor the data flow comprises determining whether the media content in the data flow indicates transfer of video and/or images larger than a predetermined size.

3. The method of claim 1, wherein the statistical information collected for the data flow includes at least one of throughput, latency, size of the media content to be delivered, and duration of the flow.

4. The method of claim 1, wherein the flow record comprises at least one of:

user device source IP address, subscriber identifier, estimated bandwidth, cookie hashes, flow identifier, and transmitted and received bytes, and wherein the flow record is updated during a lifespan of the data flow.

5. The method of claim 1, wherein estimating the bandwidth to be consumed on the service provider network by the data flow comprises:

receiving an update on the transmitted or received bytes to the flow record; and

calculating the bandwidth by dividing a difference between the transmitted or received bytes since last update by an interval of time since the last update.

6. The method of claim 1, further comprising:

summing up the estimated bandwidth of monitored data flows in the service provider network; and

determining a congestion level of the service provider network by comparing the sum of bandwidth of the monitored data flows against a capacity of the service provider network.

7. The method of claim 6, further comprising:

determining whether to optimize the data flow based on the congestion level of the service provider network and a size of the media content to be delivered in the data flow; and

redirecting the request for the media content from the requesting user device to a video optimizer, wherein the video optimizer retrieves the media content from the origin server, optimizes the media content for the user device, and transmits the optimized media content to the user device.

8. The method of claim 1, wherein aggregating flow records corresponding to the subscriber in the database of the service provider network is based on matching at least one of:

user device source IP address and MAC address, TCP port number, and hashes of a cookie associated with the data flows.

9. A non-transitory computer-readable storage medium storing executable computer program instructions for selectively monitoring traffic in a service provider network, the computer program instructions comprising instructions for:

receiving a notice for a beginning of a network data flow between an origin server and a user device, the data flow for delivering media content from the origin server to the user device in response a response to a request for the media content from the user device;

determining whether to monitor the data flow between the origin server and the user device;

responsive to a determination to monitor the data flow, collecting statistic information corresponding to the data flow;

storing the statistic information collected for the data flow to a flow record in a database;

mapping the flow record in the database to a subscriber of the service provider network based on an analysis of the statistic information collected for the data flow, where mapped flow records are aggregated for the subscriber;

determining historical bandwidth provided to the subscriber based on the mapped flow records aggregated for the subscriber; and

estimating bandwidth to be consumed on the service provider network by the data flow between the user device and the origin server based on the statistic information collected for the data flow and the historical bandwidth provided to the subscriber.

10. The non-transitory computer-readable storage medium of claim 9, wherein determining whether to monitor the data flow comprises determining whether the media content in the data flow indicates transfer of video and/or images larger than a predetermined size.

11. The non-transitory computer-readable storage medium of claim 9, wherein the statistical information collected for the data flow includes at least one of throughput, latency, size of the media content to be delivered, and duration of the flow.

12. The non-transitory computer-readable storage medium of claim 9, wherein the flow record comprises at least one of:

user device source IP address, subscriber identifier, estimated bandwidth, cookie hashes, flow identifier, and transmitted and received bytes, and wherein the flow record is updated during a lifespan of the data flow.

13. The non-transitory computer-readable storage medium of claim 9, wherein estimating the bandwidth to be consumed on the service provider network by the data flow comprises:

receiving an update on the transmitted or received bytes to the flow record; and

calculating the bandwidth by dividing a difference between the transmitted or received bytes since last update by an interval of time since the last update.

14. The non-transitory computer-readable storage medium of claim 9, the computer program instruction further comprising instruction for:

summing up the estimated bandwidth of monitored data flows in the service provider network; and

determining a congestion level of the service provider network by comparing the sum of bandwidth of monitored data flows against a capacity of the service provider network.

15. The non-transitory computer-readable storage medium of claim 14, the computer program instruction further comprising instruction for:

determining whether to optimize the data flow based on the congestion level of the service provider network and a size of the media content to be delivered in the data flow; and

redirecting the request for the media content from the requesting user device to a video optimizer, wherein the video optimizer retrieves the media content from the origin server, optimizes the media content for the user device, and transmits the optimized media content to the user device.

16. The non-transitory computer-readable storage medium of claim 9, wherein aggregating flow records corresponding to the subscriber in the database of the service provider network is based on matching at least one of:

user device source IP address and MAC address, TCP port number, and hashes of a cookie associated with the data flows.

17. A system for selectively monitoring traffic in a service provider network, comprising:

a processor; and

a non-transitory computer-readable storage medium storing executable computer program instructions that when executed by the processor;

receive a notice for a beginning of a network data flow between an origin server and a user device, the data flow for delivering media content from the origin server to the user device in response to a request for the media content from the user device;

determine whether to monitor the data flow between the origin server and the user device;

collect statistic information corresponding to the data flow in response to a determination to monitor the data flow;

store the statistic information collected for the data flow to a flow record in a database;

map the flow record in the database to a subscriber of the service provider network based on an analysis of the statistic information collected for the data flow, where mapped flow records are aggregated for the subscriber;

determine historical bandwidth provided to the subscriber based on the mapped flow records aggregated for the subscriber; and

estimate bandwidth to be consumed on the server provider network by the data flow between the user device and the origin server based on the statistic information collected for the data flow and the historical bandwidth provided to the subscriber.

18. The system of claim 17, wherein the system collects statistic information of the data flow as an inline network element that receives and forwards the data flow.

19. The system of claim 17, wherein the system collects receives the data flow from an inline network element in response to a determination to monitor the data flow based on statistic information collected for the data flow, the inline network element forwarding the data flow for processing in response to the determination to monitor the data flow.

20. The system of claim 17, wherein the executable computer program instructions further comprise instructions to:

sum up the estimated bandwidth of monitored data flows in the service provider network; and

determine a congestion level of the service provider network by comparing the sum of bandwidth of the monitored data flows against a capacity of the service provider network.

21. The system of claim 17, wherein the executable computer program instructions further comprise instructions to:

determine whether to optimize the data flow based on the congestion level of the service provider network and a size of the media content in the data flow; and

redirect the request for the media content from the requesting user device to a video optimizer, wherein the video optimizer retrieves the media content from the origin server, optimizes the media content for the user device, and transmits the optimized media content to the user device.