Real-time network monitoring and subscriber identification with an on-demand appliance

• US 9,331,944 B2
• Filed: 05/23/2014
• Issued: 05/03/2016
• Est. Priority Date: 06/01/2012
• Status: Active Grant

• Current Assignee: Otello Corp. ASA
• Original Assignee: Opera Software Ireland Limited (Otello Corp. ASA)
• Inventors: Swenson, Erik R., Bhandari, Nitin
• Primary Examiner(s): PHAN, MAN U
• Application Number:
  US14/286,032
• Publication Number:
  US 20140254380A1
• Time in Patent Office: 711 Days
• Field of Search: 370232-252, 370328-343, 709224-230
• US Class Current: 1/1
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 43/0882 Utilisation of link capacity
• CPC Class Codes: H04L 47/11 Identifying congestion
• CPC Class Codes: H04L 47/12 Avoiding congestion; Recovering from congestion
• CPC Class Codes: H04L 47/2441 relying on flow classification, e.g. using integrated services [IntServ]
• CPC Class Codes: H04L 47/83 based on usage prediction
• CPC Class Codes: H04N 21/64738 Monitoring network characteristics, e.g. bandwidth, congestion level data switched network analysis H04L41/14; monitoring functioning in data switched networks H04L43/0817; flow control in packet networks H04L47/10
• CPC Class Codes: H04W 24/00 Supervisory, monitoring or testing arrangements
• CPC Class Codes: H04W 24/08 Testing, supervising or monitoring using real traffic
• CPC Class Codes: H04W 24/10 Scheduling measurement reports ; Arrangements for measurement reports
• CPC Class Codes: H04W 28/06 Optimizing the usage of the radio link, e.g. header compression, information sizing , discarding information system modifying transmission characteristic according to link quality by modifying frame length H04L1/0007; dynamic adaptation of the packet size for flow control or congestion control H04L47/365

Abstract

A system and a method are disclosed for selectively monitoring traffic in a service provider network. The system receives a notice for a beginning of a network data flow, which responds to a request from a user device for content at an origin server. The system then determines whether to monitor the data flow from the origin server to the user device. If so determined, the system collects statistic information of the data flow and stores the statistic information to a flow record in a database. The system also maps the flow record to a subscriber of the service provider network by analyzing the statistic information of the data flow and estimates bandwidth provided to the data flow by the service provider'"'"'s network based on the analysis of the statistic information of the data flow.

Claims

1. A method for selectively monitoring traffic in a service provider network, the method comprising:

receiving a notice for a beginning of a network data flow between an origin server and a user device, the data flow for delivering media content from the origin server to the user device in response to a request for the media content from the user device;

determining whether to monitor the data flow between the origin server and the user device;

collecting, responsive to a determination to monitor the data flow, statistic information corresponding to the data flow;

storing the statistic information collected for the data flow to a flow record in a database;

mapping the flow record in the database to a subscriber of the service provider network based on an analysis of the statistic information collected for the data flow;

estimating bandwidth to be consumed on the service provider network by the data flow between the user device and the origin server based on the statistic information collected for the data flow;

summing the estimated bandwidth of the data flow and estimated bandwidths of at least one other monitored data flow in the service provider network;

determining a congestion level of the service provider network based on a comparison of the sum of the estimated bandwidths against a capacity of the service provider network; and

determining whether to optimize the data flow based on the congestion level of the service provider network.

2. The method of claim 1, wherein determining whether to monitor the data flow comprises determining whether the media content in the data flow indicates transfer of video and/or images larger than a predetermined size.

3. The method of claim 1, wherein the statistical information collected for the data flow includes at least one of throughput, latency, size of the media content to be delivered, and duration of the flow.

4. The method of claim 1, wherein the flow record comprises at least one of:

user device source IP address, subscriber identifier, estimated bandwidth, cookie hashes, flow identifier, and transmitted and received bytes, and wherein the flow record is updated during a lifespan of the data flow.

5. The method of claim 1, wherein estimating the bandwidth to be consumed on the service provider network by the data flow comprises:

receiving an update on transmitted or received bytes in the flow record; and

calculating the estimated bandwidth by dividing a difference between the transmitted or received bytes since a last update by an interval of time since the last update.

6. The method of claim 1, wherein determining whether to optimize the data flow based on the congestion level of the service provider network comprises determining to optimize the data flow if the congestion level of the service provider network exceeds the capacity of the service provider network.

7. The method of claim 1, wherein determining whether to optimize the data flow based on the congestion level of the service provider network comprises determining not to optimize the data flow if the congestion level of the service provider network does not exceed the capacity of the service provider network.

8. The method of claim 1, wherein the determination of whether to optimize the data flow is further based on a size of the media content to be delivered in the data flow.

9. The method of claim 1, further comprising redirecting, in response to determining to optimize the data flow, the request for the media content from the requesting user device to a video optimizer, the video optimizer retrieving the media content from the origin server, optimizing the media content for the user device, and transmitting the optimized media content to the user device.

10. The method of claim 1, wherein aggregating flow records corresponding to the subscriber in the database of the service provider network is based on matching at least one of:

user device source IP address and MAC address, TCP port number, and hashes of a cookie associated with the data flows.

11. A non-transitory computer-readable storage medium storing executable computer program instructions for selectively monitoring traffic in a service provider network, the computer program instructions comprising instructions that cause a processor to:

receive a notice for a beginning of a network data flow between an origin server and a user device, the data flow for delivering media content from the origin server to the user device in response to a request for the media content from the user device;

determine whether to monitor the data flow between the origin server and the user device;

collect, in responsive to a determination to monitor the data flow, statistic information corresponding to the data flow;

store the statistic information collected for the data flow to a flow record in a database;

map the flow record in the database to a subscriber of the service provider network based on an analysis of the statistic information collected for the data flow;

estimate bandwidth to be consumed on the service provider network by the data flow between the user device and the origin server based on the statistic information collected for the data flow;

sum the estimated bandwidth of the data flow and estimated bandwidths of at least one other monitored data flow in the service provider network;

determine a congestion level of the service provider network based on a comparison of the sum of the estimated bandwidths against a capacity of the service provider network; and

determine whether to optimize the data flow based on the congestion level of the service provider network.

12. The non-transitory computer-readable storage medium of claim 11, wherein the instructions that cause the processor to determine whether to monitor the data flow comprises instructions that cause the processor to determine whether the media content in the data flow indicates transfer of video and/or images larger than a predetermined size.

13. The non-transitory computer-readable storage medium of claim 11, wherein the statistical information collected for the data flow includes at least one of throughput, latency, size of the media content to be delivered, and duration of the flow.

14. The non-transitory computer-readable storage medium of claim 11, wherein the flow record comprises at least one of:

user device source IP address, subscriber identifier, estimated bandwidth, cookie hashes, flow identifier, and transmitted and received bytes, and wherein the flow record is updated during a lifespan of the data flow.

15. The non-transitory computer-readable storage medium of claim 11, wherein the instructions that cause the processor to estimate the bandwidth to be consumed on the service provider network by the data flow comprises instructions that cause the processor to:

receive an update on transmitted or received bytes in the flow record; and

calculate the estimated bandwidth by dividing a difference between the transmitted or received bytes since a last update by an interval of time since the last update.

16. The non-transitory computer-readable storage medium of claim 11, wherein the instructions that cause the processor to determine whether to optimize the data flow based on the congestion level of the service provider network comprises instructions that cause the processor to determine to optimize the data flow if the congestion level of the service provider network exceeds the capacity of the service provider network.

17. The non-transitory computer-readable storage medium of claim 11, wherein the instructions that cause the processor to determine whether to optimize the data flow based on the congestion level of the service provider network comprises instructions that cause the processor to determine not to optimize the data flow if the congestion level of the service provider network does not exceed the capacity of the service provider network.

18. The non-transitory computer-readable storage medium of claim 11, the computer program instructions further comprising instructions that cause the processor to redirect, in response to determining to optimize the data flow, the request for the media content from the requesting user device to a video optimizer, the video optimizer retrieving the media content from the origin server, optimizing the media content for the user device, and transmitting the optimized media content to the user device.

19. The non-transitory computer-readable storage medium of claim 11, wherein the instructions that cause the processor to aggregate flow records corresponding to the subscriber in the database of the service provider network comprises instructions to match to at least one of:

user device source IP address and MAC address, TCP port number, and hashes of a cookie associated with the data flows.

20. A system for selectively monitoring traffic in a service provider network, comprising:

a processor; and

a non-transitory computer-readable storage medium storing executable computer program instructions that when executed by the processor;

receive a notice for a beginning of a network data flow between an origin server and a user device, the data flow for delivering media content from the origin server to the user device in response to a request for the media content from the user device;

determine whether to monitor the data flow between the origin server and the user device;

collect statistic information corresponding to the data flow in response to a determination to monitor the data flow;

store the statistic information collected for the data flow to a flow record in a database;

map the flow record in the database to a subscriber of the service provider network based on an analysis of the statistic information collected for the data flow; and

estimate bandwidth to be consumed on the server provider network by the data flow between the user device and the origin server based on the statistic information collected for the data flow;

sum the estimated bandwidth of the data flow and estimated bandwidths of at least one other monitored data flow in the service provider network;

determine a congestion level of the service provider network based on a comparison of the sum of the estimated bandwidths against a capacity of the service provider network; and

determine whether to optimize the data flow based on the congestion level of the service provider network.

21. The system of claim 20, wherein the system collects statistic information of the data flow as an inline network element that receives and forwards the data flow.

22. The system of claim 20, wherein the system receives the data flow from an inline network element in response to an initial determination to monitor the data flow based on initial statistic information collected for the data flow, the inline network element forwarding the data flow for processing in response to the initial determination to monitor the data flow.

23. The system of claim 22, wherein determining whether to monitor the data flow between the origin server and the user device further comprises, in response to a determination to not monitor the data flow, transmitting an instruction causing the inline network element to cease forwarding of the data flow through the system.

24. The system of claim 20, wherein estimating the bandwidth to be consumed on the service provider network by the data flow comprises:

receiving an update on transmitted or received bytes in the flow record; and

calculating the estimated bandwidth by dividing a difference between the transmitted or received bytes since a last update by an interval of time since the last update.

25. The system of claim 20, wherein determining whether to optimize the data flow based on the congestion level of the service provider network comprises determining to optimize the data flow if the congestion level of the service provider network exceeds the capacity of the service provider network.

26. The system of claim 20, wherein determining whether to optimize the data flow based on the congestion level of the service provider network comprises determining not to optimize the data flow if the congestion level of the service provider network does not exceed the capacity of the service provider network.

27. The system of claim 20, the computer program instructions further comprising instructions for redirecting, in response to determining to optimize the data flow, the request for the media content from the requesting user device to a video optimizer, the video optimizer retrieving the media content from the origin server, optimizing the media content for the user device, and transmitting the optimized media content to the user device.