Measurement-Based Validation of a Simple Model for Panoramic Profiling of Subnet-Level Network Data Traffic

• US 20100034102A1
• Filed: 08/05/2008
• Published: 02/11/2010
• Est. Priority Date: 08/05/2008
• Status: Abandoned Application

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Wang, Jia, Ge, Zihui, Jiang, Hongbo, Jin, Shudong
• Application Number:
  US12/186,113
• Publication Number:
  US 20100034102A1
• Time in Patent Office: Days
• US Class Current: 370/252
• CPC Class Codes: H04L 41/0213 Standardised network management protocols, e.g. simple network management protocol [SNMP]
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 43/0876 Network utilisation, e.g. volume of load or congestion level

Abstract

A system and method for profiling subnet-level aggregate network data traffic is disclosed. The system allows a user to define a collection of features that combined characterize the subnet-level aggregate traffic behavior. Preferably, the features include daily traffic volume, time-of-day behavior, spatial traffic distribution, traffic balance in flow direction, and traffic distribution in type of application. The system then applies machine learning techniques to classify the subnets into a number of clusters on each of the features, by assigning a membership probability vector to each network thus allowing panoramic traffic profiles to be created for each network on all features combined. These membership probability vectors may optionally be used to detect network anomalies, or to predict future network traffic.

Claims

1. A method of profiling network traffic comprising:

determining a probabilistic classification of a plurality of subnets into a plurality of clusters based on at least one network traffic feature; and

deriving a network profile using said probabilistic classification and traffic measurement data associated with at least one of said plurality of subnets.

2. The method of claim 1, wherein said at least one network traffic feature includes at least one of daily aggregate traffic volume, traffic distribution in time, traffic distribution in space, traffic distribution in application, flow size distribution, and traffic balance in flow direction.

3. The method of claim 1, wherein determining a probabilistic classification comprises using at least one of a Bayes classifier or a K-means clustering algorithm.

4. The method of claim 1, wherein the number of clusters is selected probabilistically.

5. The method of claim 4, wherein probabilistically selecting the number of cluster comprises using at least one of an Akaike information criterion (AIC) algorithm or a Bayesian information criterion (BIC) algorithm.

6. The method of claim 1, wherein said network profile comprises information associated with anomalous network traffic.

7. The method of claim 1, wherein deriving a network profile comprises:

determining a target cluster membership probability vector for at least one subnet of said plurality of subnets based on at least one target network traffic feature;

calculating a predicted cluster membership probability vector for said subnet based on a set of cluster membership probability vectors, said set of cluster membership probability vectors comprising at least one cluster membership probability vector determined for said subnet based on said at least one target network traffic feature; and

comparing the difference between said target cluster membership probability vector and said predicted cluster membership probability vector to a threshold.

8. The method of claim 7, wherein said threshold is a function of the variance of said set of cluster membership probability vectors.

9. The method of claim 1, wherein said network profile comprises at least one network traffic feature value and a prediction of network traffic exhibiting said at least one network traffic feature value.

10. A system for profiling network traffic comprising a computing device, the computing device being configured to probabilistically classify a plurality of subnets into a plurality of clusters based on at least one network traffic feature, the computing device being configured to derive a network profile in response to receiving traffic measurement data associated with at least one of said subnets.

11. The system of claim 10, wherein said at least one network traffic feature includes at least one of daily aggregate traffic volume, traffic distribution in time, traffic distribution in space, traffic distribution in application, flow size distribution, and traffic balance in flow direction.

12. The system of claim 10, wherein the computing device uses at least one of a Bayes classifier or a K-means clustering algorithm to probabilistically classify.

13. The system of claim 10, wherein the computing device selects the number of clusters probabilistically.

14. The system of claim 13, wherein the computing device uses at least one of an Akaike information criterion (AIC) algorithm or a Bayesian information criterion (BIC) algorithm to select the number of clusters.

15. The system of claim 10, wherein said network profile comprises information associated with anomalous network traffic.

16. The system of claim 15, wherein said computing device determines a target cluster membership probability vector for at least one subnet of said plurality of subnets based on at least one target network traffic feature, said computing device calculating a predicted cluster membership probability vector for said subnet based on a set of cluster membership probability vectors, said set of cluster membership probability vectors including at least one cluster membership probability vector determined for said subnet based on said at least one target network traffic feature, said computing device comparing the difference between said target cluster membership probability vector and said predicted cluster membership probability vector to a threshold.

17. The system of claim 16, wherein said threshold is a function of the variance of said set of cluster membership probability vectors.

18. The system of claim 10, wherein said network profile comprises at least one network traffic feature value and a prediction of network traffic exhibiting said at least one network traffic feature value.

19. A computer readable medium comprising instructions executable by a computing device that, when applied to the computing device, cause the device to:

determine a probabilistic classification of a plurality of subnets into a plurality of clusters based on at least one network traffic feature; and

derive a network profile in using said probabilistic classification and traffic measurement data associated with at least one of said plurality of subnets.

20. The computer readable medium of claim 19, wherein said at least one network traffic feature includes at least one of daily aggregate traffic volume, traffic distribution in time, traffic distribution in space, traffic distribution in application, flow size distribution, and traffic balance in flow direction.