Tracking the dynamics of application-centric clusters in a virtualized datacenter

• US 10,742,673 B2
• Filed: 12/08/2017
• Issued: 08/11/2020
• Est. Priority Date: 12/08/2017
• Status: Active Grant

• Current Assignee: Nicira Incorporated (Broadcom, Inc.)
• Original Assignee: Nicira Incorporated (Broadcom, Inc.)
• Inventors: Tiagi, Alok, Jain, Jayant, Sengupta, Anirban, Manuguri, Subrahmanyam, Saran, Vedant
• Primary Examiner(s): Truong, Thanhnga B
• Application Number:
  US15/835,865
• Publication Number:
  US 20190182276A1
• Time in Patent Office: 977 Days
• CPC Class Codes: H04L 41/0894 Policy-based network configuration management
• CPC Class Codes: H04L 41/145 involving simulating, designing, planning or modelling of a network
• CPC Class Codes: H04L 43/062 related to network traffic
• CPC Class Codes: H04L 43/067 using time frame reporting
• CPC Class Codes: H04L 43/106 using time related information in packets, e.g. by adding timestamps
• CPC Class Codes: H04L 47/20 Traffic policing
• CPC Class Codes: H04L 47/2441 relying on flow classification, e.g. using integrated services [IntServ]
• CPC Class Codes: H04L 63/0227 Filtering policies mail message filtering H04L51/212
• CPC Class Codes: H04L 63/0272 Virtual private networks
• CPC Class Codes: H04L 63/101 Access control lists [ACL]
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/20 for managing network security; network security policies in general filtering policies H04L63/0227

Abstract

For a managed network including multiple nodes providing multiple services and executing multiple applications some embodiments provide a method for generating groupings of network addresses associated with different applications or services. The method analyzes network traffic patterns using a probabilistic topic modeling algorithm to generate the groupings of network addresses. In some embodiments, data is collected and analyzed periodically. A network administrator defines the granularity of the time stamps in some embodiments to monitor changes in network traffic patterns over time for each network address or node and/or for the network as a whole. For each network address or node, a probability distribution over the topics at a given time is stored in some embodiments. The stored distributions are then used to determine a divergence over time of the application or service provided by the network address or node. Additionally, the stored distributions can be used to detect anomalous behavior.

Claims

1. A method for detecting anomalies in traffic patterns related to a first network address associated with an element in a computer network, the method comprising:

during each of a plurality of time periods, analyzing network traffic characteristics for traffic relating to the first network address to create a probabilistic distribution for flows that start or end at the first network address;

comparing the probabilistic distributions of the flows that are created for the plurality of time periods to identify at least one particular time period that has a particular probabilistic distribution of flows that diverges from other probabilistic distributions created for other time periods; and

identifying the particular time period for a network administrator to analyze in order to identify a source for an anomalous event that has caused the particular probabilistic distribution to diverge.

2. The method of claim 1, wherein

analyzing network traffic characteristics comprises using a probabilistic topic modeling algorithm to generate a plurality of probabilistic distribution of topics for a plurality of documents;

said modeling comprising defining sets of flows as topics and sets of network addresses as documents;

each probabilistic distribution of topics produced by the modeling representing a distribution of sets of flows for the network address associated with the probabilistic distribution.

3. The method of claim 1, wherein a plurality of flows starts or terminates on compute nodes that are part of one or more compute clusters, the method further comprising, before identifying the particular time period,

using data regarding membership of one or more compute clusters to assess whether the identified divergence of the particular probabilistic distribution corresponds to a change in the membership of a cluster;

and

identifying the particular time period when the identified divergence does not correspond to a change in the membership of a cluster.

4. The method of claim 3, wherein the compute nodes comprise virtual machines or containers.

5. The method of claim 1,

wherein a plurality of flows starts or terminates on compute nodes that are part of one or more compute clusters,

wherein analyzing network traffic characteristics comprises:

identifying a plurality of clusters of compute nodes;

providing recommendations for specifying microsegmentation service rules for each of the clusters; and

for each accepted recommendation, assigning the microsegmentation service rules associated with the recommendation to the cluster for which the recommendation is provided.

6. The method of claim 1 further comprising:

during each of the plurality of time periods, analyzing network traffic characteristics for traffic relating to a second network address to create a probabilistic distribution for flows that start or end at the second network address;

comparing the probabilistic distributions of the flows that are created for the plurality of time periods related to the second network address to identify at least a second particular time period that has a second particular probabilistic distribution of flows related to the second network address that diverges from other probabilistic distributions related to the second network address created for other time periods; and

identifying the second particular time period for a network administrator to analyze in order to identify a source for an anomalous event that has caused the second particular probabilistic distribution to diverge.

7. The method of claim 1, wherein a plurality of flows are associated with applications that are part of one or more application clusters, the method further comprising, before identifying the particular time period,

using data regarding membership of one or more application clusters to assess whether the identified divergence of the particular probabilistic distribution corresponds to a change in the membership of a cluster;

and

identifying the particular time period when the identified divergence does not correspond to a change in the membership of a cluster.

8. The method of claim 1, wherein analyzing network traffic comprises analyzing a plurality of stored flow records created during the plurality of time periods.

9. The method of claim 1, wherein at least two time periods have two different durations.

10. The method of claim 1, wherein each time period has a duration that is set by the network administrator.

11. The method of claim 5, wherein a microsegment recommendation made for one time period after a first time period is based on data collected during the particular time period and a set of previous time periods.

12. A non-transitory machine readable medium storing a program for execution by at least one processing unit, the program for detecting anomalies in traffic patterns related to a first network address associated with an element in a computer network, the program comprising sets of instructions for:

during each of a plurality of time periods, analyzing network traffic characteristics for traffic relating to the first network address to create a probabilistic distribution for flows that start or end at the first network address;

comparing the probabilistic distributions of the flows that are created for the plurality of time periods to identify at least one particular time period that has a particular probabilistic distribution of flows that diverges from other probabilistic distributions created for other time periods; and

identifying the particular time period for a network administrator to analyze in order to identify a source for an anomalous event that has caused the particular probabilistic distribution to diverge.

13. The non-transitory machine readable medium of claim 12, wherein

analyzing network traffic characteristics comprises using a probabilistic topic modeling algorithm to generate a plurality of probabilistic distribution of topics for a plurality of documents;

said modeling comprising defining sets of flows as topics and sets of network addresses as documents;

each probabilistic distribution of topics produced by the modeling representing a distribution of sets of flows for the network address associated with the probabilistic distribution.

14. The non-transitory machine readable medium of claim 12, wherein a plurality of flows starts or terminates on compute nodes that are part of one or more compute clusters, the method further comprising, before identifying the particular time period,

using data regarding membership of one or more compute clusters to assess whether the identified divergence of the particular probabilistic distribution corresponds to a change in the membership of a cluster;

and

identifying the particular time period when the identified divergence does not correspond to a change in the membership of a cluster.

15. The non-transitory machine readable medium of claim 14, wherein the compute nodes comprise virtual machines or containers.

16. The non-transitory machine readable medium of claim 12,

wherein a plurality of flows starts or terminates on compute nodes that are part of one or more compute clusters,

wherein analyzing network traffic characteristics comprises:

identifying a plurality of clusters of compute nodes;

providing recommendations for specifying microsegmentation service rules for each of the clusters; and

for each accepted recommendation, assigning the microsegmentation service rules associated with the recommendation to the cluster for which the recommendation is provided.

17. The non-transitory machine readable medium of claim 12 further comprising:

during each of the plurality of time periods, analyzing network traffic characteristics for traffic relating to a second network address to create a probabilistic distribution for flows that start or end at the second network address;

comparing the probabilistic distributions of the flows that are created for the plurality of time periods related to the second network address to identify at least a second particular time period that has a second particular probabilistic distribution of flows related to the second network address that diverges from other probabilistic distributions related to the second network address created for other time periods; and

identifying the second particular time period for a network administrator to analyze in order to identify a source for an anomalous event that has caused the second particular probabilistic distribution to diverge.

18. The non-transitory machine readable medium of claim 12, wherein a plurality of flows are associated with applications that are part of one or more application clusters, the method further comprising, before identifying the particular time period,

using data regarding membership of one or more application clusters to assess whether the identified divergence of the particular probabilistic distribution corresponds to a change in the membership of a cluster;

and

identifying the particular time period when the identified divergence does not correspond to a change in the membership of a cluster.

19. The non-transitory machine readable medium of claim 12, wherein analyzing network traffic comprises analyzing a plurality of stored flow records created during the plurality of time periods.

20. The non-transitory machine readable medium of claim 12, wherein at least two time periods have two different durations.