System and method for managing computer networks

• US 20040103211A1
• Filed: 11/21/2002
• Published: 05/27/2004
• Est. Priority Date: 11/21/2002
• Status: Active Grant

• Current Assignee: Arbor Networks Incorporated (NetScout Systems Incorporated)
• Original Assignee: Arbor Networks Incorporated (NetScout Systems Incorporated)
• Inventors: Malan, Gerald R., Fleis, Lawrence Benjamin, Dysart, Aidan Christopher, Song, Douglas J., Jackson, Eric S.
• Granted Patent: US 7,359,930 B2
• Time in Patent Office: Days
• US Class Current: 709/244
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/0894 Policy-based network configuration management
• CPC Class Codes: H04L 41/142 using statistical or mathematical methods
• CPC Class Codes: H04L 43/026 using flow identification
• CPC Class Codes: Y02D 30/50 in wire-line communication networks, e.g. low power modes or reduced link rate

Abstract

A system and method are provided for monitoring traffic in an enterprise network. Similar hosts may be grouped using flow information. Network policy may then be created at the group level based on the signatures of the hosts and groups of hosts in the enterprise. Hosts may be arranged in hierarchical clusters. Some of these clusters may be selected as groups based on a desired degree of similarity between hosts in a group. The similarity between hosts may be determined based on similarity of network behavior of the hosts.

Claims

1. A method for monitoring traffic in a computer network comprising acts of:

receiving flow information related to network traffic a plurality of hosts in the computer network; and

determining similarity between the plurality of hosts in the computer network based on the flow information.

2. The method of claim 1, further comprising an act of:

creating network policy for the plurality of hosts in the network based on the flow information.

3. The method of claim 2, wherein the act of creating a network policy for the plurality of hosts is performed automatically.

4. The method of claim 1, further comprising an act of:

arranging the plurality of hosts into a plurality of groups based on the similarity between the plurality of hosts.

5. The method of claim 4, wherein the act of arranging the plurality of hosts into a plurality of groups is performed automatically.

6. The method of claim 4, further comprising an act of:

creating network policy for the plurality of groups of hosts based on the flow information.

7. The method of claim 6, wherein the act of creating network policy for the plurality of groups is performed automatically.

8. The method of claim 1, wherein the act of determining similarity further comprises modifying the flow information according to a predetermined set of rules.

9. The method of claim 4, further comprising acts of:

arranging the hosts into hierarchical clusters; and

selecting at least some of the hierarchical clusters as groups.

10. The method of claim 2, wherein the act of creating network policy for the group of hosts further comprises an act of determining a baseline of network traffic for the plurality of hosts based on the flow information relating to the plurality of hosts in the group.

11. The method of claim 10, further comprising an act of manually altering the baseline.

12. The method of claim 10, further comprising an act of monitoring network traffic relating to the plurality of hosts for at least one deviation from the baseline.

13. The method of claim 12, further comprising an act of providing at least one alert if a deviation from the baseline is detected.

14. A system for monitoring traffic in a computer network comprising:

at least one flow capture device configured to create flow information based on network traffic of a plurality of hosts in the computer network; and

a flow controller configured to determine similarity between the plurality of hosts in the computer network based on the flow information.

15. The system of claim 14, wherein the at least one flow capture device comprises:

a network interface configured to receive data from the network; and

a packet analyzer and flow generation engine configured to receive network traffic from the network interface and generate flow information based on the network traffic.

16. The system of claim 15, wherein the packet analyzer and flow generation engine is implemented in hardware.

17. The system of claim 15, wherein the packet analyzer and flow generation engine is implemented in software.

18. The system of claim 15, further comprising an interface processor configured to provide an interface between a user and the packet analyzer and flow generation engine.

19. The system of claim 18, wherein the interface processor is configured to provide the user with remote access to the packet analyzer and flow generation engine.

20. The system of claim 19, wherein the remote access includes world wide web (WWW) access.

21. The system of claim 19, wherein the remote access includes simple network management protocol (SNMP) access.

22. The system of claim 19, wherein the packet analyzer and flow generation engine is configured to export the flow information to the flow controller.

23. The system of claim 15, wherein the flow controller comprises a network interface configured to receive data from the network;

and

an aggregation engine configured to collect flows received from the at least one flow capture device.

24. The system of claim 23, wherein the flow controller further comprises a database management system configured to store the flow received from the at least one flow capture device.

25. The system of claim 23, wherein the flow controller further comprises a signature generation engine configured to receive the flow information from the aggregation engine and generate signatures based on the flow information.

26. The system of claim 25, wherein the flow controller further comprises a grouping engine configured to receive the signatures from the signature generation engine and determine similarity between the plurality of hosts.

27. The system of claim 26, further comprising a policy management engine configured to create and manage network policy based on the similarity between the plurality of hosts determined by the policy management engine.

28. The system of claim 15, further comprising an interface processor configured to provide an interface between a user and the flow controller.

29. The system of claim 18, wherein the interface processor is configured to provide the user with remote access to the flow controller.

30. The system of claim 19, wherein the remote access includes world wide web (WWW) access.

31. A method of grouping a plurality of entities comprising acts of:

a) determining similarity values between pairs of the plurality of entities;

b) arranging the plurality of entities into hierarchical clusters based on the similarity values; and

c) selecting at least some of the hierarchical clusters as groups.

32. The method of claim 31, wherein the plurality of entities includes a plurality of hosts in a network and the act of determining similarity values further comprises an act of determining similarity between pairs of the plurality of hosts based on network behavior.

35. The method of claim 31, wherein the act c) further comprises an act of selecting at least some of the hierarchical clusters as groups based on a mean and a standard deviation of similarity values between the hierarchical clusters.

36. The method of claim 31, wherein the act c) further comprises an act of selecting at least some of the hierarchical clusters as groups based on a desired degree of similarity between entities in a group.

37. The method of claim 32, further comprising an act of:

d) applying a network policy to the plurality of hosts based on the groups.

33. The method of 31, wherein the act b) further comprises acts of:

b1) arranging each of the plurality of entities in a separate cluster; and

b2) merging the two most similar clusters into a single cluster.

34. The method of claim 33, further comprising an act of:

b3) repeating the act b2 until the plurality of entities are in a single cluster.

38. A system configured to group a plurality of entities comprising:

a) means for determining similarity values between pairs of the plurality of entities;

b) means for arranging the plurality of entities into hierarchical clusters based on the similarity values; and

c) means for selecting at least some of the hierarchical clusters as groups.

39. The system of claim 38, wherein the means for arranging the plurality of entities further comprises:

b1) means for arranging each of the plurality of entities in a separate cluster; and

b2) means for merging the two most similar clusters into a single cluster.

40. The system of claim 39, further comprising:

b3) means for merging the two most similar clusters until the plurality of entities are in a single cluster.

41. The system of claim 38, wherein the means for selecting further comprises means for selecting at least some of the hierarchical clusters as groups based on a mean and a standard deviation of similarity values between the hierarchical clusters.

42. A method of determining similarity between a first host and a second host based on the network behavior of the first and second hosts comprising acts of:

a) determining a first score based on the similarity between network traffic of top services using the first and host and network traffic of top services using the second host; and

b) determining a second score based on the total network traffic of the first host and the total network traffic of the second host.

43. The method of claim 42, further comprising an act of:

c) combining the first and second scores to generate a similarity value.

44. The method of claim 42, wherein the act a) further comprises an act of:

determining the first score based on the similarity between network traffic of a top four of the top services using the first host and network traffic of a top four of the top services using the second host.

45. The method of claim 43, wherein the act b) further comprises an act of:

determining the second score based on a directionality and a magnitude of the total traffic of the first and second hosts.

46. The method of claim 45, further comprising acts of:

b1) representing the first host as a first point on a Cartesian plane based on the directionality and magnitude of the total traffic of the first host;

b2) representing the second host as a second point on a Cartesian plane based on the directionality and magnitude of the total traffic of the second host; and

b3) computing a Euclidean distance between the first and second points.

47. The method of claim 46, further comprising an act of:

combining the Euclidean distance and the first score to generate the similarity value.

48. A system configured to determine similarity between a first host and a second host based on network behavior of the first and second hosts comprising:

a) means for determining a first score based on the similarity between network traffic of top services using the first and host and network traffic of top services using the second host; and

b) means for determining a second score based on the total network traffic of the first host and the total network traffic of the second host.

49. The system of claim 48, further comprising:

c) means for combining the first and second scores to generate a similarity value.

50. The system of claim 49, wherein the means for determining a first score further comprises:

means for determining the first score based on the similarity between network traffic of a top four of the top services using the first host and network traffic of a top four of the top services using the second host.

51. The system of claim 49, wherein the means for the determining the second score further comprises:

means for determining the second score based on a directionality and a magnitude of the total traffic of the first and second hosts.

52. The system of claim 51, further comprising:

b1) means for representing the first host as a first point on a Cartesian plane based on the directionality and magnitude of the total traffic of the first host;

b2) means for representing the second host as a second point on a Cartesian plane based on the directionality and magnitude of the total traffic of the second host; and

b3) means for computing a Euclidean distance between the first and second points.

53. The system of claim 52, further comprising:

means for combining the Euclidean distance and the first score to generate the similarity value.

54. In a computer network having a plurality of hosts, a plurality of flow capture devices configured to capture a plurality of flows, and at least one flow controller for managing flow information received from the plurality of flow capture devices, a method comprising acts of:

a) identifying a flow from a first source;

b) identifying a flow from a second source;

c) determining if the flow from the first source is the same as the flow from the second source; and

d) discarding the flow from the first source if it is the same as the flow from the second source.

55. The method of claim 54, wherein the act c) further comprises acts of:

computing a hash based on a first packet in each flow received by one of the plurality of flow capture devices; and

transmitting each flow and its corresponding hash to the flow controller.

56. The method of claim 55, further comprises acts of:

maintaining a list of received hashes at the flow controller; and

comparing each hash received by the flow controller to the list of received hashes.

57. The method of claim 56, wherein the act of computing the hash further comprises an act of computing the hash based on a packet header of the first packet.

58. The method of claim 57, wherein the act of maintaining a list of received hashes further comprises an act of maintaining a least-recently-used list of received hashes.

59. A computer-readable medium having computer-readable signals stored thereon that define instructions that, as a result of being executed by a computer, instruct the computer to perform a method of monitoring traffic in a computer network comprising acts of:

receiving flow information related to network traffic a plurality of hosts in the computer network; and

determining similarity between the plurality of hosts in the computer network based on the flow information.

60. The computer-readable medium of claim 59, wherein the method further comprises an act of:

creating network policy for the plurality of hosts in the network based on the flow information.

61. The computer-readable medium of claim 60, wherein the act of creating a network policy for the plurality of hosts is performed automatically.

62. The computer-readable medium of claim 59, wherein the method further comprises an act of:

arranging the plurality of hosts into a plurality of groups based on the similarity between the plurality of hosts.

63. The computer-readable medium of claim 62, wherein the act of arranging the plurality of hosts into a plurality of groups is performed automatically.

64. The computer-readable medium of claim 62, wherein the method further comprises an act of:

creating network policy for the plurality of groups of hosts based on the flow information.

65. The computer-readable medium of claim 64, wherein the act of creating network policy for the plurality of groups is performed automatically.

66. The computer-readable medium of claim 59, wherein the act of determining similarity further comprises modifying the flow information according to a predetermined set of rules.

67. The computer-readable medium of claim 62, wherein the method further comprises acts of:

arranging the hosts into hierarchical clusters; and

selecting at least some of the hierarchical clusters as groups.

68. The computer-readable medium of claim 60, wherein the act of creating network policy for the group of hosts further comprises an act of determining a baseline of network traffic for the plurality of hosts based on the flow information relating to the plurality of hosts in the group.

69. The computer-readable medium of claim 68, wherein the method further comprises an act of manually altering the baseline.

70. The computer-readable medium of claim 68, wherein the method further comprises an act of monitoring network traffic relating to the plurality of hosts for at least one deviation from the baseline.

71. The computer-readable medium of claim 70, wherein the method further comprises an act of providing at least one alert if a deviation from the baseline is detected.

72. A computer-readable medium having computer-readable signals stored thereon that define instructions that, as a result of being executed by a computer, instruct the computer to perform a method of grouping a plurality of entities comprising acts of:

a) determining similarity values between pairs of the plurality of entities;

b) arranging the plurality of entities into hierarchical clusters based on the similarity values; and

c) selecting at least some of the hierarchical clusters as groups.

73. The computer-readable medium of claim 72, wherein the plurality of entities includes a plurality of hosts in a network and the act of determining similarity values further comprises an act of determining similarity between pairs of the plurality of hosts based on network behavior.

76. The computer-readable medium of claim 72, wherein the act c) further comprises an act of selecting at least some of the hierarchical clusters as groups based on a mean and a standard deviation of similarity values between the hierarchical clusters.

77. The computer-readable medium of claim 72, wherein the act c) further comprises an act of selecting at least some of the hierarchical clusters as groups based on a desired degree of similarity between entities in a group.

78. The computer-readable medium of claim 72, wherein the method further comprises an act of:

d) applying a network policy to the plurality of hosts based on the groups.

74. The computer-readable medium of 72, wherein the act b) further comprises acts of:

b1) arranging each of the plurality of entities in a separate cluster; and

b2) merging the two most similar clusters into a single cluster.

75. The computer-readable medium of claim 74, wherein the method further comprises an act of:

b3) repeating the act b2 until the plurality of entities are in a single cluster.

79. A computer-readable medium having computer-readable signals stored thereon that define instructions that, as a result of being executed by a computer, instruct the computer to perform a method of determining similarity between a first host and a second host based on the network behavior of the first and second hosts comprising acts of:

a) determining a first score based on the similarity between network traffic of top services using the first and host and network traffic of top services using the second host; and

b) determining a second score based on the total network traffic of the first host and the total network traffic of the second host.

80. The computer-readable medium of claim 79, wherein the method further comprises an act of:

c) combining the first and second scores to generate a similarity value.

81. The computer-readable medium of claim 79, wherein the act a) further comprises an act of:

determining the first score based on the similarity between network traffic of a top four of the top services using the first host and network traffic of a top four of the top services using the second host.

82. The computer-readable medium of claim 80, wherein the act b) further comprises an act of:

determining the second score based on a directionality and a magnitude of the total traffic of the first and second hosts.

83. The computer-readable medium of claim 82, wherein the method further comprises acts of:

b1) representing the first host as a first point on a Cartesian plane based on the directionality and magnitude of the total traffic of the first host;

b2) representing the second host as a second point on a Cartesian plane based on the directionality and magnitude of the total traffic of the second host; and

b3) computing a Euclidean distance between the first and second points.

84. The computer-readable medium of claim 83, wherein the method further comprises an act of:

combining the Euclidean distance and the first score to generate the similarity value.

85. In a computer network having a plurality of hosts, a plurality of flow capture devices configured to capture a plurality of flows, and at least one flow controller for managing flow information received from the plurality of flow capture devices, a computer-readable medium having computer-readable signals stored thereon that define instructions that, as a result of being executed by a computer, instruct the computer to perform a method comprising acts of:

a) identifying a flow from a first source;

b) identifying a flow from a second source;

c) determining if the flow from the first source is the same as the flow from the second source; and

d) discarding the flow from the first source if it is the same as the flow from the second source.

86. The computer-readable medium of claim 85, wherein the act c) further comprises acts of:

computing a hash based on a first packet in each flow received by one of the plurality of flow capture devices; and

transmitting each flow and its corresponding hash to the flow controller.

87. The computer-readable medium of claim 86, wherein the method further comprises acts of:

maintaining a list of received hashes at the flow controller; and

comparing each hash received by the flow controller to the list of received hashes.

88. The computer-readable medium of claim 87, wherein the act of computing the hash further comprises an act of computing the hash based on a packet header of the first packet.

89. The computer-readable medium of claim 88, wherein the act of maintaining a list of received hashes further comprises an act of maintaining a least-recently-used list of received hashes.