CLUSTER DISCOVERY VIA MULTI-DOMAIN FUSION FOR APPLICATION DEPENDENCY MAPPING

• US 20160359680A1
• Filed: 05/03/2016
• Published: 12/08/2016
• Est. Priority Date: 06/05/2015
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Parandehgheibi, Ali, Madani, Omid, Jeyakumar, Vimalkumar, Scheib, Ellen Christine, Yadav, Navindra, Alizadeh Attar, Mohammadreza
• Granted Patent: US 10,728,119 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/137 Hash-based content-based indexing of textual data G06F16/31
• CPC Class Codes: G06F 16/162 Delete operations erasing in storage systems G06F3/0652
• CPC Class Codes: G06F 16/17 Details of further file system functions
• CPC Class Codes: G06F 16/173 Customisation support for file systems, e.g. localisation, multi-language support, personalisation
• CPC Class Codes: G06F 16/174 Redundancy elimination performed by the file system management of the data involved in backup or backup restore using de-duplication of the data G06F11/14
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/2322 using timestamps
• CPC Class Codes: G06F 16/235 Update request formulation
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/288 Entity relationship models
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2009/45587 Isolation or security of virtual machine instances
• CPC Class Codes: G06F 2009/45591 Monitoring or debugging support
• CPC Class Codes: G06F 2009/45595 : Network integration; Enabling network access in virtual machine instances
• CPC Class Codes: G06F 21/53 : by executing in a restricted environment, e.g. sandbox or secure virtual machine
• CPC Class Codes: G06F 21/552 : involving long-term monitoring or reporting
• CPC Class Codes: G06F 21/556 : involving covert channels, i.e. data leakage between processes inhibiting the analysis of circuitry or operation with measures against power attack G06F21/755
• CPC Class Codes: G06F 21/566 : Dynamic detection, i.e. detection performed at run-time, e.g. emulation, suspicious activities
• CPC Class Codes: G06F 2221/033 : Test or assess software
• CPC Class Codes: G06F 2221/2101 : Auditing as a secondary aspect
• CPC Class Codes: G06F 2221/2105 : Dual mode as a secondary aspect
• CPC Class Codes: G06F 2221/2111 : Location-sensitive, e.g. geographical location, GPS
• CPC Class Codes: G06F 2221/2115 : Third party
• CPC Class Codes: G06F 2221/2145 : Inheriting rights or properties, e.g., propagation of permissions or restrictions within a hierarchy
• CPC Class Codes: G06F 3/0482 : Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 : Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 : Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 9/45558 : Hypervisor-specific management and integration aspects
• CPC Class Codes: G06N 20/00 : Machine learning
• CPC Class Codes: G06N 99/00 : Subject matter not provided for in other groups of this subclass
• CPC Class Codes: G06T 11/206 : Drawing of charts or graphs
• CPC Class Codes: H04J 3/0661 : using timestamps
• CPC Class Codes: H04J 3/14 : Monitoring arrangements for SDH/SONET rings H04J3/085
• CPC Class Codes: H04L 1/242 : by comparing a transmitted test signal with a locally generated replica
• CPC Class Codes: H04L 41/046 : comprising network management agents or mobile agents therefor
• CPC Class Codes: H04L 41/0668 : by dynamic selection of recovery network elements, e.g. replacement by the most appropriate element after failure
• CPC Class Codes: H04L 41/0803 : Configuration setting
• CPC Class Codes: H04L 41/0806 : for initial configuration or provisioning, e.g. plug-and-play
• CPC Class Codes: H04L 41/0816 : the condition being an adaptation, e.g. in response to network events
• CPC Class Codes: H04L 41/0893 : Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/0894 : Policy-based network configuration management
• CPC Class Codes: H04L 41/12 : Discovery or management of network topologies
• CPC Class Codes: H04L 41/16 : using machine learning or artificial intelligence
• CPC Class Codes: H04L 41/22 : comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 41/40 : using virtualisation of network functions or resources, e.g. SDN or NFV entities
• CPC Class Codes: H04L 43/02 : Capturing of monitoring data
• CPC Class Codes: H04L 43/026 : using flow identification
• CPC Class Codes: H04L 43/04 : Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 43/045 : for graphical visualisation of monitoring data
• CPC Class Codes: H04L 43/062 : related to network traffic
• CPC Class Codes: H04L 43/08 : Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
• CPC Class Codes: H04L 43/0805 : by checking availability
• CPC Class Codes: H04L 43/0811 : by checking connectivity
• CPC Class Codes: H04L 43/0829 : Packet loss
• CPC Class Codes: H04L 43/0841 : Round trip packet loss
• CPC Class Codes: H04L 43/0858 : One way delays
• CPC Class Codes: H04L 43/0864 : Round trip delays
• CPC Class Codes: H04L 43/0876 : Network utilisation, e.g. volume of load or congestion level
• CPC Class Codes: H04L 43/0882 : Utilisation of link capacity
• CPC Class Codes: H04L 43/0888 : Throughput
• CPC Class Codes: H04L 43/10 : Active monitoring, e.g. heartbeat, ping or trace-route
• CPC Class Codes: H04L 43/106 : using time related information in packets, e.g. by adding timestamps
• CPC Class Codes: H04L 43/12 : Network monitoring probes
• CPC Class Codes: H04L 43/16 : Threshold monitoring
• CPC Class Codes: H04L 43/20 : the monitoring system or the monitored elements being virtualised, abstracted or software-defined entities, e.g. SDN or NFV
• CPC Class Codes: H04L 45/306 : Route determination based on the nature of the carried application
• CPC Class Codes: H04L 45/38 : Flow based routing
• CPC Class Codes: H04L 45/46 : Cluster building
• CPC Class Codes: H04L 45/507 : Label distribution
• CPC Class Codes: H04L 45/66 : Layer 2 routing, e.g. in Ethernet based MAN's
• CPC Class Codes: H04L 45/74 : Address processing for routing
• CPC Class Codes: H04L 47/11 : Identifying congestion
• CPC Class Codes: H04L 47/20 : Traffic policing
• CPC Class Codes: H04L 47/2441 : relying on flow classification, e.g. using integrated services [IntServ]
• CPC Class Codes: H04L 47/2483 : involving identification of individual flows
• CPC Class Codes: H04L 47/28 : in relation to timing considerations
• CPC Class Codes: H04L 47/31 : by tagging of packets, e.g. using discard eligibility [DE] bits
• CPC Class Codes: H04L 47/32 : by discarding or delaying data units, e.g. packets or frames
• CPC Class Codes: H04L 61/5007 : Internet protocol [IP] addresses
• CPC Class Codes: H04L 63/0227 : Filtering policies mail message filtering H04L51/212
• CPC Class Codes: H04L 63/0263 : Rule management
• CPC Class Codes: H04L 63/06 : for supporting key management in a packet data network cryptographic mechanisms or cryptographic arrangements for key management H04L9/08
• CPC Class Codes: H04L 63/0876 : based on the identity of the terminal or configuration, e.g. MAC address, hardware or software configuration or device fingerprint
• CPC Class Codes: H04L 63/1408 : by monitoring network traffic monitoring network traffic per se H04L43/00
• CPC Class Codes: H04L 63/1416 : Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 : Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/1433 : Vulnerability analysis
• CPC Class Codes: H04L 63/1441 : Countermeasures against malicious traffic countermeasures against attacks on cryptographic mechanisms H04L9/002
• CPC Class Codes: H04L 63/145 : the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms
• CPC Class Codes: H04L 63/1458 : Denial of Service
• CPC Class Codes: H04L 63/1466 : Active attacks involving interception, injection, modification, spoofing of data unit addresses, e.g. hijacking, packet injection or TCP sequence number attacks
• CPC Class Codes: H04L 63/16 : Implementing security features at a particular protocol layer
• CPC Class Codes: H04L 63/20 : for managing network security; network security policies in general filtering policies H04L63/0227
• CPC Class Codes: H04L 67/01 : Protocols
• CPC Class Codes: H04L 67/10 : in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1001 : for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/12 : specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04L 67/51 : Discovery or management thereof, e.g. service location protocol [SLP] or web services
• CPC Class Codes: H04L 67/535 : Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04L 67/75 : Indicating network or usage conditions on the user display
• CPC Class Codes: H04L 69/16 : Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
• CPC Class Codes: H04L 69/22 : Parsing or analysis of headers
• CPC Class Codes: H04L 7/10 : Arrangements for initial synchronisation
• CPC Class Codes: H04L 9/0866 : involving user or device identifiers, e.g. serial number, physical or biometrical information, DNA, hand-signature or measurable physical characteristics
• CPC Class Codes: H04L 9/3239 : involving non-keyed hash functions, e.g. modification detection codes [MDCs], MD5, SHA or RIPEMD
• CPC Class Codes: H04L 9/3242 : involving keyed hash functions, e.g. message authentication codes [MACs], CBC-MAC or HMAC
• CPC Class Codes: H04W 72/54 : based on quality criteria
• CPC Class Codes: H04W 84/18 : Self-organising networks, e.g. ad-hoc networks or sensor networks

Abstract

Application dependency mapping (ADM) can be automated in a network. The network can determine whether certain nodes form a cluster of a tier of an application. The network can monitor network data and process data for traffic passing through the network using a sensor network that provides multiple perspectives for the traffic. The network can analyze the network data and process data to determine respective feature vectors for nodes. A feature vector may represent a combination of the features corresponding to the network data and the features corresponding to the process data of a node. The network can compare the similarity of the respective feature vectors and determine each node'"'"'s cluster based on similarity measures between nodes.

Claims

1. A method comprising:

capturing, using a first sensor of a first endpoint of a network, first network data and first process data corresponding to the first endpoint;

capturing, using a second sensor of a second endpoint of the network, second network data and second process data corresponding to the second endpoint;

determining first features corresponding to the first network data, second features corresponding to the first process data, third features corresponding to the second network data, and fourth features corresponding to the second process data; and

determining the first endpoint and the second endpoint form at least part of a cluster of an application executing in the network based at least in part on a similarity of the first features and the second features with respect to the third features and the fourth features.

2. The method of claim 1, further comprising:

generating a first feature vector by combining the first features and the second features;

generating a second feature vector by combining the third features and the fourth features; and

determining the similarity based at least in part by comparing the first feature vector and the second feature vector.

3. The method of claim 1, further comprising:

determining a first similarity between the first features and the third features;

determining a second similarity between the second features and the fourth features; and

determining the similarity based on the first similarity and the second similarity.

4. The method of claim 3, wherein the first similarity corresponds to a first machine learning implementation, and the second similarity corresponds to a second machine learning implementation.

5. The method of claim 3, wherein the first similarity is measured based on a first similarity function and the second similarity is measured based on a second similarity function.

6. The method of claim 3, wherein determining the similarity includes:

applying at least one of an averaging function, a binning function, or a weighted averaging function to the first similarity and the second similarity.

7. The method of claim 1, further comprising:

capturing, using the first sensor, first domain data corresponding to a third domain; and

capturing, using the second sensor, second domain data corresponding to the third domain,

wherein the first endpoint and the second endpoint are further determined to form at least part of the cluster based on the first domain data and the second domain data.

8. The method of claim 7, wherein the first domain data includes at least one of first host data, first user data, first virtualization data, or first application data corresponding to the first endpoint, and the second domain data includes at least one of second host data, second user data, second virtualization data, or second application data corresponding to the second endpoint.

9. The method of claim 1, further comprising:

capturing, using a third sensor of a first networking device connected to the first endpoint, third network data corresponding to at least a portion of the first network data.

10. The method of claim 1, further comprising:

determining a first silhouette score for the first endpoint with respect to the cluster;

determining a second silhouette score for the first endpoint with respect to a cluster; and

validating the cluster based at least in part on the first silhouette score exceeding the second silhouette score.

11. The method of claim 1, further comprising:

determining a number of nodes for an application dependency map;

determining a number of edges for the application dependency map;

determining a number of clusters for the application dependency map; and

displaying the number of nodes, the number of edges, and the number of clusters.

12. The method of claim 1, further comprising:

determining a dependency between the cluster and a third endpoint of the network; and

determining a policy between a first endpoint group corresponding to the cluster and a second endpoint group corresponding to the third endpoint based on the dependency.

13. A system comprising:

a processor; and

memory including instructions that, upon being executed by the processor, cause the system to;

receive network data and process data corresponding to a set of selected nodes;

determine a set of unselected nodes in communication with the set of selected nodes based at least in part on the network data and the process data;

determine respective features for each node of the set of selected nodes and the set of unselected nodes based at least in part on the network data and the process data;

compute respective similarities between each node of the set of selected nodes and the set of unselected nodes; and

determine a plurality of clusters for the set of selected nodes and the set of unselected nodes based on the respective similarities.

14. The system of claim 13, wherein the instructions upon being executed further cause the system to:

determine a first feature vector by combining first features of a first node corresponding to the network data and second features of the first node corresponding to the process data; and

determine a second feature vector by combining third features of a second node corresponding to the network data and fourth features of the second node corresponding to the process data,

wherein a similarity between the first node and the second node is computed based at least in part by comparing the first feature vector and the second feature vector.

15. The system of claim 13, wherein the instructions upon being executed further cause the system to:

determine a first similarity between first features of a first node corresponding to the network data and second features of a second node corresponding to the network data; and

determine a second similarity between third features of the first node corresponding to the process data and fourth features of the second node corresponding to the process data, and

wherein a similarity between the first node and the second node is computed based at least in part on the first similarity and the second similarity.

16. The system of claim 15, wherein the first similarity corresponds to at least one of a first machine learning implementation, a first similarity function, or a first averaging function, and the second similarity corresponds to at least one of a second machine learning implementation, a second similarity function, or a second averaging function.

17. A non-transitory computer-readable medium having computer readable instructions that, upon being executed by a processor, cause the processor to:

receive network data and process data corresponding to a set of selected nodes;

determine a set of unselected nodes in communication with the set of selected nodes based at least in part on the network data and the process data;

determine respective features for each node of the set of selected nodes and the set of unselected nodes based at least in part on the network data and the process data;

compute respective similarities between each node of the set of selected nodes and the set of unselected nodes based at least in part on the respective features; and

determine a plurality of clusters for the set of selected nodes and the set of unselected nodes based on the respective similarities.

18. The non-transitory computer-readable medium of claim 17, wherein the instructions upon being executed further cause the processor to:

receive third domain data corresponding to the set of selected nodes,

wherein the respective features for each node further includes first features corresponding to the third domain data,

wherein the respective similarities between each node is computed based at least in part on the respective features, and

wherein the third domain data includes at least one of host data, user data, virtualization data, or application data.

19. The non-transitory computer-readable medium of claim 17, wherein the instructions further cause the processor to:

determine a first silhouette score for a first node with respect to a first cluster;

determine a second silhouette score for the first node with respect to the first cluster; and

validate the first cluster based at least in part on the first silhouette score exceeding the second silhouette score.

20. The non-transitory computer-readable medium of claim 17, wherein the instructions further cause the processor to:

determine a dependency between a first cluster and a second cluster of the plurality of cluster; and

determine a policy between a first endpoint group corresponding to the first cluster and a second endpoint group corresponding to the second cluster based on the dependency.