Technologies for managing compromised sensors in virtualized environments

• US 10,505,828 B2
• Filed: 06/02/2016
• Issued: 12/10/2019
• Est. Priority Date: 06/05/2015
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Yadav, Navindra, Singh, Abhishek Ranjan, Gupta, Anubhav, Gandham, Shashidhar, Pang, Jackson Ngoc Ki, Chang, Shih-Chun, Vu, Hai Trong
• Primary Examiner(s): Nguyen, Trong H
• Application Number:
  US15/171,763
• Publication Number:
  US 20160359889A1
• Time in Patent Office: 1,286 Days
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/137 Hash-based content-based indexing of textual data G06F16/31
• CPC Class Codes: G06F 16/162 Delete operations erasing in storage systems G06F3/0652
• CPC Class Codes: G06F 16/17 Details of further file system functions
• CPC Class Codes: G06F 16/173 Customisation support for file systems, e.g. localisation, multi-language support, personalisation
• CPC Class Codes: G06F 16/174 Redundancy elimination performed by the file system management of the data involved in backup or backup restore using de-duplication of the data G06F11/14
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/2322 using timestamps
• CPC Class Codes: G06F 16/235 Update request formulation
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/288 Entity relationship models
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2009/45587 Isolation or security of virtual machine instances
• CPC Class Codes: G06F 2009/45591 Monitoring or debugging support
• CPC Class Codes: G06F 2009/45595 : Network integration; Enabling network access in virtual machine instances
• CPC Class Codes: G06F 21/53 : by executing in a restricted environment, e.g. sandbox or secure virtual machine
• CPC Class Codes: G06F 21/552 : involving long-term monitoring or reporting
• CPC Class Codes: G06F 21/556 : involving covert channels, i.e. data leakage between processes inhibiting the analysis of circuitry or operation with measures against power attack G06F21/755
• CPC Class Codes: G06F 21/566 : Dynamic detection, i.e. detection performed at run-time, e.g. emulation, suspicious activities
• CPC Class Codes: G06F 2221/033 : Test or assess software
• CPC Class Codes: G06F 2221/2101 : Auditing as a secondary aspect
• CPC Class Codes: G06F 2221/2105 : Dual mode as a secondary aspect
• CPC Class Codes: G06F 2221/2111 : Location-sensitive, e.g. geographical location, GPS
• CPC Class Codes: G06F 2221/2115 : Third party
• CPC Class Codes: G06F 2221/2145 : Inheriting rights or properties, e.g., propagation of permissions or restrictions within a hierarchy
• CPC Class Codes: G06F 3/0482 : Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 : Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 : Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 9/45558 : Hypervisor-specific management and integration aspects
• CPC Class Codes: G06N 20/00 : Machine learning
• CPC Class Codes: G06N 99/00 : Subject matter not provided for in other groups of this subclass
• CPC Class Codes: G06T 11/206 : Drawing of charts or graphs
• CPC Class Codes: H04J 3/0661 : using timestamps
• CPC Class Codes: H04J 3/14 : Monitoring arrangements for SDH/SONET rings H04J3/085
• CPC Class Codes: H04L 1/242 : by comparing a transmitted test signal with a locally generated replica
• CPC Class Codes: H04L 41/046 : comprising network management agents or mobile agents therefor
• CPC Class Codes: H04L 41/0668 : by dynamic selection of recovery network elements, e.g. replacement by the most appropriate element after failure
• CPC Class Codes: H04L 41/0803 : Configuration setting
• CPC Class Codes: H04L 41/0806 : for initial configuration or provisioning, e.g. plug-and-play
• CPC Class Codes: H04L 41/0816 : the condition being an adaptation, e.g. in response to network events
• CPC Class Codes: H04L 41/0893 : Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/0894 : Policy-based network configuration management
• CPC Class Codes: H04L 41/12 : Discovery or management of network topologies
• CPC Class Codes: H04L 41/16 : using machine learning or artificial intelligence
• CPC Class Codes: H04L 41/22 : comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 41/40 : using virtualisation of network functions or resources, e.g. SDN or NFV entities
• CPC Class Codes: H04L 43/02 : Capturing of monitoring data
• CPC Class Codes: H04L 43/026 : using flow identification
• CPC Class Codes: H04L 43/04 : Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 43/045 : for graphical visualisation of monitoring data
• CPC Class Codes: H04L 43/062 : related to network traffic
• CPC Class Codes: H04L 43/08 : Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
• CPC Class Codes: H04L 43/0805 : by checking availability
• CPC Class Codes: H04L 43/0811 : by checking connectivity
• CPC Class Codes: H04L 43/0829 : Packet loss
• CPC Class Codes: H04L 43/0841 : Round trip packet loss
• CPC Class Codes: H04L 43/0858 : One way delays
• CPC Class Codes: H04L 43/0864 : Round trip delays
• CPC Class Codes: H04L 43/0876 : Network utilisation, e.g. volume of load or congestion level
• CPC Class Codes: H04L 43/0882 : Utilisation of link capacity
• CPC Class Codes: H04L 43/0888 : Throughput
• CPC Class Codes: H04L 43/10 : Active monitoring, e.g. heartbeat, ping or trace-route
• CPC Class Codes: H04L 43/106 : using time related information in packets, e.g. by adding timestamps
• CPC Class Codes: H04L 43/12 : Network monitoring probes
• CPC Class Codes: H04L 43/16 : Threshold monitoring
• CPC Class Codes: H04L 43/20 : the monitoring system or the monitored elements being virtualised, abstracted or software-defined entities, e.g. SDN or NFV
• CPC Class Codes: H04L 45/306 : Route determination based on the nature of the carried application
• CPC Class Codes: H04L 45/38 : Flow based routing
• CPC Class Codes: H04L 45/46 : Cluster building
• CPC Class Codes: H04L 45/507 : Label distribution
• CPC Class Codes: H04L 45/66 : Layer 2 routing, e.g. in Ethernet based MAN's
• CPC Class Codes: H04L 45/74 : Address processing for routing
• CPC Class Codes: H04L 47/11 : Identifying congestion
• CPC Class Codes: H04L 47/20 : Traffic policing
• CPC Class Codes: H04L 47/2441 : relying on flow classification, e.g. using integrated services [IntServ]
• CPC Class Codes: H04L 47/2483 : involving identification of individual flows
• CPC Class Codes: H04L 47/28 : in relation to timing considerations
• CPC Class Codes: H04L 47/31 : by tagging of packets, e.g. using discard eligibility [DE] bits
• CPC Class Codes: H04L 47/32 : by discarding or delaying data units, e.g. packets or frames
• CPC Class Codes: H04L 61/5007 : Internet protocol [IP] addresses
• CPC Class Codes: H04L 63/0227 : Filtering policies mail message filtering H04L51/212
• CPC Class Codes: H04L 63/0263 : Rule management
• CPC Class Codes: H04L 63/06 : for supporting key management in a packet data network cryptographic mechanisms or cryptographic arrangements for key management H04L9/08
• CPC Class Codes: H04L 63/0876 : based on the identity of the terminal or configuration, e.g. MAC address, hardware or software configuration or device fingerprint
• CPC Class Codes: H04L 63/1408 : by monitoring network traffic monitoring network traffic per se H04L43/00
• CPC Class Codes: H04L 63/1416 : Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1425 : Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/1433 : Vulnerability analysis
• CPC Class Codes: H04L 63/1441 : Countermeasures against malicious traffic countermeasures against attacks on cryptographic mechanisms H04L9/002
• CPC Class Codes: H04L 63/145 : the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms
• CPC Class Codes: H04L 63/1458 : Denial of Service
• CPC Class Codes: H04L 63/1466 : Active attacks involving interception, injection, modification, spoofing of data unit addresses, e.g. hijacking, packet injection or TCP sequence number attacks
• CPC Class Codes: H04L 63/16 : Implementing security features at a particular protocol layer
• CPC Class Codes: H04L 63/20 : for managing network security; network security policies in general filtering policies H04L63/0227
• CPC Class Codes: H04L 67/01 : Protocols
• CPC Class Codes: H04L 67/10 : in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1001 : for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/12 : specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04L 67/51 : Discovery or management thereof, e.g. service location protocol [SLP] or web services
• CPC Class Codes: H04L 67/535 : Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04L 67/75 : Indicating network or usage conditions on the user display
• CPC Class Codes: H04L 69/16 : Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
• CPC Class Codes: H04L 69/22 : Parsing or analysis of headers
• CPC Class Codes: H04L 7/10 : Arrangements for initial synchronisation
• CPC Class Codes: H04L 9/0866 : involving user or device identifiers, e.g. serial number, physical or biometrical information, DNA, hand-signature or measurable physical characteristics
• CPC Class Codes: H04L 9/3239 : involving non-keyed hash functions, e.g. modification detection codes [MDCs], MD5, SHA or RIPEMD
• CPC Class Codes: H04L 9/3242 : involving keyed hash functions, e.g. message authentication codes [MACs], CBC-MAC or HMAC
• CPC Class Codes: H04W 72/54 : based on quality criteria
• CPC Class Codes: H04W 84/18 : Self-organising networks, e.g. ad-hoc networks or sensor networks

Abstract

Systems, methods, and computer-readable media for managing compromised sensors in multi-tiered virtualized environments. In some embodiments, a system can receive, from a first capturing agent deployed in a virtualization layer of a first device, data reports generated based on traffic captured by the first capturing agent. The system can also receive, from a second capturing agent deployed in a hardware layer of a second device, data reports generated based on traffic captured by the second capturing agent. Based on the data reports, the system can determine characteristics of the traffic captured by the first capturing agent and the second capturing agent. The system can then compare the characteristics to determine a multi-layer difference in traffic characteristics. Based on the multi-layer difference in traffic characteristics, the system can determine that the first capturing agent or the second capturing agent is in a faulty state.

Claims

1. A method comprising:

receiving, from a first capturing agent deployed in a virtualization layer of a first device, a first data report generated by the first capturing agent based on traffic at the first device captured by the first capturing agent at the virtualization layer of the first device;

receiving, from a second capturing agent deployed in a hardware layer of a second device different than the first device, a second data report generated by the second capturing agent based on traffic at the second device captured by the second capturing agent at the hardware layer of the second device;

based on the first data report and the second data report, determining a first set of characteristics of the traffic captured by the first capturing agent and a second set of characteristics of the traffic captured by the second capturing agent;

comparing the first set of characteristics of the traffic captured by the first capturing agent with the second set of characteristics captured by the second capturing agent to determine a multi-layer difference in traffic characteristics; and

based on the multi-layer difference in traffic characteristics, determining that one of the first capturing agent or the second capturing agent is in a faulty state,

wherein,

the first data report and the second data report are of a plurality of data reports generated during reporting intervals based on observed data, statistics, and/or metadata about one or more packets, flows, communications, processes, events, and/or activities at the first device and the second device.

2. The method of claim 1, wherein the virtualization layer comprises one of a virtual machine or a hypervisor and the second device comprises a leaf switch in a spine-leaf network fabric, the first device comprising a host of the hypervisor coupled with the spine-leaf network fabric via the leaf switch.

3. The method of claim 2, wherein the faulty state comprises unauthorized activity, and wherein the first set of characteristics and the second set of characteristics comprise respective amounts of traffic captured at the virtualization layer of the first device and the hardware layer of the second device.

4. The method of claim 3, wherein the determining that one of the first capturing agent or the second capturing agent is in the faulty state comprises:

determining that the respective amounts of traffic indicate a threshold discrepancy between a first amount of traffic captured by the first capturing agent at the host and a second amount of traffic captured by the second capturing agent at the leaf switch; and

determining that the threshold discrepancy is at least partially a result of unauthorized activity at at least one of the first capturing agent or the second capturing agent.

5. The method of claim 4, wherein the determining that the threshold discrepancy is at least partially the result of the unauthorized activity comprises determining the first capturing agent at the host is in the faulty state when the first amount of traffic at the host is greater than the second amount of traffic by a threshold amount.

6. The method of claim 4, wherein the first amount of traffic comprises a number of hits to a database residing at the hypervisor on the host.

7. The method of claim 1, wherein the determining that one of the first capturing agent or the second capturing agent is in the faulty state further comprises:

based on the first data report from the first capturing agent, determining a first traffic pattern for traffic captured by the first capturing agent at the first device during a first period of time;

based on the second data report from the second capturing agent, determining a second traffic pattern for traffic captured by the second capturing agent at the second device during the first period of time;

based on a first previous report from the first capturing agent, determining a third traffic pattern for traffic captured by the first capturing agent at the first device during a second period of time before the first period of time;

based on a second previous report from the second capturing agent, determining a fourth traffic pattern for traffic captured by the second capturing agent at the second device during the second period of time;

comparing the first traffic pattern with the third traffic pattern to identify a first traffic pattern delta between the first traffic pattern and the third traffic pattern;

comparing the second traffic pattern with the fourth traffic pattern to identify a second traffic pattern delta between the second traffic pattern and the fourth traffic pattern;

determining whether the first traffic pattern delta or the second traffic pattern delta exceed a delta threshold;

when the first traffic pattern delta exceeds the delta threshold, determining that the first capturing agent is in the faulty state; and

when the second traffic pattern delta exceeds the delta threshold, determining that the second capturing agent is in the faulty state.

8. The method of claim 7, wherein:

the first traffic pattern delta comprises a first delta in at least one of;

a first amount of traffic captured by the first capturing agent during the first period of time and the second period of time;

or

a first frequency of traffic captured by the first capturing agent during the first period of time and the second period of time; and

the second traffic pattern delta comprises a second delta in at least one of;

a second amount of traffic captured by the second capturing agent during the first period of time and the second period of time;

or

a second frequency of traffic captured by the second capturing agent during the first period of time and the second period of time.

9. The method of claim 1, wherein the first capturing agent and the second capturing agent comprise at least one of a process, a kernel module, or a software driver.

10. The method of claim 1, further comprising:

in response to the determining that at least one of the first capturing agent or the second capturing agent is in the faulty state, marking traffic or packets in at least one of the first data report and/or the second data report as unreliable.

11. The method of claim 1, further comprising:

in response to the determining that at least one of the first capturing agent or the second capturing agent is in the faulty state;

aggregating or summarizing data in at least one of the first data report from the first capturing agent or the second data report from the second capturing agent;

or

reducing an amount of data in at least one of the first data report from the first capturing agent or the second data report from the second capturing agent retained after receiving one of the first data report or the second data report.

12. The method of claim 1, further comprising:

in response to the determining that at least one of the first capturing agent or the second capturing agent is in the faulty state,

increasing a time interval for receiving subsequent reports from at least one of the first capturing agent or the second capturing agent, to reduce an amount of data reported from the at least one of the first capturing agent or the second capturing agent during the faulty state.

13. The method of claim 1, wherein the first data report from the first capturing agent and the second data report from the second capturing agent are received via a collector device, the method further comprising:

in response to the determining that at least one of the first capturing agent or the second capturing agent is in the faulty state;

dropping, by the collector device, data transmitted by the at least one of the first capturing agent or the second capturing agent;

or

preventing, by the collector device, access by other devices to data received from the at least one of the first capturing agent or the second capturing agent.

14. The method of claim 1, further comprising:

determining that at least one other capturing agent is in the faulty state based on a topology of an associated network and a placement in the associated network of the at least one of the first capturing agent or the second capturing agent relative to the at least one other capturing agent.

15. A system comprising:

one or more processors; and

one or more computer-readable storage devices having stored therein instructions which, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

receiving, from a first capturing agent deployed in a virtualization layer of a first device, a first data report generated by the first capturing agent based on traffic at the first device captured by the first capturing agent at the virtualization layer of the first device;

receiving, from a second capturing agent deployed in a hardware layer of a second device different than the first device, a second data report generated by the second capturing agent based on traffic at the second device captured by the second capturing agent at the hardware layer of the second device;

based on the first data report and the second data report, determining a first set of characteristics of the traffic captured by the first capturing agent and a second set of characteristics of the traffic captured by the second capturing agent;

comparing the first set of characteristics of the traffic captured by the first capturing agent with the second set of characteristics captured by the second capturing agent to determine a multi-layer difference in traffic characteristics; and

based on the multi-layer difference in traffic characteristics, determining that one of the first capturing agent or the second capturing agent is in a faulty state,

wherein,

the first data report and the second data report are of a plurality of data reports generated during reporting intervals based on observed data, statistics, and/or metadata about one or more packets, flows, communications, processes, events, and/or activities at the first device and the second device.

16. The system of claim 15, wherein the virtualization layer comprises a hypervisor and the second device comprises a switch in a network fabric, the first device comprising a host of the hypervisor and being coupled with the network fabric via the switch, wherein:

the first set of characteristics and the second set of characteristics comprise respective amounts of traffic captured at the virtualization layer of the first device and the hardware layer of the second device; and

the determining that at least one of the first capturing agent or the second capturing agent is in the faulty state comprises;

determining that the respective amounts of traffic indicate a threshold discrepancy between a first amount of traffic captured by the first capturing agent at the host and a second amount of traffic captured by the second capturing agent at a leaf switch; and

determining that the threshold discrepancy is at least partially a result of unauthorized activity at the at least one of the first capturing agent or the second capturing agent.

17. The system of claim 15, wherein the second device is a switch and the first device is a server coupled with a network fabric via the switch, and wherein the determining that at least one of the first capturing agent or the second capturing agent is in the faulty state further comprises:

based on the first data report from the first capturing agent and the second data report from the second capturing agent, determining a first traffic pattern for traffic captured by the first capturing agent during a first period of time and a second traffic pattern for traffic captured by the second capturing agent during the first period of time;

based on a first previous report from the first capturing agent and a second previous data report from the second capturing agent, determining a third traffic pattern for traffic captured by the first capturing agent during a second period of time and a fourth traffic pattern for traffic captured by the second capturing agent during the second period of time;

comparing the first traffic pattern with the third traffic pattern and the second traffic pattern with the fourth traffic pattern to identify a first traffic pattern delta between the first traffic pattern and the third traffic pattern and a second traffic pattern delta between the second traffic pattern and the fourth traffic pattern;

when the first traffic pattern delta exceeds a delta threshold, determining that the first capturing agent is in the faulty state; and

when the second traffic pattern delta exceeds the delta threshold, determining that the second capturing agent is in the faulty state.

18. The system of claim 15, the one or more computer-readable storage devices storing additional instructions which, when executed by the one or more processors, cause the one or more processors to perform operations comprising, in response to the determining that at least one of the first capturing agent or the second capturing agent is in the faulty state:

flagging traffic or packets in at least one of the first data report from the first capturing agent or the second data report from the second capturing agent;

dropping data transmitted by the at least one of the first capturing agent or the second capturing agent;

or

preventing access by other devices to data received from the at least one of the first capturing agent or the second capturing agent.

19. The system of claim 15, the one or more computer-readable storage devices storing additional instructions which, when executed by the one or more processors, cause the one or more processors to perform operations comprising, in response to the determining that at least one of the first capturing agent or the second capturing agent is in the faulty state:

increasing a time interval for receiving subsequent reports from at least one of the first capturing agent or the second capturing agent, to reduce an amount of data reported from the at least one of the first capturing agent or the second capturing agent during the faulty state;

aggregating or summarizing data in subsequent reports from at least one of the first capturing agent or the second capturing agent;

or

reducing an amount of data, from subsequent reports received from the first capturing agent or the second capturing agent, retained after receipt.

20. A computer-readable storage device storing instructions which, when executed by a processor, cause the processor to perform operations comprising:

receiving, from a first capturing agent deployed in a virtualization layer of a first device, one or more data reports generated by the first capturing agent based on traffic at the first device captured by the first capturing agent at the virtualization layer of the first device;

receiving, from a second capturing agent deployed in a hardware layer of a second device different than the first device, one or more data reports generated by the second capturing agent based on traffic at the second device captured by the second capturing agent at the hardware layer of the second device;

based on the one or more data reports from the first capturing agent and the second capturing agent, determining a first set of characteristics of the traffic captured by the first capturing agent and a second set of characteristics of the traffic captured by the second capturing agent;

comparing the first set of characteristics of the traffic captured by the first capturing agent with the second set of characteristics captured by the second capturing agent to determine a multi-layer difference in traffic characteristics; and

based on the multi-layer difference in traffic characteristics, determining that one of the first capturing agent or the second capturing agent is in a faulty state, wherein, the first data report and the second data report are of a plurality of data reports generated during reporting intervals based on observed data, statistics, and/or metadata about one or more packets, flows, communications, processes, events, and/or activities at the first device and the second device.