Correlating causes and effects associated with network activity

• US 10,411,978 B1
• Filed: 08/09/2018
• Issued: 09/10/2019
• Est. Priority Date: 08/09/2018
• Status: Active Grant

• Current Assignee: ExtraHop Networks, Incorporated
• Original Assignee: ExtraHop Networks, Incorporated
• Inventors: Ball, Eric Jacob, Hammerle, Eric Joseph, Higgins, Benjamin Thomas, Khanal, Bhushan Prasad, Montague, Michael Kerber Krause, Wu, Xue Jun
• Primary Examiner(s): Joo, Joshua
• Application Number:
  US16/100,116
• Time in Patent Office: 397 Days
• Field of Search: None
• CPC Class Codes: H04L 41/0816 the condition being an adaptation, e.g. in response to network events
• CPC Class Codes: H04L 41/145 involving simulating, designing, planning or modelling of a network
• CPC Class Codes: H04L 41/147 for predicting network behaviour
• CPC Class Codes: H04L 41/16 using machine learning or artificial intelligence
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 43/062 related to network traffic
• CPC Class Codes: H04L 43/08 Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
• CPC Class Codes: H04L 43/12 Network monitoring probes
• CPC Class Codes: H04L 43/20 the monitoring system or the monitored elements being virtualised, abstracted or software-defined entities, e.g. SDN or NFV

Abstract

Embodiments are directed to monitoring network traffic using a monitoring engine that monitors network traffic in networks to provide metrics. An inference engine may provide activity profiles based on portions of the network traffic where each activity profile includes features associated with the portions of network traffic. The inference engine may determine other activity profiles correlated with the activity profiles based on correlation models such that the determination of the other activity profiles occurs prior to monitoring an occurrence of other portions of the network traffic. The inference engine may modify monitoring actions of the monitoring engine based on the other activity profiles. The inference engine may provide reports based on the portions of the network traffic, the activity profiles, the other portions of the network traffic, or the other activity profiles.

Claims

1. A method for monitoring network traffic using one or more network computers, wherein execution of instructions by the one or more network computers perform the method comprising:

instantiating a monitoring engine to perform actions, including;

monitoring one or more portions of the network traffic that are associated with a plurality of entities in one or more networks to provide one or more metrics; and

instantiating an inference engine that performs actions, including;

providing one or more activity profiles based on the plurality of entities and the one or more portions of the network traffic, wherein each activity profile includes features based on the one or more metrics, the plurality of entities, or the one or more portions of the network traffic;

determining one or more other activity profiles that correlate with the one or more activity profiles based on one or more correlation models;

monitoring one or more other portions of the network traffic associated with the one or more other activity profiles, wherein the determination of the one or more other activity profiles occurs separate from the monitoring of the one or more other portions of the network traffic that are associated with the one or more other activity profiles;

modifying one or more actions of the monitoring engine based on the one or more other activity profiles; and

providing one or more reports based on the one or more portions of the network traffic, the one or more activity profiles, the one or more other portions of the network traffic, or the one or more other activity profiles, wherein the one or more reports are provided to one or more users.

2. The method of claim 1, wherein modifying the one or more actions of the monitoring engine, further comprises, modifying the monitoring of the network traffic based on the one or more other activity profiles, wherein the modifications include collecting one or more additional metrics associated with the one or more other portions of the network traffic that are associated with the one or more other activity profiles.

3. The method of claim 1, wherein the inference engine performs further actions, including, associating the one or more portions of the network traffic with the one or more activity profiles based on the one or more portions of the network traffic matching one or more of the features that are associated with the one or more activity profiles.

4. The method of claim 1, wherein providing the one or more activity profiles, further comprises, matching the one or more portions of the network traffic to the one or more activity profiles based on one or more characteristics of the one or more portions of the network traffic, wherein the one or more characteristics include one or more of network flow features, packet size, bit rate of the network traffic, communication protocol, encryption status, encryption state, encryption version, encryption type, encryption cipher, error rate, device relations based on one or more device relation models, time-of-day, day-of-week, user, user group, source tuple information, destination tuple information, application type, service type, payload or header content, or geolocation information.

5. The method of claim 1, wherein the inference engine performs further actions, including, training the one or more correlation models based on the network traffic and the one or more activity profiles, wherein the one or more correlation models provide correlations that predict a likelihood of the occurrence of the one or more other portions of network traffic subsequent to the occurrence of the one or more portions of the network traffic based on one or more dependencies associated with the one or more activity profiles and the one or more other activity profiles.

6. The method of claim 1, wherein the inference engine performs further actions, including:

determining a score for each of the one or more correlation models based on the occurrence of the one or more other portions of network traffic subsequent to the one or more portions of the network traffic;

determining the one or more correlation models that require re-training based on the determined score having a value that is below a threshold value; and

re-training the one or more determined correlation models based on the network traffic.

7. The method of claim 1, wherein providing the one or more activity profiles, further comprises, providing computer readable instructions that are executed to enforce one or more matching rules or filter rules that are associated with the one or more activity profiles, wherein the one or more matching rules or filter rules include one or more of regular expressions, one or more compound rules, one or more cascading rules, or one or more sub-rules.

8. The method of claim 1, wherein providing the one or more reports, further comprises, one or more of detecting one or more anomalies associated with the one or more activity profiles, identifying one or more capacity planning actions based on the one or more activity profiles, or tracing one or more transactions based on the one or more activity profiles.

9. A processor readable non-transitory storage media that includes instructions for monitoring network traffic using one or more network monitoring computers, wherein execution of the instructions by the one or more network computers perform the method comprising:

instantiating a monitoring engine to perform actions, including;

monitoring one or more portions of the network traffic that are associated with a plurality of entities in one or more networks to provide one or more metrics; and

instantiating an inference engine that performs actions, including;

providing one or more activity profiles based on the plurality of entities and the one or more portions of the network traffic, wherein each activity profile includes features based on the one or more metrics, the plurality of entities, or the one or more portions of the network traffic;

determining one or more other activity profiles that correlate with the one or more activity profiles based on one or more correlation models;

monitoring one or more other portions of the network traffic associated with the one or more other activity profiles, wherein the determination of the one or more other activity profiles occurs separate from the monitoring of the one or more other portions of the network traffic that are associated with the one or more other activity profiles;

modifying one or more actions of the monitoring engine based on the one or more other activity profiles; and

providing one or more reports based on the one or more portions of the network traffic, the one or more activity profiles, the one or more other portions of the network traffic, or the one or more other activity profiles, wherein the one or more reports are provided to one or more users.

10. The media of claim 9, wherein modifying the one or more actions of the monitoring engine, further comprises, modifying the monitoring of the network traffic based on the one or more other activity profiles, wherein the modifications include collecting one or more additional metrics associated with the one or more other portions of the network traffic that are associated with the one or more other activity profiles.

11. The media of claim 9, wherein the inference engine performs further actions, including, associating the one or more portions of the network traffic with the one or more activity profiles based on the one or more portions of the network traffic matching one or more of the features that are associated with the one or more activity profiles.

12. The media of claim 9, wherein providing the one or more activity profiles, further comprises, matching the one or more portions of the network traffic to the one or more activity profiles based on one or more characteristics of the one or more portions of the network traffic, wherein the one or more characteristics include one or more of network flow features, packet size, bit rate of the network traffic, communication protocol, encryption status, encryption state, encryption version, encryption type, encryption cipher, error rate, device relations based on one or more device relation models, time-of-day, day-of-week, user, user group, source tuple information, destination tuple information, application type, service type, payload or header content, or geolocation information.

13. The media of claim 9, wherein the inference engine performs further actions, including, training the one or more correlation models based on the network traffic and the one or more activity profiles, wherein the one or more correlation models provide correlations that predict a likelihood of the occurrence of the one or more other portions of network traffic subsequent to the occurrence of the one or more portions of the network traffic based on one or more dependencies associated with the one or more activity profiles and the one or more other activity profiles.

14. The media of claim 9, wherein the inference engine performs further actions, including:

determining a score for each of the one or more correlation models based on the occurrence of the one or more other portions of network traffic subsequent to the one or more portions of the network traffic;

determining the one or more correlation models that require re-training based on the determined score having a value that is below a threshold value; and

re-training the one or more determined correlation models based on the network traffic.

15. The media of claim 9, wherein providing the one or more activity profiles, further comprises, providing computer readable instructions that are executed to enforce one or more matching rules or filter rules that are associated with the one or more activity profiles, wherein the one or more matching rules or filter rules include one or more of regular expressions, one or more compound rules, one or more cascading rules, or one or more sub-rules.

16. A system for monitoring network traffic in a network:

one or more network computers, comprising;

a transceiver that communicates over the network;

a memory that stores at least instructions; and

one or more processors that execute instructions that perform actions, including;

instantiating a monitoring engine to perform actions, including;

monitoring one or more portions of the network traffic that are associated with a plurality of entities in one or more networks to provide one or more metrics; and

instantiating an inference engine that performs actions, including;

providing one or more activity profiles based on the plurality of entities and the one or more portions of the network traffic, wherein each activity profile includes features based on the one or more metrics, the plurality of entities, or the one or more portions of the network traffic;

determining one or more other activity profiles that correlate with the one or more activity profiles based on one or more correlation models;

monitoring one or more other portions of the network traffic associated with the one or more other activity profiles, wherein the determination of the one or more other activity profiles occurs separate from the monitoring of the one or more other portions of the network traffic that are associated with the one or more other activity profiles;

modifying one or more actions of the monitoring engine based on the one or more other activity profiles; and

providing one or more reports based on the one or more portions of the network traffic, the one or more activity profiles, the one or more other portions of the network traffic, or the one or more other activity profiles, wherein the one or more reports are provided to one or more users; and

one or more client computers, comprising;

a transceiver that communicates over the network;

a memory that stores at least instructions; and

one or more processors that execute instructions that perform actions, including;

providing one or more of the one or more portions of the network traffic.

17. The system of claim 16, wherein modifying the one or more actions of the monitoring engine, further comprises, modifying the monitoring of the network traffic based on the one or more other activity profiles, wherein the modifications include collecting one or more additional metrics associated with the one or more other portions of the network traffic that are associated with the one or more other activity profiles.

18. The system of claim 16, wherein the inference engine performs further actions, including, associating the one or more portions of the network traffic with the one or more activity profiles based on the one or more portions of the network traffic matching one or more of the features that are associated with the one or more activity profiles.

19. The system of claim 16, wherein providing the one or more activity profiles, further comprises, matching the one or more portions of the network traffic to the one or more activity profiles based on one or more characteristics of the one or more portions of the network traffic, wherein the one or more characteristics include one or more of network flow features, packet size, bit rate of the network traffic, communication protocol, encryption status, encryption state, encryption version, encryption type, encryption cipher, error rate, device relations based on one or more device relation models, time-of-day, day-of-week, user, user group, source tuple information, destination tuple information, application type, service type, payload or header content, or geolocation information.

20. The system of claim 16, wherein the inference engine performs further actions, including, training the one or more correlation models based on the network traffic and the one or more activity profiles, wherein the one or more correlation models provide correlations that predict a likelihood of the occurrence of the one or more other portions of network traffic subsequent to the occurrence of the one or more portions of the network traffic based on one or more dependencies associated with the one or more activity profiles and the one or more other activity profiles.

21. The system of claim 16, wherein the inference engine performs further actions, including:

determining a score for each of the one or more correlation models based on the occurrence of the one or more other portions of network traffic subsequent to the one or more portions of the network traffic;

determining the one or more correlation models that require re-training based on the determined score having a value that is below a threshold value; and

re-training the one or more determined correlation models based on the network traffic.

22. The system of claim 16, wherein providing the one or more activity profiles, further comprises, providing computer readable instructions that are executed to enforce one or more matching rules or filter rules that are associated with the one or more activity profiles, wherein the one or more matching rules or filter rules include one or more of regular expressions, one or more compound rules, one or more cascading rules, or one or more sub-rules.

23. The system of claim 16, wherein providing the one or more reports, further comprises, one or more of detecting one or more anomalies associated with the one or more activity profiles, identifying one or more capacity planning actions based on the one or more activity profiles, or tracing one or more transactions based on the one or more activity profiles.

24. A network computer for monitoring communication over a network between two or more computers, comprising:

a transceiver that communicates over the network;

a memory that stores at least instructions; and

one or more processors that execute instructions that perform actions, including;

instantiating a monitoring engine to perform actions, including;

monitoring one or more portions of the network traffic that are associated with a plurality of entities in one or more networks to provide one or more metrics; and

instantiating an inference engine that performs actions, including;

providing one or more activity profiles based on the plurality of entities and the one or more portions of the network traffic, wherein each activity profile includes features based on the one or more metrics, the plurality of entities, or the one or more portions of the network traffic;

determining one or more other activity profiles that correlate with the one or more activity profiles based on one or more correlation models;

monitoring one or more other portions of the network traffic associated with the one or more other activity profiles, wherein the determination of the one or more other activity profiles occurs separate from the monitoring of the one or more other portions of the network traffic that are associated with the one or more other activity profiles;

modifying one or more actions of the monitoring engine based on the one or more other activity profiles; and

providing one or more reports based on the one or more portions of the network traffic, the one or more activity profiles, the one or more other portions of the network traffic, or the one or more other activity profiles, wherein the one or more reports are provided to one or more users.

25. The network computer of claim 24, wherein modifying the one or more actions of the monitoring engine, further comprises, modifying the monitoring of the network traffic based on the one or more other activity profiles, wherein the modifications include collecting one or more additional metrics associated with the one or more other portions of the network traffic that are associated with the one or more other activity profiles.

26. The network computer of claim 24, wherein the inference engine performs further actions, including, associating the one or more portions of the network traffic with the one or more activity profiles based on the one or more portions of the network traffic matching one or more of the features that are associated with the one or more activity profiles.

27. The network computer of claim 24, wherein providing the one or more activity profiles, further comprises, matching the one or more portions of the network traffic to the one or more activity profiles based on one or more characteristics of the one or more portions of the network traffic, wherein the one or more characteristics include one or more of network flow features, packet size, bit rate of the network traffic, communication protocol, encryption status, encryption state, encryption version, encryption type, encryption cipher, error rate, device relations based on one or more device relation models, time-of-day, day-of-week, user, user group, source tuple information, destination tuple information, application type, service type, payload or header content, or geolocation information.

28. The network computer of claim 24, wherein the inference engine performs further actions, including, training the one or more correlation models based on the network traffic and the one or more activity profiles, wherein the one or more correlation models provide correlations that predict a likelihood of the occurrence of the one or more other portions of network traffic subsequent to the occurrence of the one or more portions of the network traffic based on one or more dependencies associated with the one or more activity profiles and the one or more other activity profiles.

29. The network computer of claim 24, wherein the inference engine performs further actions, including:

determining a score for each of the one or more correlation models based on the occurrence of the one or more other portions of network traffic subsequent to the one or more portions of the network traffic;

determining the one or more correlation models that require re-training based on the determined score having a value that is below a threshold value; and

re-training the one or more determined correlation models based on the network traffic.

30. The network computer of claim 24, wherein providing the one or more activity profiles, further comprises, providing computer readable instructions that are executed to enforce one or more matching rules or filter rules that are associated with the one or more activity profiles, wherein the one or more matching rules or filter rules include one or more of regular expressions, one or more compound rules, one or more cascading rules, or one or more sub-rules.