GENERATING EVENTS FROM HOST BASED LOGGING FOR CONSUMPTION BY A NETWORK LOGGING HOST

• US 20200136938A1
• Filed: 10/31/2018
• Published: 04/30/2020
• Est. Priority Date: 10/31/2018
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: ATKINSON, Jeffery S.
• Granted Patent: US 11,190,420 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 2221/2113 Multi-level security, e.g. mandatory access control
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 43/045 for graphical visualisation of monitoring data
• CPC Class Codes: H04L 43/062 related to network traffic
• CPC Class Codes: H04L 43/12 Network monitoring probes
• CPC Class Codes: H04L 63/0236 Filtering by address, protocol, port number or service, e.g. IP-address or URL
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 63/101 Access control lists [ACL]
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/303 Terminal profiles
• CPC Class Codes: H04L 9/3066 involving algebraic varieties, e.g. elliptic or hyper-elliptic curves
• CPC Class Codes: H04L 9/3236 using cryptographic hash functions

Abstract

In an example, a network traffic analysis system including a network based logging host may include a transport module, an event parser, and one or more consumer modules. The transport module may collect one or more messages including one or more event logs from one or more remote hosts, respectively. The event parser may generate normalized events consumable by the network logging host from the collected messages. The consumer modules may host process metadata of the event out to file for analysis. Other embodiments may be disclosed and/or claimed.

Claims

1. A network traffic analysis system including a network based logging host, the system comprising:

a processing system; and

a memory device coupled to the processing system and including instructions stored thereon that, in response to execution by the processing system, are operable to perform operations including;

collecting, using a log transport module, one or more messages including one or more host event logs from the one or more remote hosts, respectively; and

inputting the collected messages into an event parser, the event parser to generate normalized events consumable by the network logging host from the collected messages, the event parser configured to;

classify each message based on one of a plurality of predetermined event types;

apply a rule of a plurality of predetermined rules to each event based on the classified event types to select content from the message; and

extract the selected content and generate an event based on the extracted content and the corresponding rule; and

exposing the generated events to one or more consumer modules of the network logging host.

2. The network traffic analysis system of claim 1, wherein a consumer module of the one or more consumer modules is configured to extend at least one of the generated events with one or more additional values to attribute a network process identified using network traffic analysis to a host process indicated by the generated event.

3. The network traffic analysis system of claim 2, wherein at least one of the one or more additional values comprises an SSL (secure socket layer) client fingerprint.

4. The network traffic analysis system of claim 3, wherein the SSL client fingerprint is generated by a client fingerprinting system comprising a database and a client fingerprinting module.

5. The network traffic analysis system of claim 1, wherein an operation system of the network based logging host is different than one or more operating systems of the one or more remote hosts, respectively.

6. The network traffic analysis system of claim 1, wherein a consumer module of the one or more consumer modules is configured to:

identify a network event based on network traffic analysis;

use a process identified (PID) of the identified event to lookup a network connection from a network connection table;

use the network connection to lookup a hash indicative of a host process associated with one of the one or more remote hosts from a hash table; and

log out a file including information about the network event and the hash.

7. A network traffic analysis system including a network based logging platform, the network traffic analysis system comprising:

a transport module configured to collect one or more messages over a network, the one or more messages including one or more host event logs from the one or more remote hosts, respectively;

wherein the network based logging platform comprises one or more processors, and the network traffic analysis system further comprises;

a script configured to establish a communication link with the one or more processors;

the script configured to receive data from the transport module in a predetermined format, the data based on the one or more host event logs;

the script configured to extract key values from the received data and assign the key values to variables;

the script configured to construct one or more events using the variables and provide the one or more events over the communication link.

8. The network traffic analysis system of claim 7, wherein the script comprises Python code.

9. The network traffic analysis system of claim 7, wherein the predetermined format comprises JSON (JavaScript Object Notation).

10. The network traffic analysis system of claim 7, wherein the host event logs originate from a first operating system of the one or more remote hosts, and wherein a second operating system of the network based logging platform is different than the first operating system.

11. The network traffic analysis system of claim 7, wherein the network based logging platform comprises a Linux host and the one or more remote hosts comprise one or more Windows hosts.

12. The network traffic analysis system of claim 7, wherein the network traffic analysis system further comprises one or more additional scripts to correlate network based logging metadata originating from the network based logging platform with host based logging metadata of the one or more events.

13. The network traffic analysis system of claim 12, wherein the one or more processors are configured to generate a first table to track process create events associated with the one or more host event logs, to generate a second table to track network connection events associated with the one or more host event logs;

wherein an additional script of the one or more additional scripts is configured to, in response to identification of a network connection event based on inspecting network traffic of a link;

lookup a PID (process identifier) for the network connection event by using the second table;

lookup a hash value for the PID using the first table; and

create a log file indicative of network based logging and host based logging using the hash value and information of a corresponding event of the one or more events.

14. The network traffic analysis system of claim 13, wherein the additional script is further to insert a JA3 or an X.509 certificate into the file.

15. The network traffic analysis system of claim 7, further comprising an SSL (secure socket layer) client fingerprint system including a fingerprinting database to store SSL client fingerprints generated inspecting network traffic on a link coupled to a database system.

16. A method, comprising:

collecting, at a network traffic analysis system including a first network based logging host and over a network, one or more messages including one or more host event logs from the one or more second hosts, respectively;

establishing a communication link with the first network based logging host;

generating object notation data from data of the one or more messages;

extracting key values from the object notation data and assign the key values to variables;

constructing one or more events using the variables; and

transmitting the one or more events over the communication link.

17. The method of claim 16, wherein object notation data is JSON (JavaScript Object Notation) formatted.

18. The method of claim 16, further comprising logging out the one or more events to a file using the first network logging host following receipt over the communication link.

19. The method of claim 16, further comprising:

attempting to correlate a network event identified based on network traffic analysis of a link coupled to a database system with the one or more events;

in response to a correlation of the network event with an event of the one or more events, logging out information of the network event and information of the event of the one or more events to a file using the first network logging host.

20. The method of claim 16, wherein an operating system of the first network based logging host is different than an operating system of the one or more second hosts.