Enabling network intrusion detection by representing network activity in graphical form utilizing distributed data sensors to detect and transmit activity data

• US 8,266,697 B2
• Filed: 03/04/2006
• Issued: 09/11/2012
• Est. Priority Date: 03/04/2006
• Status: Active Grant

• Current Assignee: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Original Assignee: 21st Century Technologies, Inc.
• Inventors: Coffman, Thayne Richard
• Primary Examiner(s): Backer, Firmin
• Assistant Examiner(s): ALGIBHAH, HAMZA N
• Application Number:
  US11/367,944
• Publication Number:
  US 20070209075A1
• Time in Patent Office: 2,383 Days
• Field of Search: 726/23
• US Class Current: 726/23
• CPC Class Codes: H04L 63/14 for detecting or protecting against malicious traffic

Abstract

A method, system, and computer program product for detecting and mapping activity occurring at and between devices on a computer network for utilization within an intrusion detection mechanism. An enhanced graph matching intrusion detection system (eGMIDS) utility executing on a control server provides data collection functions and data fusion techniques. The eGMIDS comprises multiple sensors and associated unique adaptors that are located at different remote devices of the network and utilized to detect specific types of activity occurring at the respective devices relevant to eGMIDS processing. The sensors convert the data into eGMIDS format and encapsulate the data in a special transmission packet that is transmitted to the control server. The eGMIDS utility converts the activity data within these packets into eGMIDS-usable format and then processes the converted data via a data fusion technique to generate a graphical representation of the network (devices) and the activity occurring at/amongst the various devices.

Claims

1. A method for generating a network activity graph comprising:

at a control server, receiving from a first sensor at a first remote device, a message containing remote device information including an identification of the remote device and activity occurring at the remote device, wherein said first sensor comprises an adapter configured to enable sensed activity occurring at the first remote device to be packaged in a specialized format for transmission to the control server, and wherein said first sensor encapsulates the sensed activity into a specific transmission message recognizable by receiving components at the control server and forwards the message to the control server;

the control server fusing activity data retrieved from multiple ones of said message from respective multiple sensors, including the first sensor, located within the network, into an activity graph representative of the devices on the network and the activity and inter-activity occurring at and between the devices on the network;

the control server translating data within an activity report generated from the received activity data into a graph representation and incorporating the translated data into a combined activity graph;

the control server determining which elements within received activity reports are already represented by a node or edge within the activity graph in order to prevent duplication of a mapping within the activity graph of already represented elements;

the control server creating a new node or edge for only those elements not already represented within the activity graph;

the control server generating a request for secondary evidence and transmitting the request to the sensor of the remote device, wherein in response to receipt of a request for secondary evidence at the sensor of the remote device from the control server, said sensor is triggered to locate, package and transmit the requested additional evidence to the control server; and

in response to receipt of the secondary evidence at the control server, the control server automatically translating the secondary evidence into eGMIDS usable format and fusing the secondary evidence into the activity graph;

wherein said multiple sensors comprise an email sensor, which completes the functions of;

tracking emails between users on a network; and

monitoring an exchange of emails within a context, which context includes the sender'"'"'s and recipient'"'"'s other activities in addition to the exchange of emails on the network, said other activities being pre-determined to trigger said monitoring and which occur prior to or concurrent with the exchange of emails to trigger said monitoring, and wherein said monitoring includes monitoring a content of the email for key words that may be associated with a threat when placed in proper context, given the other activity of the sender and/or recipient of the email.

2. The method of claim 1, wherein:

the activity and interactivity comprises user activity at the remote device and communication among two or more devices on the network; and

said receiving comprises one or more sensor clients at the control server receiving from the multiple sensors located within the network multiple different types of messages each respectively associated with a different type of sensor, wherein said receiving occurs via a transmission protocol; and

the method further comprises;

translating message data of each of the received multiple different types of message into an eGMIDS usable format;

forwarding the message data translated into the eGMIDS usable format to an eGMIDS sensor fusion mechanism for incorporation into the activity graph; and

generating and outputting the activity graph with events and alerts represented therein from the translated message data.

3. The method of claim 1, further comprising:

from the control server, activating one or more sensor(s) within one or more device(s) on the network, including the first sensor at the first remote device.

4. The method of claim 1, wherein the activity graph is generated for an enhanced graph matching intrusion detection system (eGMIDS) and said fusing of activity data is provided by an eGMIDS utility that executes to perform functions including:

receiving the specific transmission message in the specialized format;

retrieving data relevant to a specific activity being monitored by the sensor at a remote device from which the specific transmission message is received;

translating the received message data into an eGMIDS usable format;

mapping said data into the activity graph; and

performing signature-based intrusion detection, wherein a structure and a relationship between multiple events are utilized along with other information to distinguish threatening activity from benign activity.

5. The method of claim 1, further comprising:

enabling said first sensor to convert, via a provided eGMIDS host adapter associated with the first sensor, the sensed activity data at the first device into eGMIDS readable format for transmission to the control server;

wherein said first sensor is a specialized sensor configured for detecting sensed activity for packaging and forwarding to the control server;

providing to one or more remote sensors that are third-party sensors an eGMIDS sensor adapter, which translates an output from respective third-party sensors into an eGMIDS message format, wherein the eGMIDS sensor adapter is provided to translate the output from a particular third-party sensor; and

wherein the eGMIDS sensor adapter wraps and modifies the output received from the third party sensors and translates information about sensed activity into eGMIDS message format.

6. The method of claim 1, wherein:

each remote device on the network comprises one or more sensors from among multiple available sensors and a specific adapter designed for the associated sensor at the remote device; and

the method further comprises;

translating sensed activity data into sensor-specific event objects;

packaging the sensor-specific event objects into an eGMIDS message having eGMIDS message format; and

forwarding the packaged eGMIDS message to the control server;

wherein said one or more sensors comprise a Snort sensor, a Tripwire sensor, a traffic summary sensor, a keystroke sensor, an encrypted session sensor, and a host device fingerprinting sensor; and

wherein each sensor has a corresponding adapter configured to support eGMIDS message packaging functions of the specific sensor.

7. The method of claim 1, wherein said multiple sensors further comprise one or more of:

a keystroke sensor which monitors keystroke timing to distinguish among users;

a second sensor that determines whether or not a particular TCP session is transferring encrypted or plain text data, based on statistics of the information being transferred, wherein said information is utilized to understand a context of activities on the network; and

a clock skew fingerprinting sensor by which an individual computer is identified even if sending or receiving data under multiple IP address aliases, wherein an eGMIDS utility restructures the nodes and edges in the activity graph representing the individual computer to more accurately reflect this action.

8. A system for generating a graph representation of sensed activity data within a network, said system comprising:

a control server having a processor which executes a software utility which comprises functional components that complete the functions of;

receiving from a first sensor at a first remote device, a message containing remote device information including an identification of the remote device and activity occurring at the remote device, wherein said first sensor comprises an adapter configured to enable sensed activity occurring at the first remote device to be packaged in a specialized format for transmission to the control server, and wherein said first sensor encapsulates the sensed activity into a specific transmission message recognizable by receiving components at the control server and forwards the message to the control server;

wherein the multiple sensors comprise;

(a) an email sensor that tracks emails between users on the network and monitors a content for key words that may be associated with a threat when placed in proper context, given other activity of a sender and/or recipient of an email; and

(b) one or more of;

a chat log sensor that identifies an occurrence of an exchange of communication between users;

a traffic summary sensor that tracks a volume of traffic moving through each point in the network, as well as a source and destination of traffic data;

a keystroke sensor that distinguishes among users at a host device based on a timing of keystrokes of the users;

an encrypted session sensor that that determines whether or not a particular transmission control protocol (TCP) session is transferring encrypted or plain text data, based on statistics of the data being transferred to enable detection of encrypted sessions on ports that should be plain text and non-encrypted sessions on ports that should be encrypted, wherein information from the sensor is utilized as additional context to distinguish between threatening and benign activity; and

a host device fingerprinting sensor that allows an individual remote computer to be identified even if the computer is sending or receiving data under multiple internet protocol (IP) address aliases, wherein information received enables restructuring of nodes and edges representing that host to more accurately reflect the activity of the host and expose attackers that are trying to disguise their attacks by spreading the attacks across multiple source IP addresses;

a sensor data fusion utility executing within the control server and that configures the processor to;

fuse activity data retrieved from multiple ones of said message received from respective multiple sensors, including the first sensor and one or more other sensors located within the network, into an activity graph representative of the devices on the network and the activity and inter-activity occurring at and between the devices on the network,

translate data within an activity report generated from the received activity data into a graph representation and incorporating the translated data into a combined activity graph;

determine which elements within received activity reports are already represented by a node or edge within the activity graph in order to prevent duplication of a mapping within the activity graph of already represented elements; and

create a new node or edge for only those elements not already represented within the activity graph;

the control server generates a request for secondary evidence and transmits the request to the sensor of the remote device, wherein in response to receiving a request for secondary evidence at the sensor of the remote device from the control server, said sensor locates, packages and transmits the requested additional evidence to the control server; and

in response to receipt of the secondary evidence at the control server, the control server automatically translates the secondary evidence into eGMIDS usable format and fuses the secondary evidence into the activity graph.

9. The system of claim 8, said functional components further comprising:

a client adapter that generates the activity reports corresponding to activity data from a host device.

10. The system of claim 9, wherein:

the activity and interactivity comprises user activity at the remote device and communication among two or more devices on the network;

said functional components that perform the receiving comprises components for generating one or more sensor clients at the control server which receive the message via a 21Messaging transmission protocol;

said nodes represent hosts, users, files and events;

said edges represent communication, packet flow between hosts, event participation, file location and user account location; and

each node defines attributes possessed by the node and each edge defines attributes possessed by the edge, wherein said attributes are utilized to improve accuracy of pattern searches.

11. The system of claim 8, further comprising functional components that

activate from the control server one or more sensor(s) within one or more device(s) on the network, including a first sensor at the first remote device.

12. The system of claim 8, wherein the activity graph is generated for an enhanced graph matching intrusion detection system (eGMIDS) and said fusing of activity data is provided by an eGMIDS utility that executes to perform functions including:

receiving the specific transmission message in the specialized format;

retrieving data relevant to a specific activity being monitored by the sensor at a remote device from which the specific transmission message is received;

translating the received message data into an eGMIDS usable format;

mapping said data into the activity graph; and

performing signature-based intrusion detection, wherein a structure and a relationship between multiple events are utilized along with other information to distinguish threatening activity from benign activity.

13. The system of claim 12, further comprising functional components that:

enable said first sensor to convert, via a provided eGMIDS host adapter associated with the first sensor, the sensed activity data at the first device into eGMIDS readable format for transmission to the control server;

wherein said first sensor is a specialized sensor configured for detecting sensed activity for packaging and forwarding to the control server; and

provide to one or more remote sensors that are third-party sensors an eGMIDS sensor adapter, which translates an output from respective third-party sensors into an eGMIDS message format, wherein the eGMIDS sensor adapter is provided to translate the output from a particular third-party sensor;

wherein the eGMIDS sensor adapter wraps and modifies the output received from the third party sensors and translates information about sensed activity into eGMIDS message format.

14. The system of claim 12, wherein;

each remote device on the network comprises one or more sensors from among multiple available sensors and the specific adapter designed for the associated sensor at the remote device; and

the system comprises functional components that;

translate sensed activity data into sensor-specific event objects;

package the sensor-specific event objects into an eGMIDS message having eGMIDS message format; and

forward the packaged eGMIDS message to the IP address of the control server;

wherein said one or more sensors comprise a Snort sensor, a Tripwire sensor, a traffic summary sensor, an email sensor, a keystroke sensor, an encrypted session sensor, and a host device fingerprinting sensor; and

wherein each sensor has a corresponding adapter configured to support eGMIDS message packaging functions of the specific sensor.

15. The system of claim 8, wherein said sensors comprise a clock skew fingerprinting sensor by which an individual computer is identified even if sending or receiving data under multiple IP address aliases, wherein an eGMIDS utility restructures the nodes and edges in the activity graph representing the individual computer to more accurately reflect this action.

16. The system of claim 8, wherein said utility comprises code that configures the processor to:

receive from the multiple sensors located within the network multiple different types of messages each respectively associated with a different type of sensor;

translate message data of each of the received multiple different types of message into an eGMIDS usable format;

forward the data to an eGMIDS sensor fusion mechanism for incorporation into the activity graph; and

generate and output the activity graph with events and alerts represented therein from the translated message data.

17. A computer program product comprising:

a non-transitory recordable type medium; and

program code on the recordable type medium for generating a graph representation of sensed activity data within a network, said program code comprising a software utility executing at a control server and which comprises functional components for completing the functions of;

at the control server, receiving from a first sensor at a first remote device, a message containing remote device information including an identification of the remote device and activity occurring at the remote device, wherein said first sensor comprises an adapter configured to enable sensed activity occurring at the first remote device to be packaged in a specialized format for transmission to the control server, and wherein said first sensor encapsulates the sensed activity into a specific transmission message recognizable by receiving components at the control server and forwards the message to the control server; and

fusing activity data retrieved from multiple ones of said message from multiple sensors, including the first sensor, located within the network, into an activity graph representative of the devices on the network and the activity and inter-activity occurring at and between the devices on the network;

translating data within an activity report generated from the received activity data into a graph representation and incorporating the translated data into a combined activity graph;

determining which elements within received activity reports are already represented by a node or edge within the activity graph in order to prevent duplication of a mapping within the activity graph of already represented elements;

creating a new node or edge for only those elements not already represented within the activity graph;

generating a request for secondary evidence and transmitting the request to the sensor of the remote device, wherein in response to receipt of a request for secondary evidence at the sensor of the remote device from the control server, said sensor is triggered to locate, package and transmit the requested additional evidence to the control server; and

in response to receipt of the secondary evidence at the control server, automatically translating the secondary evidence into eGMIDS usable format and fusing the secondary evidence into the activity graph;

wherein said multiple sensors comprise an email sensor, which completes the functions of;

tracking emails between users on a network; and

monitoring an exchange of emails within a context, which context includes the sender'"'"'s and recipient'"'"'s other activities in addition to the exchange of emails on the network, said other activities being pre-determined to trigger said monitoring and which occur prior to or concurrent with the exchange of emails to trigger said monitoring, and wherein said monitoring includes monitoring a content of the email for key words that may be associated with a threat when placed in proper context, given the other activity of the sender and/or recipient of the email.

18. The computer program product of claim 17, said functional components further comprising:

a client adapter that generates activity reports corresponding to activity data from a host device.

19. The computer program product of claim 18, wherein:

the activity and interactivity comprises user activity at the remote device and communication among two or more devices on the network;

said program code for receiving comprises program code for generating one or more sensor clients at the control server which receive the message via a 21Messaging transmission protocol;

said nodes represent hosts, users, files and events;

said edges represent communication, packet flow between hosts, event participation, file location and user account location; and

each node defines attributes possessed by the node and each edge defines attributes possessed by the edge, wherein said attributes are utilized to improve accuracy of pattern searches.

20. The computer program product of claim 17, further comprising:

program code for activating, from the control server, one or more sensor(s) within one or more device(s) on the network, including the first sensor at the first remote device.

21. The computer program product of claim 17, wherein the activity graph is generated for an enhanced graph matching intrusion detection computer program product (eGMIDS) and said fusing of activity data is provided by an eGMIDS utility that executes to perform functions including:

receiving the specific transmission message in the specialized format;

retrieving data relevant to a specific activity being monitored by the sensor at a remote device from which the specific transmission message is received;

translating the received message data into an eGMIDS usable format;

mapping said data into the activity graph; and

performing signature-based intrusion detection, wherein a structure and a relationship between multiple events are utilized along with other information to distinguish threatening activity from benign activity.

22. The computer program product of claim 21, further comprising:

program code for enabling said first sensor to convert, via a provided eGMIDS host adapter associated with the first sensor, the sensed activity data at the first device into eGMIDS readable format for transmission to the control server;

wherein said first sensor is a specialized sensor configured for detecting sensed activity for packaging and forwarding to the control server; and

program code for providing to one or more remote sensors that are third-party sensors an eGMIDS sensor adapter, which translates an output from respective third-party sensors into an eGMIDS message format, wherein the eGMIDS sensor adapter is provided to translate the output from a particular third-party sensor;

wherein the eGMIDS sensor adapter wraps and modifies the output received from the third party sensors and translates information about sensed activity into eGMIDS message format.

23. The computer program product of claim 21, wherein:

each remote device on the network comprises one or more sensors from among multiple available sensors and the specific adapter designed for the associated sensor at the remote device; and

the computer program product further comprises code for;

translating sensed activity data into sensor-specific event objects;

packaging the sensor-specific event objects into an eGMIDS message having eGMIDS message format; and

forwarding the packaged eGMIDS message to the IP address of the control server;

wherein said one or more sensors comprise;

a Snort sensor;

a Tripwire sensor;

a traffic summary sensor that tracks a volume of traffic moving through each point in the network, as well as a source and destination of traffic data;

a keystroke sensor that distinguishes among users at a host device based on a timing of keystrokes of the users;

a chat log sensor that identifies an occurrence of an exchange of communication between users;

an encrypted session sensor that determines whether or not a particular transmission control protocol (TCP) session is transferring encrypted or plain text data, based on statistics of the data being transferred to enable detection of encrypted sessions on ports that should be plain text and non-encrypted sessions on ports that should be encrypted, wherein information from the sensor is utilized as additional context to distinguish between threatening and benign activity; and

a host device fingerprinting sensor that allows an individual remote computer to be identified even if the computer is sending or receiving data under multiple internet protocol (IP) address aliases, wherein information received enables restructuring of nodes and edges representing that host to more accurately reflect the activity of the host and expose attackers that are trying to disguise their attacks by spreading the attacks across multiple source IP addresses; and

wherein each sensor has a corresponding adapter configured to support eGMIDS message packaging functions of the specific sensor.

24. The computer program product of claim 17, wherein said sensors comprise a keystroke sensor which monitors keystroke timing to distinguish users.

25. The computer program product of claim 17, wherein said utility comprises code for completing the following functions:

receiving from the multiple sensors located within the network multiple different types of messages each respectively associated with a different type of sensor;

translating message data of each of the received multiple different types of message into an eGMIDS usable format;

forwarding the data to an eGMIDS sensor fusion mechanism for incorporation into the activity graph; and

generating and outputting the activity graph events and alerts represented therein from the translated message data.

26. A method for generating a network activity graph comprising:

at a control server, receiving from a first sensor at a first remote device, a message containing remote device information including an identification of the remote device and activity occurring at the remote device, wherein said first sensor comprises an adapter configured to enable sensed activity occurring at the first remote device to be packaged in a specialized format for transmission to the control server, and wherein said first sensor encapsulates the sensed activity into a specific transmission message recognizable by receiving components at the control server and forwards the message to the control server; and

the control server fusing activity data retrieved from multiple ones of said message from multiple sensors, including the first sensor, located within the network into an activity graph representative of the devices on the network and the activity and inter-activity occurring at and between the devices on the network;

the control server translating data within an activity report generated from the received activity data into a graph representation and incorporating the translated data into a combined activity graph;

the control server determining which elements within received activity reports are already represented by a node or edge within the activity graph in order to prevent duplication of a mapping within the activity graph of already represented elements;

the control server creating a new node or edge for only those elements not already represented within the activity graph;

the control server generating a request for secondary evidence and transmitting the request to the sensor of the remote device, wherein in response to receipt of a request for secondary evidence at the sensor of the remote device from the control server, said sensor is triggered to locate, package and transmit the requested additional evidence to the control server; and

in response to receipt of the secondary evidence at the control server, the control server automatically translating the secondary evidence into eGMIDS usable format and fusing the secondary evidence into the activity graph;

wherein the multiple sensors comprise at least two sensors from among;

a Snort sensor, a Tripwire sensor, a traffic summary sensor, a keystroke sensor, an encrypted session sensor, and a host device fingerprinting sensor.

27. The method of claim 26, wherein said multiple sensors comprise a traffic summary sensor that tracks a volume of traffic moving through each point in the network, as well as a source and destination of the traffic data.

28. The method of claim 26, wherein said multiple sensors comprise a keystroke sensor that distinguishes among users at a host based on a timing of keystrokes of the users.

29. The method of claim 26, wherein said multiple sensors comprise an encrypted session sensor that determines whether or not a particular transmission control protocol (TCP) session is transferring encrypted or plain text data, based on statistics of the data being transferred to enable detection of encrypted sessions on ports that should be plain text and non-encrypted sessions on ports that should be encrypted, wherein information from the sensor is utilized as additional context to distinguish between threatening and benign activity.

30. The method of claim 26, wherein multiple said sensors comprise a host device fingerprinting sensor that allows an individual remote computer to be identified even if the computer is sending or receiving data under multiple internet protocol (IP) address aliases, wherein information received enables restructuring of nodes and edges representing that host to more accurately reflect the activity of the host and expose attackers that are trying to disguise their attacks by spreading the attacks across multiple source IP addresses.

31. The method of claim 26, wherein the multiple sensors comprise a chat log sensor that identifies an occurrence of an exchange of communication between users.