Method for cooperative intrusion prevention through collaborative inference

• US 8,230,505 B1
• Filed: 08/11/2006
• Issued: 07/24/2012
• Est. Priority Date: 08/11/2006
• Status: Active Grant

• Current Assignee: Avaya Incorporated
• Original Assignee: Avaya Incorporated
• Inventors: Ahrens, David, Mani, Mahalingam
• Primary Examiner(s): Arani, Taghi
• Assistant Examiner(s): Rahman, Mahfuzur
• Application Number:
  US11/502,884
• Time in Patent Office: 2,174 Days
• Field of Search: 726/23
• US Class Current: 726/23
• CPC Class Codes: G06Q 10/02 Reservations, e.g. for tickets, services or events
• CPC Class Codes: G06Q 10/06315 Needs-based resource requirements planning or analysis
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/168 above the transport layer

Abstract

The present invention is directed to an intrusion detection/prevention system that uses application layer event information to identify potential intrusions and notifies remote trusted peers in other enterprise networks of potential intrusions emanating therefrom.

Claims

1. An intrusion prevention method executing in an enterprise network, the method comprising:

providing a media server, the media server operable to direct communication contacts between two or more communication devices, wherein the media server comprises a processor, a memory, and an application behavior anomaly detector, wherein;

the application behavior anomaly detector receiving application level information from a communication application, wherein the communication application executes a communications-related task;

the application behavior anomaly detector selecting at least one event, at the application layer, from the application level information, the selected event being at least one of an error, exception, policy violation, and handling rate;

the application behavior anomaly detector detecting a potential attack;

the application behavior anomaly detector sending application-level attack information to an enterprise behavioral anomaly system;

providing an enterprise behavioral anomaly system in communication with the media server, which comprises a processor and memory, executing an interface, anomaly source location engine, and an anomaly prevention engine, wherein;

the interface receiving the application-level attack information;

the anomaly source location engine locating, with the application-level attack information, an access-level source for the potential attack;

when the application-level attack information is indicative of the intrusion, locating at least one of an entry point and an address of a source of the intrusion; and

when the application-level attack information is indicative of an intrusion, the anomaly prevention engine implementing an appropriate response to the intrusion, wherein implementing the appropriate response comprises blocking or neutralizing a particular transport address.

2. The method of claim 1, wherein the application layer event is at least one of an error and exception and wherein the at least one of an error and exception is associated with at least one of the following:

a syntax, a string length, session type, session length, stack state, buffer state, memory state, input/output state, application semantic, parameter processing, protocol, queue state, handling state, packet traffic pattern, and packet traffic volume.

3. The method of claim 1, wherein the application layer event is a policy violation and wherein the policy violation is associated with at least one of the following:

login privilege, login attempt, use privilege, informational access privilege, and right to use restriction.

4. The method of claim 1, wherein the application layer event is a handling rate and wherein the handling rate is associated with at least one of a rate of receipt of a type of signal, a half open connection rate, a port scan rate, a network scan rate, and a Central Processor Unit or CPU usage rate.

5. The method of claim 1, wherein locating, with the application-level attack information, an access-level source for the potential attack comprises:

comparing the selected at least one event against a first signature file, the first signature file comprising at least one of intrusion signatures and rules indicative of specific intrusion types; and

wherein, the application-level attack information comprises a plurality of source information associated with the intruder, an address of the source making the intrusion, an address of the destination of the intrusion, a type of access of the intrusion, a type of violation, an identifier and/or type of a port used by the intrusion packet, a number of intrusion attempts, an indicator of the likelihood of an intrusion in fact occurring, and an indicator of the significance of the intrusion.

6. The method of claim 5, wherein locating, with the application-level attack information, an access-level source for the potential attack further comprises:

determining whether the selected event corresponds to at least one other event performed at a lower layer of the Open Systems Interconnection Model;

when the selected event corresponds to at least one other event, associating the selected event and at least one other event;

comparing the selected at least one event against a second signature file, the second signature file comprising at least one of intrusion signatures and rules indicative of specific intrusion types and an appropriate response for each type of intrusion; and

implementing an appropriate response to the intrusion comprises, based on the comparison step (h3), implementing the appropriate response to the type of intrusion corresponding to the selected event.

7. The method of claim 1, wherein detecting a potential attack comprises:

determining that the selected application layer event is indicative of an intrusion;

determining an address of a source of the intrusion;

when the address of the source of the intrusion is associated with a second enterprise network, determining whether the first enterprise network is in a federated relationship with the second enterprise network; and

when the first enterprise network is in a federated relationship with the second enterprise network and when the address of the intrusion source is associated with the second enterprise network, providing a trusted peer in the second enterprise network with information describing the intrusion.

8. The method of claim 7, wherein the trusted peer is provided with the information describing the intrusion via at least one of a control and data channel established using a voice and/or text communication protocol.

9. An enterprise network, comprising:

a plurality of communication applications operable to execute communications-related tasks;

a media server operable to direct communication contacts between two or more communication devices, the media server comprising;

a memory;

a processor in communication with the memory, the processor operable to execute;

an application behavior anomaly detecting agent operable to;

select at least one event at the application layer, the selected at least one event being at least one of an error, exception, policy violation, and handling rate identified by one or more of the plurality of communication applications;

when a selected application layer event is indicative of potential anomalous behavior, provide metadata about the selected application layer event to an enterprise behavioral anomaly system;

the enterprise behavioral anomaly system in communication with the media server, the enterprise behavioral system comprising;

a second memory;

a second processor in communication with the second memory, the second processor operable to;

examine the metadata to determine whether the metadata corresponds to an intrusion;

combine the metadata with second information about a second event, wherein the metadata and the second information are associated with a similar intrusion;

when the selected application layer is indicative of the intrusion, locating at least one of an entry point and an address of a source of the intrusion; and

determine an appropriate response to the similar intrusion; and

implement the appropriate response to the intrusion, wherein

operation to implement the appropriate response to the intrusion comprises blocking or neutralizing a particular transport address.

10. The media server of claim 9, wherein the event is at least one of an error and exception and wherein the at least one of an error and exception is associated with at least one of the following:

a syntax, a string length, session type, session length, stack state, buffer state, memory state, input/output state, application semantic, parameter processing, protocol, queue state, handling state, packet traffic pattern, and packet traffic volume.

11. The media server of claim 9, wherein the event is a policy violation and wherein the policy violation is associated with at least one of the following:

login privilege, login attempt, use privilege, informational access privilege, and right to use restriction.

12. The media server of claim 9, wherein the event is a handling rate and wherein the handling rate is associated with at least one of a rate of receipt of a type of signal, a half open connection rate, a port scan rate, a network scan rate, and a Central Processor Unit or CPU usage rate.

13. The media server of claim 9, wherein:

the operation to select at least one event at the application layer comprises comparing the selected at least one event against a first signature file, the first signature file comprising at least one of intrusion signatures and rules indicative of specific intrusion types; and

the operation to provide metadata about the selected application layer event to an enterprise behavioral anomaly system comprises collecting the metadata, wherein the metadata comprises a plurality of source information associated with the intruder, an address of the source making the intrusion, an address of the destination of the intrusion, a type of access of the intrusion, a type of violation, an identifier and/or type of a port used by the intrusion packet, a number of intrusion attempts, an indicator of the likelihood of an intrusion in fact occurring, and an indicator of the significance of the intrusion.

14. The media server of claim 13, wherein:

the operation to examine the metadata to determine whether the metadata corresponds to an intrusion further comprises the suboperations;

determining whether the selected event corresponds to at least one other event performed at a lower layer of the Open Systems Interconnection Model;

when the selected event corresponds to the second event, associating the selected event and the second event;

the operation to combine the metadata with second information about a second event comprises comparing the combined metadata and the second information against a second signature file, the second signature file comprising at least one of intrusion signatures, rules indicative of specific intrusion types and an appropriate response for each type of intrusion; and

the operation to implement the appropriate response to the intrusion comprises, based on the comparison in the operation to combine the metadata with second information about a second event, implementing the appropriate response to the type of similar intrusion corresponding to the selected event and the second event.

15. The media server of claim 9, wherein the application behavior anomaly detecting agent is in a first enterprise network;

wherein the comparison the operation to examine the metadata to determine whether the metadata corresponds to an intrusion comprises the suboperations of;

determining that the selected application layer event is indicative of an intrusion;

determining an address of a source of the intrusion;

when the address of the source of the intrusion is associated with a second enterprise network, determining whether the first enterprise network is in a federated relationship with the second enterprise network; and

wherein the operation to implement the appropriate response to the intrusion comprises, when the first enterprise network is in a federated relationship with the second enterprise network and when the address of the intrusion source is associated with the second enterprise network, providing a trusted peer in the second enterprise network with information describing the intrusion.

16. The media server of claim 15, wherein the trusted peer is provided with the metadata describing the intrusion via at least one of a control and data channel established using a voice and/or text communication protocol.

17. An enterprise network, comprising:

a plurality of communication applications operable to execute communications-related tasks;

a media server operable to direct communication contacts between two or more communication devices, the media server comprising;

a memory;

a processor in communication with the memory, the processor operable to execute;

an application behavior anomaly detecting agent operable to;

select at least one event at the application layer, the selected at least one event being at least one of an error, exception, policy violation, and handling rate identified by one or more of the plurality of communication applications;

when a selected application layer event is indicative of potential anomalous behavior, provide metadata about the selected application layer event to an enterprise behavioral anomaly system;

the enterprise behavioral anomaly system in communication with the media server, the enterprise behavioral system comprising;

a second memory;

a second processor in communication with the second memory, the second processor operable to;

examine the metadata to determine whether the metadata corresponds to an intrusion;

when the selected application layer is indicative of the intrusion, locating at least one of an entry point and an address of a source of the intrusion; and

determine an appropriate response to the intrusion;

providing two or more trusted peers with information about the intrusion; and

implement the appropriate response to the intrusion, wherein

operation to implement the appropriate response to the intrusion comprises blocking or neutralizing a particular transport address.

18. The media server of claim 17, wherein providing the two or more trusted peers with the information about the instruction further comprises:

determining an address of a source of the intrusion;

when the address of the source of the intrusion is associated with a second enterprise network, determining whether the first enterprise network is in a federated relationship with the second enterprise network; and

when the first enterprise network is in a federated relationship with the second enterprise network and when the address of the intrusion source is associated with the second enterprise network, providing the trusted peer in the second enterprise network with the information describing the intrusion.