System and method for preventing network misuse

• US 7,237,264 B1
• Filed: 06/04/2001
• Issued: 06/26/2007
• Est. Priority Date: 06/04/2001
• Status: Active Grant

• Current Assignee: Kyndryl Incorporated
• Original Assignee: Internet Security Systems Incorporated (International Business Machines Corporation)
• Inventors: Graham, Robert David, Kavaler, Peter
• Primary Examiner(s): Zand; Kambiz
• Assistant Examiner(s): Nalven; Andrew L.
• Application Number:
  US09/874,574
• Time in Patent Office: 2,213 Days
• Field of Search: 709/225, 709/223, 713/201, 713/200, 726/25, 726/23
• US Class Current: 726/23
• CPC Class Codes: H04L 63/0227 Filtering policies mail message filtering H04L51/212
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection

Abstract

A system and method for preventing misuse conditions on a data network are described. Embodiments of the system and method evaluate potential network misuse signatures by analyzing variables such as the state of the network and/or target, the context in which the potential misuse signatures are detected, the response/reaction of the target and/or the fingerprint of the target. These and other variables may be factored in to the misuse determination, either alone, or in combination.

Claims

1. A computer-implemented method comprising:

identifying a plurality of data signatures relevant to computer security;

designating an alert condition value to each data signature based on each data signature itself and contextual information associated with the data signature, each alert condition value comprising a ranked value that is unique to each combination of data signature and contextual information associated with a particular data signature, the contextual information comprising at least one of an application layer data field type used to encapsulate the data signature and an application layer protocol type used to transmit the data signature, the alert condition value indicating a security risk level relative to different data signatures and relative to other identical data signatures associated with different contextual information;

creating a table comprising the contextual information, the data signatures, and the alert condition values;

detecting a data signature by evaluating communications at an application layer level between a target and a suspect;

correlating said data signature with an application layer fingerprint of the target to determine to what extent said target is vulnerable to said data signature;

evaluating contextual information related to the data signature by comparing the contextual information and the data signature to the table in order to determine a likelihood that said target is under attack; and

assigning an alert condition value to the data signature based on the comparison of the contextual information and data signature to data in the table.

2. The method as in claim 1 wherein said fingerprint includes said target node'"'"'s operating system version.

3. The method as in claim 1 wherein said fingerprint includes said target node'"'"'s processor type.

4. The method as in claim 1 further comprising:

generating a first alert condition upon determining that said target node is vulnerable to said data signature.

5. The method as in claim 1 further comprising:

listening for a response to said data signature from said target.

6. The method as in claim 5 further comprising:

determining whether said target node'"'"'s response or lack of a response is suspicious.

7. The method as in claim 6 wherein determining whether said target'"'"'s response is suspicious comprises determining whether said target'"'"'s response is an “

unknown command”

response.

8. The method as in claim 6 further comprising:

generating a second alert condition upon determining that said target node'"'"'s response or lack of a response is suspicious.

9. The method as in claim 8 further comprising:

combining the second alert with the first, thereby updating the first alert with information within the second alert.

10. The method as in claim 9 wherein said target node'"'"'s suspicious behavior comprises transmitting a root shell prompt to a suspect node.

11. The method as in claim 1 further comprising:

listening for behavior of said target node; and

generating a second alert condition upon determining that said target node'"'"'s behavior is suspicious.

12. A computer-implemented method comprising:

identifying a plurality of data signatures relevant to computer security;

designating an alert condition value to each data signature based on each data signature itself and contextual information associated with the data signature, each alert condition value comprising a ranked value that is unique to each combination of data signature and contextual information associated with a particular data signature, the contextual information comprising at least one of an application layer data field type used to encapsulate the data signature and an application layer protocol type used to transmit the data signature, the alert condition value indicating a security risk level relative to different data signatures and relative to other identical data signatures associated with different contextual information;

creating a table comprising the data signatures, contextual information, and alert condition values;

identifying a data signature encapsulated in an application layer data field and directed at a target using an application layer protocol;

evaluating a context of the data signature by one of;

reviewing the application layer data field type;

reviewing the application layer protocol type;

comparing the evaluated context of the data signature to the table;

determining whether said data signature poses a threat based on said context of said data signature; and

assigning an alert condition value to the data signature based on the comparison of the context to data in the table.

13. The method as in claim 12 wherein said protocol is the HyperText Transport Protocol (“

HTTP”

).

14. The method as in claim 13 further comprising:

determining that said data signature poses a threat if said data signature is “

/cgi-bin/phl”

embedded in the header of said HTTP data transmission.

15. The method as in claim 12 further comprising

evaluating whether said data signature poses a threat based on a fingerprint of said target.

16. The method as in claim 15 wherein said fingerprint is comprised of a particular service executed on said target.

17. The method as in claim 15 wherein said fingerprint is comprised of a particular operating system executed on said target.

18. The method as in claim 15 wherein said fingerprint is comprised of a particular hardware platform of said target.

19. The method as in claim 12 further comprising:

monitoring responses from said target following said data signature; and

determining a likelihood of whether said target is under attack based on data signatures of said responses.

20. The method as in claim 19 wherein said target response is a non-protocol response.

21. The method as in claim 20 wherein said data signature is transmitted to the target using the file transfer protocol (“

FTP”

) and said non-protocol response indicates a raw shell connection to said target.

22. A computer-implemented method comprising:

identifying a plurality of data signatures relevant to computer security;

designating a relative alert condition value to each data signature based on each data signature itself and contextual information associated with the data signature, each alert condition value comprising a ranked value that is unique to each combination of data signature and contextual information associated with a particular data signature, the contextual information comprising at least one of an application layer data field type used to encapsulate the data signature and an application layer protocol type used to transmit the data signature, the alert condition value indicating a security risk level relative to different data signatures and relative to other identical data signatures associated with different contextual information;

creating a table comprising the contextual information, the data signatures, and the relative alert condition values;

monitoring a plurality of data transmissions at an applications layer level between a suspect and a target to identify one or more data signatures, said data transmissions indicating a current state of communication between said suspect and said target;

evaluating contextual information related to each data signature by comparing the contextual information and data signatures to the table;

evaluating a likelihood that said target is under attack based on the contextual information of one or more data signatures of said transmissions and said current state of communication; and

assigning a relative alert condition value to the data signature based on the comparison of the contextual information to data in the table.

23. The method as in claim 22 wherein said current state of communication is based on a known protocol with which said data transmissions are transmitted/received between said suspect and target.

24. The method as in claim 23 wherein said known protocol is FTP.

25. The method as in claim 24 wherein one of said data signatures is the filename “

passwd”

in a context in which filenames are likely to appear.

26. The method as in claim 23 wherein said known protocol is HTTP.

27. The method as in claim 23 wherein said known protocol is RPC.

28. The method as in claim 22 further comprising:

monitoring responses from said target following said data signature; and

determining a likelihood of whether said target is under attack based on data signatures of said responses.

29. The method as in claim 22 wherein said current state comprises any outbound connection from said target is following a detected signature.

30. The method as in claim 22 wherein said current state comprises an inbound connection to a new port following a detected signature.

31. The method as in claim 22 monitoring said current state comprises:

profiling said target to determine which ports are open by passively listening to what traffic succeeds in talking to/from the target.

32. The method as in claim 22 monitoring said current state comprises:

detecting non-protocol requests or responses transmitted to/from said target.

33. The method as in claim 22 further comprising:

determining a fingerprint of said target; and

further evaluating a likelihood that said target is under attack based on said fingerprint.

34. A machine-readable physical medium having program code stored thereon which, when executed by a machine, causes said machine to perform the operations of:

identifying a plurality of data signatures relevant to computer security;

designating a relative alert condition value to each data signature based on each data signature itself and contextual information associated with the data signature, each alert condition value comprising a ranked value that is unique to each combination of data signature and contextual information associated with a particular data signature, the contextual information comprising at least one of an application layer data field type used to encapsulate the data signature and an application layer protocol type used to transmit the data signature, the relative alert condition value indicating a security risk level relative to different data signatures and relative to other identical data signatures associated with different contextual information;

creating a table comprising the contextual information, the data signatures, and the relative alert condition values;

detecting a data signature by evaluating communications at an application layer level between a target and a suspect;

correlating said data signature with a fingerprint of the target to determine to what extent said target is vulnerable to said data signature; and

evaluating contextual information related to the data signature by comparing the contextual information and the data signature to the table in order to determine a likelihood that said target is under attack; and

assigning a relative alert condition value to the data signature based on the comparison of the contextual information and data signature to data in the table.

35. The machine-readable medium as in claim 34 further comprising program code to cause said machine to perform the operations of:

evaluating contextual information related to said data signature to determine a likelihood that said target is under attack.

36. The machine-readable medium as in claim 34 wherein said fingerprint includes said target node'"'"'s operating system version.

37. The machine-readable medium as in claim 34 wherein said fingerprint includes said target node'"'"'s processor type.

38. The machine-readable medium as in claim 34 further comprising program code to cause said machine to perform the operations of:

generating a first alert condition upon determining that said target node is vulnerable to said data signature.

39. The machine-readable medium as in claim 34 further comprising program code to cause said machine to perform the operations of:

listening for a response to said data signature from said target.

40. The machine-readable medium as in claim 39 further comprising program code to cause said machine to perform the operations of:

determining whether said target node'"'"'s response or lack of a response is suspicious.

41. The machine-readable medium as in claim 40 wherein determining whether said target'"'"'s response is suspicious comprises determining whether said target'"'"'s response is an “

unknown command”

response.

42. The machine-readable medium as in claim 40 further comprising program code to cause said machine to perform the operations of:

generating a second alert condition upon determining that said target node'"'"'s response or lack of a response is suspicious.

43. The machine-readable medium as in claim 42 further comprising program code to cause said machine to perform the operations of:

combining the second alert with the first, thereby updating the first alert with information within the second alert.

44. The machine-readable medium as in claim 43 wherein said target node'"'"'s suspicious behavior comprises transmitting a root shell prompt to a suspect node.

45. The machine-readable medium as in claim 34 further comprising program code to cause said machine to perform the operations of:

listening for behavior of said target node; and

generating a second alert condition upon determining that said target node'"'"'s behavior is suspicious.

46. A machine-readable physical medium having program code stored thereon which, when executed by a machine, causes said machine to perform the operations of:

identifying a plurality of data signatures relevant to computer security;

designating an alert condition value to each data signature based on each data signature itself and contextual information associated with the data signature, each alert condition value comprising a ranked value that is unique to each combination of data signature and contextual information associated with a particular data signature, the contextual information comprising at least one of an application layer data field type used to encapsulate the data signature and an application layer protocol type used to transmit the data signature, the alert condition value indicating a security risk level relative to different data signatures and relative to other identical data signatures associated with different contextual information;

creating a table comprising the data signatures, the contextual information, and the alert condition values;

identifying a data signature encapsulated in an application layer data field directed at a target using an application layer protocol;

evaluating a context of the data signature by one of;

reviewing the application layer data field type;

reviewing the application layer protocol type; and

comparing the evaluated context of the data signature to the table;

determining whether said data signature poses a threat based on said context of said data signature; and

assigning an alert condition value to the data signature based on the comparison of the context to data in the table.

47. The machine-readable medium as in claim 46 wherein said protocol is the HyperText Transport Protocol (“

HTTP”

).

48. The machine-readable medium as in claim 47 further comprising program code to cause said machine to perform the operations of:

determining that said data signature poses a threat if said data signature is “

Icgi-bin/phf”

embedded in the header of said HTTP data transmission.

49. The machine-readable medium as in claim 46 further comprising program code to cause said machine to perform the operations of:

further evaluating whether said data signature poses a threat based on a fingerprint of said target.

50. The machine-readable medium as in claim 49 wherein said fingerprint is comprised of a particular service executed on said target.

51. A machine-readable physical medium having program code stored thereon which, when executed by a machine, causes said machine to perform the operations of:

identifying a plurality of data signatures relevant to computer security;

designating a relative alert condition value to each data signature based on each data signature itself and contextual information associated with the data signature, each alert condition value comprising a ranked value that is unique to each combination of data signature and contextual information associated with a particular data signature, the contextual information comprising at least one of an application layer data field type used to encapsulate the data signature and an application layer protocol type used to transmit the data signature, the relative alert condition value indicating a security risk level relative to different data signatures and relative to other identical data signatures associated with different contextual information;

creating a table comprising the contextual information, the data signatures, and the relative alert condition values;

monitoring a plurality of data transmissions at an applications layer level between a suspect and a target to identify one or more data signatures, said data transmissions indicating a current state of communication between said suspect and said target;

evaluating contextual information related to each data signature by comparing the contextual information and data signatures to the table;

evaluating a likelihood that said target is under attack based on the contextual information of one or more data signatures of said transmissions and said current state of communication; and

assigning a relative alert condition value to the data signature based on the comparison of the contextual information to data in the table.

52. The machine-readable medium as in claim 51 comprising program code to cause said machine to perform the additional operations of:

monitoring responses from said target following said data signature; and

determining a likelihood of whether said target is under attack based on data signatures of said responses.