Systems and Methods for Correlating and Distributing Intrusion Alert Information Among Collaborating Computer Systems

• US 20100281541A1
• Filed: 07/09/2010
• Published: 11/04/2010
• Est. Priority Date: 05/11/2004
• Status: Active Grant

• Current Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Original Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Inventors: Locasto, Michael, Keromytis, Angelos D., Stolfo, Salvatore J., Parekh, Janak, Misra, Vishal, Malkin, Tal
• Granted Patent: US 8,381,295 B2
• Time in Patent Office: Days
• US Class Current: 726/23
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: G06F 21/554 involving event detection and direct action
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/145 the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms
• CPC Class Codes: H04L 63/1475 Passive attacks, e.g. eavesdropping or listening without modification of the traffic monitored
• CPC Class Codes: H04L 63/1491 using deception as countermeasure, e.g. honeypots, honeynets, decoys or entrapment

Abstract

Systems and methods for correlating and distributing intrusion alert information among collaborating computer systems are provided. These systems and methods provide an alert correlator and an alert distributor that enable early signs of an attack to be detected and rapidly disseminated to collaborating systems. The alert correlator utilizes data structures to correlate alert detections and provide a mechanism through which threat information can be revealed to other collaborating systems. The alert distributor uses an efficient technique to group collaborating systems and then pass data between certain members of those groups according to a schedule. In this way data can be routinely distributed without generating excess traffic loads.

Claims

1. A method of detecting a threat to a computer system in a plurality of collaborating computer systems, the method comprising:

receiving, at a first computer system, a first one-way data structure from a collaborating second computer system, the first one-way data structure representing first data relating to a first intrusion attempt detected by an intrusion detection system of the collaborating second computer system such that the first data is hidden in the first one-way data structure;

detecting, using an intrusion detection system of the first computer system, a second intrusion attempt;

storing second data relating to the second intrusion attempt in a second one-way data structure of the first computer system such that the second data is hidden in the second one-way data structure;

determining whether the second intrusion attempt correlates with the first intrusion attempt by comparing the first data structure and the second data structure; and

indicating that a threat is present if the second intrusion attempt is determined to correlate with the data received from the collaborating second computer system relating to the first intrusion attempt.

2. The method of claim 1, wherein the first data forms at least part of a profile of the first intrusion attempt.

3. The method of claim 2, further comprising responding to the threat based on the profile of the first intrusion attempt.

4. The method of claim 1, wherein receiving the first one-way data structure from the collaborating second computer system comprises periodically exchanging one-way data structures with the collaborating second computer system.

5. The method of claim 4, wherein the one-way data structures are exchanged through at least one proxy.

6. The method of claim 4, further comprising:

grouping the collaborating second computer system and other collaborating systems into groups so that each collaborating system in a group occupies a position in that group;

exchanging one-way data structures between collaborating systems in the same position in the different groups; and

rotating the position occupied by each member of at least one of the groups according to a schedule.

7. The method of claim 6, further comprising changing the schedule.

8. The method of claim 1, wherein the second intrusion attempt is one of a scan and probe.

9. The method of claim 1, wherein the second intrusion attempt is a blocked worm.

10. The method of claim 1, wherein the intrusion detection system is a honey pot.

11. The method of claim 1, wherein the first one-way data structure is a bloom filter.

12. The method of claim 11, wherein storing data relating to the second intrusion attempt in the second one-way data structure of the first computer system comprises:

forming a hash of the second data relating to the second intrusion attempt;

using the hash as an index to the bloom filter; and

setting corresponding bits in the bloom filter based on the index.

13. The method of claim 12, wherein forming the hash comprises using at least one of the SHA-1 and the MD-5 hashing algorithms.

14. The method of claim 11, wherein storing data relating to the second intrusion attempt in the second one-way data structure of the first computer system comprises:

forming a hash of the second data relating to the second intrusion attempt;

selecting at least one portion of the hash;

using the at least one portion of the hash as an index to the bloom filter; and

setting corresponding bits in the bloom filter based on the index.

15. The method of claim 14, wherein forming the hash comprises using at least one of the SHA-1 and the MD-5 hashing algorithms.

16. The method of claim 1, wherein indicating that the threat is present comprises outputting an IP address of the source of the threat.

17. The method of claim 1, wherein indicating that the threat is present comprises causing a firewall to block traffic from an IP address of the source of the threat.

18. The method of claim 1, wherein indicating that the threat is present comprises outputting a signature associated with the threat.

19. The method of claim 1, wherein determining whether the second intrusion attempt correlates with the first intrusion attempt comprises:

comparing bits of the first one-way data structure to corresponding bits of the second one-way data structure; and

indicating a correlation if at least a predetermined number of corresponding bits are set to the same bit value.

20. A method of detecting a threat to a computer system in a plurality of collaborating computer systems, the method comprising:

receiving, at a first computer system, a first data structure from a collaborating second computer system, the first data structure representing first data relating to a first intrusion attempt detected by an intrusion detection system of the collaborating second computer system such that the first data is hidden in the first data structure;

receiving an indication of a characteristic of the collaborating second computer system;

detecting, using an intrusion detection system of the first computer system, a second intrusion attempt;

storing second data relating to the second intrusion attempt in a second data structure of the first computer system such that the second data is hidden in the second data structure;

determining whether the second intrusion attempt correlates with the first intrusion attempt by comparing the first data structure and the second data structure; and

based on the characteristic of the collaborating second computer system, indicating that a threat is present if the second intrusion attempt is determined to correlate with the first intrusion attempt.

21. The method of claim 20, wherein the first data and the characteristic received from the collaborating second computer system form at least part of a profile of the first intrusion attempt.

22. The method of claim 21, further comprising responding to the threat based on the profile of the first intrusion attempt.

23. The method of claim 20, wherein the characteristic comprises the type of user of the collaborating system.

24. The method of claim 23, wherein the characteristic indicates that the user is a bank.

25. The method of claim 23, wherein the characteristic indicates that the user is a university.

26. The method of claim 20, wherein receiving the first data structure from the collaborating second computer system comprises periodically exchanging data structures with the collaborating second computer system.

27. The method of claim 26, wherein the data structures are exchanged through at least one proxy.

28. The method of claim 26, further comprising:

grouping the collaborating second computer system and other collaborating systems into groups so that each collaborating system in a group occupies a position in that group;

exchanging data structures between collaborating systems in the same position in the different groups; and

rotating the position occupied by each member of at least one of the groups according to a schedule.

29. The method of claim 28, further comprising changing the schedule.

30. The method of claim 20, wherein the second intrusion attempt is one of a scan and probe.

31. The method of claim 20, wherein the second intrusion attempt is a blocked worm.

32. The method of claim 20, wherein the intrusion detection system is a honey pot.

33. The method of claim 20, wherein the first data structure is a bloom filter.

34. The method of claim 33, wherein storing second data relating to the second intrusion attempt in a second data structure of the first computer system comprises:

forming a hash of the second data relating to the second intrusion attempt;

using the hash as an index to the bloom filter; and

determining whether a bit setting corresponding bits in the bloom filter based on the index.

35. The method of claim 34, wherein forming the hash comprises using at least one of the SHA-1 and the MD-5 hashing algorithms.

36. The method of claim 33, wherein storing second data relating to the second intrusion attempt in the second data structure of the first computer system comprises:

forming a hash of the second data relating to the second intrusion attempt;

selecting at least one portion of the hash;

using the at least one portion of the hash as an index to the bloom filter; and

setting corresponding bits in the bloom filter based on the index.

37. The method of claim 36, wherein forming the hash comprises using at least one of the SHA-1 and the MD-5 hashing algorithms.

38. The method of claim 20, wherein indicating that the threat is present comprises outputting an IP address of the source of the threat.

39. The method of claim 20, wherein indicating that the threat is present comprises causing a firewall to block traffic from an IP address of the source of the threat.

40. The method of claim 20, wherein indicating that the threat is present comprises outputting a signature associated with the threat.

41. A system configured to detect intrusion attempts in a computer system among a plurality of collaborating computer systems, comprising:

at least one server that;

receives a first one-way data structure from a collaborating computer system, wherein the first one-way data structure represents first data relating to a first intrusion attempt detected by an intrusion detection system of the collaborating computer system such that the first data is hidden in the first one-way data structure;

detects a second intrusion attempt;

stores second data relating to the second intrusion attempt in a second one-way data structure such that the second data is hidden in the second one-way data structure;

determines whether the second intrusion attempt correlates with the first intrusion attempt by comparing the first one-way data structure and the second one-way data structure; and

indicates that a threat is present if the second intrusion attempt is determined to correlate with the first intrusion attempt.

42. The system of claim 41, wherein the at least one server, in determining whether the second data relating to the second intrusion attempt correlates with the data received from the collaborating second computer system relating to the first intrusion attempt, also:

compares bits of the first one-way data structure to corresponding bits of the second one-way data structure; and

indicates a correlation if at least a predetermined number of corresponding bits are set to the same bit value.

43. A system configured to detect intrusion attempts in a computer system among a plurality of collaborating computer systems, comprising:

at least one server that;

receives a first data structure from a collaborating computer system, wherein the first data structure represents first data relating to a first intrusion attempt detected by an intrusion detection system of the collaborating computer system such that the first data is hidden in the first data structure;

receives an indication of a characteristic of the collaborating computer system;

detects a second intrusion attempt;

stores second data relating to the second intrusion attempt in a second data structure such that the second data is hidden in the second data structure;

determines whether the second intrusion attempt correlates with the first intrusion attempt by comparing the first data structure and the second data structure; and

based on the characteristic of the collaborating computer system, indicates that a threat is present if the second intrusion attempt correlates with the first intrusion attempt.