System and methods for detecting malicious email transmission

• US 8,443,441 B2
• Filed: 12/08/2009
• Issued: 05/14/2013
• Est. Priority Date: 08/16/2001
• Status: Active Grant

• Current Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Original Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Inventors: Stolfo, Salvatore J., Eskin, Eleazar, Herskop, Shlomo, Bhattacharyya, Manasi
• Primary Examiner(s): Yalew, Fikremariam A
• Application Number:
  US12/633,493
• Publication Number:
  US 20100169970A1
• Time in Patent Office: 1,253 Days
• Field of Search: None
• US Class Current: 726/22
• CPC Class Codes: H04L 51/212 using filtering or selective blocking
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/145 the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms

Abstract

A system and methods of detecting an occurrence of a violation of an email security policy of a computer system. A model relating to the transmission of prior emails through the computer system is defined which is derived from statistics relating to the prior emails. For selected emails to be analyzed, statistics concerning the selected email are gathered. Such statistics may refer to the behavior or other features of the selected emails, attachments to emails, or email accounts. The determination of whether a violation of an email security policy has occurred is performed by applying the model of prior email transmission to the statistics relating to the selected email. The model may be statistical or probabilistic. A model of prior email transmission may include grouping email recipients into cliques. A determination of a violation of a security policy may occur if email recipients for a particular email are in more than one clique.

Claims

1. A method for monitoring transmission of email through a computer system, said computer system comprising a server and one or more clients having an email account, the method comprising:

(a) gathering statistics relating to the transmission behavior of prior emails relating to a first email account on said computer system;

(b) generating a profile relating to the transmission behavior of email relating to said first email account based on said statistics, wherein generating a profile comprises grouping email addresses into one or more cliques based on the prior occurrence of said email addresses within the same emails in a group of emails;

(c) gathering statistics relating to the transmission of selected email between said first email account and one or more additional email accounts; and

(d) comparing said statistics relating to the transmission behavior of selected email with said generated profile.

2. The method according to claim 1, wherein gathering statistics relating to the transmission behavior of prior emails comprises gathering statistics relating to the email addresses of emails sent to said first email account.

3. The method according to claim 1, wherein gathering statistics relating to the transmission behavior of prior emails comprises gathering statistics relating to the email addresses of email sent by said first email account.

4. The method according to claim 1, wherein gathering statistics relating to the transmission behavior of prior emails comprises gathering statistics relating to the number of emails sent by said first email account.

5. The method according to claim 1, wherein gathering statistics relating to the transmission behavior of prior emails further comprises gathering statistics relating to the number of emails transmitted between said first email account and said one or more additional email accounts.

6. The method according to claim 1, wherein generating a profile relating to the transmission behavior of email further comprises grouping said one or more additional email accounts according to the domain of each of said additional email accounts.

7. The method according to claim 1, wherein gathering statistics relating to the transmission behavior of prior emails further comprises assigning a unique identifier to each email sent to said first email account.

8. The method according to claim 1, wherein gathering statistics relating to the transmission behavior of prior emails further comprises assigning a unique identifier to each email sent between said first email account and one or more additional email accounts.

9. The method according to claim 1, wherein gathering statistics relating to the transmission behavior of prior emails further comprises assigning a unique identifier to each email sent to said first email account.

10. The method according to claim 1, wherein gathering statistics relating to the transmission behavior of prior emails further comprises assigning a unique identifier to each email sent between said first email account and one or more additional email accounts.

11. A method for monitoring transmission of email through a computer system, said computer system comprising a server and one or more clients having an email account, the method comprising:

(a) defining a model relating to prior transmission of email through said computer system derived from statistics relating to transmission behavior of prior emails transmitted through said computer system;

(b) generating a baseline profile relating to the normal transmission behavior of email through said computer system, wherein said baseline profile comprises a histogram of said normal transmission behavior of email through said computer system; and

(c) based on said histogram, making a determination regarding said normal transmission behavior of email through said computer system.

12. The method according to claim 11, where said model relating to prior transmission of email is based on the number of emails transmitted through said computer system during a specified period of time.

13. The method according to claim 11, wherein making a determination comprises:

(a) gathering statistics relating to transmission behavior of selected email through said computer system; and

(b) comparing said statistics relating to transmission behavior of said selected email with said baseline profile.

14. The method according to claim 13, wherein gathering statistics relating to transmission behavior of selected email through said computer system further comprises defining a histogram of said transmission behavior of selected email through said computer system.

15. The method according to claim 14, wherein comparing said statistics relating to transmission behavior of said selected email with said baseline profile comprises comparing said histogram of said normal transmission behavior of email to said histogram of said transmission behavior of selected email through said computer system.

16. The method according to claim 15, wherein comparing said histogram of said normal transmission behavior of email to said histogram of said transmission behavior of selected email comprises performing a Mahalanobis distance analysis.

17. The method according to claim 15, wherein comparing said histogram of said normal transmission behavior of email to said histogram of said transmission behavior of selected email comprises performing a Kolmogorov-Simironov test.

18. The method according to claim 15, wherein comparing said histogram of said normal transmission behavior of email to said histogram of said transmission behavior of selected email comprises performing a Chi-square test.

19. The method according to claim 16 further comprising setting a threshold value and comparing said histogram difference to said threshold value to determine whether selected email transmission behavior is normal.

20. The method according to claim 17 further comprising setting a threshold value and comparing said histogram difference to said threshold value to determine whether selected email transmission behavior is normal.

21. The method according to claim 18 further comprising setting a threshold value and comparing said histogram difference to said threshold value to determine whether selected email transmission behavior is normal.