Detecting image spam

• US 8,763,114 B2
• Filed: 01/24/2007
• Issued: 06/24/2014
• Est. Priority Date: 01/24/2007
• Status: Active Grant

• Current Assignee: McAfee, LLC
• Original Assignee: McAfee, Inc. (McAfee, LLC)
• Inventors: Alperovitch, Dmitri, Black, Nick, Gould, Jeremy, Judge, Paul, Krasser, Sven, Schneck, Phyllis Adele, Tang, Yuchun, Trivedi, Aarjav Jyotindra Neeta, Willis, Lamar Lorenzo, Yang, Weilai, Zdziarski, Jonathan Alexander
• Primary Examiner(s): SIMITOSKI, MICHAEL J
• Application Number:
  US11/626,568
• Publication Number:
  US 20080178288A1
• Time in Patent Office: 2,708 Days
• Field of Search: 726/22, 382/103, 382/209, 709/206, 709/207, 709/246
• US Class Current: 726/22
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]
• CPC Class Codes: H04L 51/212 using filtering or selective blocking
• CPC Class Codes: H04L 63/0227 Filtering policies mail message filtering H04L51/212
• CPC Class Codes: H04L 63/20 for managing network security; network security policies in general filtering policies H04L63/0227

Abstract

Methods and systems for operation upon one or more data processors for detecting image spam by detecting an image and analyzing the content of the image to determine whether the incoming communication comprises an unwanted communication.

Claims

1. A computer implemented method comprising:

receiving an incoming communication associated with a particular entity;

identifying that the communication contains one or more images;

determining whether one or more of the images includes a graphic encoding of a textual spam message, wherein the determining includes, for each of the one or more images;

normalizing the image to generate a normalized image, wherein the normalized image is a representation of the image with at least some noise removed from the image and the normalized image comprises overlapping sub-regions having image data;

determining whether the image includes graphical encoding of text corresponding to text in known spam;

determining whether aspect ratio of the image corresponds to a known aspect ratio corresponding to known spam;

generating fingerprints of the image data from the overlapping sub-regions of the normalized image, wherein the fingerprints specify attributes of the normalized image;

comparing the fingerprints from the normalized image with fingerprints of known spam images, wherein the comparison comprises for at least one of the fingerprints;

determining a first measure that the at least one of the fingerprints is similar to at least one fingerprint of a known spam image;

determining a second measure that the at least one of the fingerprints is similar to at least one fingerprint of a known non-spam image; and

classifying the image as a spam image or a non-spam image based at least in part on the determination of whether the image includes graphical encoding of text corresponding to text in known spam, the determination of whether the aspect ratio of the image corresponds to a known aspect ratio corresponding to known spam, and the comparison of the fingerprints; and

updating a reputation score for the particular entity based at least in part upon a result of the classification, wherein the reputation score for the particular entity is further based on a strength of relationship between the particular entity and a first entity having a reputable reputation score and the strength of relationship is based on similarities between content in messages sent by the particular entity and content of messages sent by the first entity.

2. The method of claim 1, wherein normalizing the image to generate a normalized image comprises removing features introduced by spammers to evade optical character recognition engines or signature-based detection techniques.

3. The method of claim 2, further comprising executing an optical character recognition engine on the normalized image to produce a text version of the communication.

4. The method of claim 2, wherein normalizing the image comprises removing obfuscation, noise or other irrelevant content from the image.

5. The method of claim 2, wherein normalizing the image comprises executing a median filter on the image.

6. The method of claim 2, wherein normalizing the image comprises quantizing the image to remove extraneous color information.

7. The method of claim 2, wherein normalizing the image comprises executing a contrast stretching engine on the image.

8. The method of claim 1, further comprising winnowing the resulting fingerprints to identify common sub-regions between the image and known image spam or other previously identified entities.

9. The method of claim 1, further comprising identifying high level features to be classified by machine learning engines.

10. The method of claim 9, wherein the machine learning engines uses algorithms such as a clustering technique or a support vector machine.

11. The method of claim 9, further comprising deriving histogram modes associated with the image to determine whether the image includes any anomalous histogram modes.

12. The method of claim 11, further comprising analyzing a frequency distribution of the image to determine whether the image includes an anomalous frequency distribution.

13. The method of claim 11, further comprising analyzing a signal to noise ratio of the image to determine whether the image includes an anomalous signal to noise ratio.

14. The method of claim 13, wherein any anomalies indicate that the communication comprises image spam.

15. The method of claim 1, further comprising analyzing formalities associated with the communication to determine whether any anomalies exist with respect to the communication or the content of the communication.

16. The method of claim 1, further comprising analyzing the communication to detect whether an image is mislabeled as a file type different from the actual file type of the image.

17. The method of claim 1, further comprising analyzing the communication comprises detecting images which are designed to be nonconforming to the corresponding image standard specifications to which the image purports to conform.

18. The method of claim 1, further comprising determining whether the image is legitimate by comparing the image to a set of whitelisted images.

19. The method of claim 1, further comprising analyzing the communication structure and layout to determine whether the communication appears similar to a previously identified malicious communication.

20. The method of claim 1, further comprising identifying features that may be classified by a machine learning engine.

21. The method of claim 20, further comprising executing a Bayesian classification of the communication.

22. The method of claim 1, wherein the incoming communication has a hypertext markup language structure, the method further comprising:

defining meta-tokens based on the hypertext markup language structure of the communication, wherein the meta-tokens specify hypertext markup language content of the communication including image content from the one or more images;

determining that the communication is similar to at least one other communication based on the defined meta-tokens, wherein the at least one other communication is a known unwanted communication; and

classifying the communication as an unwanted communication based on the determination that the communication is similar to the at least one other communication.

23. The method of claim 1, wherein the first measure is based at least in part on whether the at least one of the fingerprints is similar to a first number of known spam image fingerprints and the second measure is based at least in part on whether the at least one of the fingerprints is similar to a second number of known non-spam image fingerprints, each of the first and second numbers being greater than zero.

24. The method of claim 1, wherein updating the reputation score comprises updating both a global version of the reputation score for the particular entity and a local version of the reputation score for the particular entity.

25. An image spam detection system, comprising:

at least one processor;

at least one memory element;

a communications interface operable to receive a communication via a network, wherein the communication is associated with a particular entity;

a detector operable, when executed by the at least one processor, to identify whether the communication comprises an image;

an analyzer operable, when executed by the at least one processor, to, in response to the identification that the communication comprises an image, determine whether the image corresponds to a graphic encoding of a textual spam message, wherein the determining includes;

normalizing the image to generate a normalized image, wherein the normalized image is a representation of the image with at least some noise removed from the image and the normalized image comprises overlapping sub-regions having image data;

determining whether the image includes graphical encoding of text corresponding to text in known spam;

determining whether aspect ratio of the image corresponds to a known aspect ratio corresponding to known spam;

generating fingerprints of the image data from the overlapping sub-regions of the normalized image, wherein the fingerprints specify attributes of the normalized image;

comparing the fingerprints from the normalized image with fingerprints of known spam images, wherein the comparison comprises for at least one of the fingerprints;

determining a first measure that the at least one of the fingerprints is similar to at least one fingerprint of a known spam image;

determining a second measure that the at least one of the fingerprints is similar to at least one fingerprint of a known non-spam image; and

classifying the image as a spam image or a non-spam image based at least in part on the determination of whether the image includes graphical encoding of text corresponding to text in known spam, the determination of whether the aspect ratio of the image corresponds to a known aspect ratio corresponding to known spam, and the comparison of the fingerprints; and

a communications control engine operable to determine an action to perform with respect to the communication based at least in part upon results of the classification performed by the analyzer, wherein the action comprises updating a reputation score for the particular entity based at least in part upon a result of the classification, the reputation score for the particular entity is further based on a strength of relationship between the particular entity and a first entity having a reputable reputation score and the strength of relationship is based on similarities between content in messages sent by the particular entity and content of messages sent by the first entity.

26. The system of claim 25, wherein the analyzer comprises an edge detection engine operable process the image to produce an edge detected image, wherein the edge detection engine is operable to use digital signal processing to detect color gradations in the image.

27. The system of claim 25, further comprising an optical character recognition engine operable to process the normalized image to produce a text version of the communication.

28. The system of claim 25, wherein the analyzer is operable to remove obfuscation, noise, or other irrelevant content from the image.

29. The system of claim 25, wherein the analyzer is operable to execute a median filter on the image.

30. The system of claim 25, wherein the analyzer is operable to quantize the image to remove extraneous color information.

31. The system of claim 25, wherein the analyzer is operable to execute a contrast stretching engine on the image.

32. The system of claim 25, wherein any fingerprints generated by the analyzer are winnowed to identify common image sub-regions between the normalized image and known image spam or other previously identified entities.

33. The system of claim 25, wherein the analyzer identifies high level features to be classified by a machine learning engine.

34. The system of claim 33, wherein the machine learning engine comprises a clustering algorithm or a support vector machine.

35. The system of claim 33, wherein the analyzer is operable to derive one or more histogram modes associated with the image to determine whether the image includes any anomalous histogram modes.

36. The system of claim 35, wherein the analyzer is operable to analyze the frequency distribution of the image to determine whether the image includes an anomalous frequency distribution.

37. The system of claim 35, wherein the analyzer is operable to analyze a signal to noise ratio of the image to determine whether the image includes an anomalous signal to noise ratio.

38. The system of claim 37, wherein any anomalies indicate that the communication comprises image spam.

39. The system of claim 25, wherein the analyzer is operable to analyze formalities associated with the communication to determine whether any anomalies exist with respect to the communication or the content of the communication.

40. At least one machine accessible storage medium having instructions stored thereon, the instructions when executed on a machine, cause the machine to:

identify a communication that contains one or more images and is associated with a particular entity;

for each of the one or more images;

normalize the image to generate a normalized image, wherein the normalized image is a representation of the image with at least some noise removed from the image and the normalized image comprises overlapping sub-regions having image data;

determine whether the image includes graphical encoding of text corresponding to text in known spam;

determine whether aspect ratio of the image corresponds to a known aspect ratio corresponding to known spam;

generate fingerprints of the image data from the overlapping sub-regions of the normalized image, wherein the fingerprints specify attributes of the normalized image;

compare the fingerprints from the normalized image with fingerprints of known spam images, wherein the comparison comprises for at least one of the fingerprints;

determining a first measure that the at least one of the fingerprints is similar to at least one fingerprint of a known spam image;

determining a second measure that the at least one of the fingerprints is similar to at least one fingerprint of a known non-spam image; and

classify the image as a spam image or a non-spam image based at least in part on the determination of whether the image includes graphical encoding of text corresponding to text in known spam, the determination of whether the aspect ratio of the image corresponds to a known aspect ratio corresponding to known spam, and the comparison of the fingerprints; and

perform an action on the communication based at least in part upon a result of the classifications, wherein the action comprises updating a reputation score for the particular entity based at least in part upon a result of the classification, wherein the reputation score for the particular entity is further based on a strength of relationship between the particular entity and a strength of relationship between the particular entity and a first entity having a reputable reputation score and the strength of relationship is based on similarities between content in messages sent by the particular entity and content of messages sent by the first entity.