Apparatus and method for performing spam detection and filtering using an image history table

• US 8,356,076 B1
• Filed: 01/30/2007
• Issued: 01/15/2013
• Est. Priority Date: 01/30/2007
• Status: Active Grant

• Current Assignee: Proofpoint Incorporated
• Original Assignee: Proofpoint Incorporated
• Inventors: Wittel, Greg, Yang, Yanyan, Panzer, Scott, Lewis, Steven
• Primary Examiner(s): Duong, Oanh
• Application Number:
  US11/700,467
• Time in Patent Office: 2,177 Days
• Field of Search: 709/206, 709/207, 709/223, 726/22, 726/26, 726/4, 382/190
• US Class Current: 709/206
• CPC Class Codes: H04L 51/212 using filtering or selective blocking

Abstract

A system and method are described for using different types of image fingerprints to detect spam. Specifically, one embodiment of the invention dynamically calculates both an image-specific signature (e.g., MD5 checksum) and a generic signature for each image and, if a specified number of images have the same generic signature but different image-specific signatures, then a feature is fired within a spam engine to indicate that the images are spam. By way of example, a computer-implemented method is described for detecting spam images within a stream of messages comprising: generating generic signatures and image-specific signatures for images within the stream of messages; determining if two or more images with matching generic signatures have different image-specific signatures; and updating a spam filtering engine to include the matching generic signatures if the number of images with matching generic signatures but different image-specific signatures reach a specified threshold value.

Claims

1. A computer-implemented method for detecting spam images within a stream of messages comprising:

extracting email messages having images with specified characteristics from the stream and performing the following operations only for those email messages;

generating a generic signature and an image-specific signature for a first images within the stream messages;

determining if the generic signature of the first image matches a generic signature of a second image;

determining if the first and second images with matching generic signatures have different image-specific signatures; and

including the matching generic signatures of the first and second images and one or more additional images in a spam filtering engine if a number of images with matching generic signatures but different image-specific signatures including the first and second images reaches a specified threshold value.

2. The method as in claim 1 wherein the generic signatures are stored in a lookup table, wherein each entry in the lookup table points to a table containing one or more of the image-specific signatures.

3. The method as in claim 1 wherein the generic signatures comprise down-converted representations of portions of the images.

4. The method as in claim 1 wherein the image-specific signatures comprise MD5 signatures.

5. The method as in claim 1 wherein if the number of images with the matching generic signatures but different image-specific signatures reaches a second specified threshold, then increasing a weight associated with the matching generic signatures within the spam filtering engine.

6. The method as in claim 1 wherein the specified threshold comprises two image-specific signatures for the matched generic signature.

7. The method as in claim 1 wherein each of the generic signatures are represented by a vector and wherein a match between two generic signatures is determined if absolute differences between values of the two vectors are cumulatively below a specified threshold.

8. The method as in claim 1 wherein the operations of generating and determining are performed at a spam analysis site and wherein the spam filtering engine is updated at a spam reception site.

9. A system for detecting spam images within a stream of messages, the system comprising a memory for storing program code and a processor for processing code to perform the operations of:

extracting email messages having images with specified characteristics from the stream and performing the following operations only for those email messages;

generating a generic signature and an image-specific signature for a first images within the stream of messages;

determining if the generic signature of the first image matches a generic signature of a second image;

determining if the first and second images with matching generic signatures have different image-specific signatures; and

including the matching generic signatures of the first and second images and one or more additional images in a spam filtering engine if a number of images with matching generic signatures but different image-specific signatures including the first and second images reaches a specified threshold value.

10. The system as in claim 9 further comprising:

a lookup table to store the generic signatures, wherein each entry in the lookup table points to a table containing one or more of the image-specific signatures.

11. The system as in claim 9 wherein the generic signatures comprise down-converted representations of portions of the images.

12. The system as in claim 9 wherein the image-specific signatures comprise MD5 signatures.

13. The system as in claim 9 wherein if the number of images with the matching generic signatures but different image-specific signatures reaches a second specified threshold, then the program code performs the additional operation of increasing a weight associated with the matching generic signatures within the spam filtering engine.

14. The system as in claim 9 wherein the specified threshold comprises two image-specific signatures for the matched generic signature.

15. The system as in claim 9 wherein each of the generic signatures are represented by a vector and wherein a match between two generic signatures is determined if absolute differences between values of the two vectors are cumulatively below a specified threshold.

16. The system for system as in claim 9 wherein the program code for generating generic signatures and image-specific signatures for images within the stream of messages and determining if two or more of the images with matching generic signatures have different image-specific signatures is implemented at a spam analysis site and the spam filter is implemented at an email reception site.

17. A non-transitory machine-readable medium having stored thereon sequences of instructions which, when executed by a machine, causes the machine to perform the operations of:

extracting email messages having images with specified characteristics from the stream and performing the following operations only for those email messages;

generating a generic signature and an image-specific signature for a first images within the stream of messages;

determining if the generic signature of the first image matches a generic signature of a second image;

determining if the first and second images with matching generic signatures have different image-specific signatures; and

including the matching generic signatures of the first and second images and one or more additional images in a spam filtering engine if a number of images with matching generic signatures but different image-specific signatures including the first and second images reaches a specified threshold value.

18. The machine-readable medium as in claim 17 wherein the generic signatures are stored in a lookup table, wherein each entry in the lookup table points to a table containing one or more of the image-specific signatures.

19. The machine-readable medium as in claim 17 wherein the generic signatures comprise down-converted representations of portions of the images.

20. The machine-readable medium as in claim 17 wherein the image-specific signatures comprise MD5 signatures.