Detection of spam images

• US 8,103,048 B2
• Filed: 12/04/2007
• Issued: 01/24/2012
• Est. Priority Date: 12/04/2007
• Status: Active Grant

• Current Assignee: McAfee, LLC
• Original Assignee: McAfee, Inc. (McAfee, LLC)
• Inventors: Sheinin, Michael, Targosz, Rob
• Primary Examiner(s): CHANG, JON CARLTON
• Application Number:
  US11/987,728
• Publication Number:
  US 20090141985A1
• Time in Patent Office: 1,512 Days
• Field of Search: 382/100, 382/199, 382/173, 382/176
• US Class Current: 382/100
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/18095 Summing image-intensity values; Projection and histogram analysis
• CPC Class Codes: G06V 30/413 Classification of content, e.g. text, photographs or tables
• CPC Class Codes: H04L 51/212 using filtering or selective blocking

Abstract

A method, system, and computer program product for analyzing image attachments to email messages and reliably determines whether the image includes spam, so that the message can be blocked. A method for processing email messages comprises processing an image included in or attached to an email message to determine whether the image includes features that indicate whether the image is spam and determining whether the image is spam based on the included features that indicate whether the image is spam.

Claims

1. A method, comprising:

processing an image included in or attached to an email message to determine whether the image includes features that indicate whether the image is spam;

detecting contiguous horizontal regions associated with the image; and

detecting horizontal and vertical edges of contiguous regions as the features that indicate whether the image is spam, wherein a ratio of a number of horizontal edges to a number of vertical edges is used, at least in part, to determine whether the image is spam.

2. The method of claim 1, wherein the processing includes:

converting the image to have luminance information; and

detecting contiguous horizontal regions with distinctive luminance.

3. The method of claim 2, wherein the contiguous horizontal regions with distinctive luminance are detected by:

performing horizontal smoothening on the image; and

performing segmentation of the image.

4. The method of claim 3, wherein the horizontal smoothening is performed by:

combining multiple pixels by setting each of the combined pixels to an average value of the combined pixels.

5. The method of claim 3, wherein the segmentation of the image is performed by:

computing an average luminance value of the image;

dividing each horizontal line in the image into a plurality of equal segments; and

for each segment of each horizontal line in the image;

determining an average and minimum luminance value of the segment, and

marking each pixel of the segment as belonging to a foreground or belonging to a background.

6. The method of claim 5, wherein the determining an average and minimum luminance value of the segment are performed by:

determining that the background is brighter than a content when the average luminance value of the segment is greater than a threshold value and that the background is darker than the content when the average luminance value is less than or equal to the threshold value;

setting a value to be used to mark a segment as belonging to the background to black and a value to be used to mark a segment as belonging to a foreground to white when the background is brighter than the content; and

setting a value to be used to mark a segment as belonging to the background to white and a value to be used to mark a segment as belonging to a foreground to black when the background is darker than the content.

7. The method of claim 6, wherein the marking each pixel of the segment as belonging to a foreground or belonging to a background are performed by:

marking all pixels of the segment as belonging to the background when the average and minimum luminance values of the segment are equal;

marking the pixel as belonging to the background when the luminance of the pixel is greater than the average luminance of the segment; and

marking the pixel as belonging to the fore when the luminance of the pixel is less than or equal to the average luminance of the segment.

8. The method of claim 1, wherein the determining includes:

determining that the image is spam when the ratio of a number of horizontal edges to a number of vertical edges is greater than a threshold value.

9. The method of claim 8, wherein the threshold value is included in a range of approximately 10 to 15.

10. A system comprising:

a processor operable to execute computer program instructions;

a memory operable to store computer program instructions executable by the processor; and

computer program instructions for performing;

processing an image included in or attached to an email message to determine whether the image includes features that indicate whether the image is spam;

determining whether the image is spam based on the included features that indicate whether the image is spam; and

detecting horizontal and vertical edges of contiguous regions as the features that indicate whether the image is spam, wherein a ratio of a number of horizontal edges to a number of vertical edges is used, at least in part, to determine whether the image is spam.

11. The system of claim 10, wherein the processing includes:

converting the image to have luminance information; and

detecting contiguous horizontal regions with distinctive luminance.

12. The system of claim 11, wherein the contiguous horizontal regions with distinctive luminance are detected by:

performing horizontal smoothening on the image; and

performing segmentation of the image.

13. The system of claim 12, wherein the horizontal smoothening is performed by:

combining multiple pixels by setting each of the combined pixels to an average value of the combined pixels.

14. The system of claim 12, wherein the segmentation of the image is performed by:

computing an average luminance value of the image;

dividing each horizontal line in the image into a plurality of equal segments; and

for each segment of each horizontal line in the image;

determining an average and minimum luminance value of the segment, and

marking each pixel of the segment as belonging to a foreground or belonging to a background.

15. The system of claim 14, wherein the determining an average and minimum luminance value of the segment are performed by:

determining that the background is brighter than a content when the average luminance value of the segment is greater than a threshold value and that the background is darker than the content when the average luminance value is less than or equal to the threshold value;

setting a value to be used to mark a segment as belonging to the background to black and a value to be used to mark a segment as belonging to a foreground to white when the background is brighter than the content; and

setting a value to be used to mark a segment as belonging to the background to white and a value to be used to mark a segment as belonging to a foreground to black when the background is darker than the content.

16. The system of claim 15, wherein the marking each pixel of the segment as belonging to a foreground or belonging to a background are performed by:

marking all pixels of the segment as belonging to the background when the average and minimum luminance values of the segment are equal;

marking the pixel as belonging to the background when the luminance of the pixel is greater than the average luminance of the segment; and

marking the pixel as belonging to the fore when the luminance of the pixel is less than or equal to the average luminance of the segment.

17. The system of claim 10, wherein the determining includes:

determining that the image is spam when the ratio of a number of horizontal edges to a number of vertical edges is greater than a threshold value.

18. The system of claim 17, wherein the threshold value is included in a range of approximately 10 to 15.

19. A computer program product for processing incoming email messages comprising:

a non-transitory computer readable storage medium;

computer program instructions, recorded on the computer readable storage medium, executable by a processor, for performing;

processing an image included in or attached to an email message to determine whether the image includes features that indicate whether the image is spam;

determining whether the image is spam based on the included features that indicate whether the image is spam; and

detecting horizontal and vertical edges of contiguous regions as the features that indicate whether the image is spam, wherein a ratio of a number of horizontal edges to a number of vertical edges is used, at least in part, to determine whether the image is spam.

20. The computer program product of claim 19, wherein the processing includes:

converting the image to have luminance information; and

detecting contiguous horizontal regions with distinctive luminance.

21. The computer program product of claim 20, wherein the contiguous horizontal regions with distinctive luminance are detected by:

performing horizontal smoothening on the image; and

performing segmentation of the image.

22. The computer program product of claim 21, wherein the horizontal smoothening is performed by:

combining multiple pixels by setting each of the combined pixels to an average value of the combined pixels

23. The computer program product of claim 21, wherein the segmentation of the image is performed by:

computing an average luminance value of the image;

dividing each horizontal line in the image into a plurality of equal segments; and

for each segment of each horizontal line in the image;

determining an average and minimum luminance value of the segment, and

marking each pixel of the segment as belonging to a foreground or belonging to a background.

24. The computer program product of claim 23, wherein the determining an average and minimum luminance value of the segment are performed by:

determining that the background is brighter than a content when the average luminance value of the segment is greater than a threshold value and that the background is darker than the content when the average luminance value is less than or equal to the threshold value;

setting a value to be used to mark a segment as belonging to the background to black and a value to be used to mark a segment as belonging to a foreground to white when the background is brighter than the content; and

setting a value to be used to mark a segment as belonging to the background to white and a value to be used to mark a segment as belonging to a foreground to black when the background is darker than the content.

25. The computer program product of claim 24, wherein the marking each pixel of the segment as belonging to a foreground or belonging to a background are performed by:

marking all pixels of the segment as belonging to the background when the average and minimum luminance values of the segment are equal;

marking the pixel as belonging to the background when the luminance of the pixel is greater than the average luminance of the segment; and

marking the pixel as belonging to the fore when the luminance of the pixel is less than or equal to the average luminance of the segment.

26. The computer program product of claim 19, wherein the determining includes:

determining that the image is spam when the ratio of a number of horizontal edges to a number of vertical edges is greater than a threshold value.

27. The computer program product of claim 26, wherein the threshold value is included in a range of approximately 10 to 15.