Systems and methods for identifying spam messages using subject information

• US 9,647,975 B1
• Filed: 09/28/2016
• Issued: 05/09/2017
• Est. Priority Date: 06/24/2016
• Status: Active Grant

• Current Assignee: Lab A.O. Kaspersky
• Original Assignee: Lab A.O. Kaspersky
• Inventors: Dedenok, Roman A.
• Primary Examiner(s): Boutah, Alina N
• Application Number:
  US15/278,512
• Time in Patent Office: 223 Days
• Field of Search: 709206, 709224, 709213, 709200, 709203, 709207, 709225, 709232, 707736, 707737, 707740, 707748, 706 20
• CPC Class Codes: H04L 51/212 using filtering or selective blocking

Abstract

Systems and methods for identifying a spam email message. A system can include a rules database configured to store a plurality of ratio determination rules, a vectors database configured to store a plurality of known vectors, a message processing tool configured to receive an email message, a gram building tool configured to build a k-skip-n-gram set of word combinations according he ratio determination rules, a vector building tool configured to receive the k-skip-n-gram set of word combinations, and build a vector for each k-skip-n-gram word combination, and a spam identification tool configured to determine a spam presence threshold based on the cosine similarity for each k-skip-n-gram word combination and the plurality of known vectors for the particular email message subject field subject category, and determine that the email message contains spam when the spam presence threshold is exceeded.

Claims

1. A system for identifying a spam email message, the system comprising:

a computing platform including computing hardware of at least one processor, a memory operably coupled to the at least one processor and configured to store instructions invoked by the at least one processor, an operating system implemented on the computing hardware, and input/output facilities;

a rules database configured to store a plurality of ratio determination rules including a set of conditions for a text string for which the rules are applied to determine an n-value of words in a gram and a k-value of words to skip in an input text;

a vectors database configured to store a plurality of known vectors, wherein the plurality of known vectors are classified by thematic category;

instructions that, when executed on the computing platform, cause the computing platform to implement;

a message processing tool configured to receive an email message via the input/output facilities, the email message containing a subject field,

a gram building tool configured to build a k-skip-n-gram set of word combinations according to the ratio of the k-value and the n-value for the subject field as the input text as determined by the ratio determination rules in the rules database,

a vector building tool configured to receive, from the gram building tool, the k-skip-n-gram set of word combinations, and build a vector for each k-skip-n-gram word combination, and

a spam identification tool configured to determine a spam presence threshold based on the cosine similarity for each k-skip-n-gram word combination and the plurality of known vectors for the particular email message subject field thematic category, and determine that the email message contains spam when the spam presence threshold is exceeded.

2. The system for identifying a spam email message of claim 1, wherein the plurality of ratio determination rules defining the n-value and the k-value includes an analysis by the gram building tool of at least one message subject text parameter including

a language of the subject field;

a quantity of words in the subject field;

a quantity of articles in the subject field;

a quantity of punctuation signs in the subject field;

a quantity of pronouns in the subject field; and

a quantity of prepositions in the subject field.

3. The system for identifying a spam email message of claim 1, wherein the k-skip-n-gram set of word combinations is built by the gram building tool according to the formula:

4. The system for identifying a spam email message of claim 1, wherein the vector building tool is configured determine a particular email message subject field thematic category by equating a high cosine similarity between the at least one of the plurality of known vectors and at least one of the vectors built for each k-skip-n-gram word combination with the thematic category of the at least one of the plurality of known vectors.

5. The system for identifying a spam email message of claim 1, wherein the determination that the email message contains spam includes at least the arithmetic mean of the cosine similarity of all built vectors.

6. The system for identifying a spam email message of claim 1, wherein the spam presence threshold is determined by a regression analysis of a test message collection.

7. The system for identifying a spam email message of claim 6, wherein the spam presence threshold is adjusted based on a quantity of k-skip-n-gram word combinations built by the gram building tool and a quantity of the plurality of known vectors that have a high cosine similarity to at least one of the built vectors.

8. The system for identifying a spam email message of claim 1, wherein the subject field comprises text of three or more words.

9. A method for identifying a spam email message with a computing platform including computing hardware of at least one processor, a memory operably coupled to the at least one processor and configured to store instructions invoked by the at least one processor, the method comprising:

receiving an email message with a messaging processing tool invoked by the at least one processor, the email message containing a subject field text;

determining, with the messaging processing tool, at least one parameter of the subject field text;

determining, with a ratio determination tool invoked by the at least one processor, an n-value of words in a gram and a k-value of words to skip in the subject field text based on the at least one parameter of the subject field text a plurality of ratio determination rules;

building, with a gram building tool invoked by the at least one processor, a k-skip-n-gram set of word combinations of the subject field text according to the ratio of the k-value and the n-value;

building, with a vector building tool invoked by the at least one processor, a vector for each k-skip-n-gram word in the set of word combinations;

calculating, with the vector building tool, a cosine similarity for each k-skip-n-gram word combination and a plurality of known vectors, wherein the plurality of known vectors are classified by thematic category;

determining, with the vector building tool, a message thematic category based on the calculated rates of cosine similarity for each k-skip-n-gram word combination to the known vectors and the classified thematic categories of the known vectors;

determining, with a spam identification tool invoked by the at least one processor, a spam presence threshold based on the cosine similarity for each k-skip-n-gram word combination and the plurality of known vectors for the determined email message thematic category;

calculating, with the spam identification tool, a current spam presence value based at least on the rates of cosine similarity between all built vectors; and

determining, with the spam identification tool, that the email message contains spam when the spam presence threshold is exceeded by the current spam presence value.

10. The method for identifying a spam email message of claim 9, wherein the at least one parameter of the subject field text comprises:

a language of the subject field text;

a quantity of words in the subject field text;

a quantity of articles in the subject field text;

a quantity of punctuation signs in the subject field text;

a quantity of pronouns in the subject field text;

or

a quantity of prepositions in the subject field text.

11. The method for identifying a spam email message of claim 9, wherein building the k-skip-n-gram set of word combinations is conducted according to the formula:

12. The method for identifying a spam email message of claim 9, wherein determining the message thematic category comprises equating a high cosine similarity between the at least one of the plurality of known vectors and at least one of the vectors built for each k-skip-n-gram word combination with the thematic category of the at least one of the plurality of known vectors.

13. The method for identifying a spam email message of claim 9, wherein the current spam presence value is determined according to the arithmetic mean of the cosine similarity of all built vectors.

14. The method for identifying a spam email message of claim 9, wherein the spam presence threshold is determined by a regression analysis of a test message collection.

15. The method for identifying a spam email message of claim 14, wherein the spam presence threshold is adjusted based on a quantity of k-skip-n-gram word combinations built by and a quantity of the plurality of known vectors that have a high cosine similarity to at least one of the built vectors.

16. The method for identifying a spam email message of claim 9, further comprising providing a rules database accessible by the computing platform and configured to store the plurality of ratio determination rules.

17. The method for identifying a spam email message of claim 9, further comprising providing a vectors database accessible by the computing platform and configured to store the plurality of known vectors.

18. A system for subject-line email analysis on a computing platform including computing hardware of at least one processor, a memory operably coupled to the at least one processor and configured to store instructions invoked by the at least one processor, the system comprising:

instructions that, when executed on the computing platform, cause the computing platform to implement;

a message processing engine communicatively coupled to an email server and configured to receive an email message, the email message containing a subject line;

a gram building engine configured to build a set of word combinations for the subject-line according to the formula;

19. The system for subject-line email analysis of claim 18, further comprising:

instructions that, when executed on the computing platform, cause the computing platform to implement a spam identification engine communicatively coupled to the vector building engine and configured to determine whether the email contains spam based on the thematic category and a calculated rate of cosine similarity between all vectors from a set built from the subject-line.

20. The system for subject-line email analysis of claim 19, wherein the spam identification engine is further configured to:

determine a limit value based on the thematic category;

determine a current value based at least on the rates of cosine similarity between all built vectors; and

determine whether the email contains spam based on whether the current value exceeds the limit value.