Apparatus and method for detecting spam

• US 9,465,789 B1
• Filed: 03/27/2013
• Issued: 10/11/2016
• Est. Priority Date: 03/27/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Chen, Roger, Frew, Kevin Thomas
• Primary Examiner(s): BOUTAH, ALINA A
• Application Number:
  US13/851,199
• Time in Patent Office: 1,294 Days
• Field of Search: 709206-207
• US Class Current: 1/1
• CPC Class Codes: G06F 40/126 Character encoding

Abstract

Provided is a process of detecting spam in websites, the process including: obtaining text from a website; detecting an amount of transitions between character sets in the text, wherein the character sets each correspond to different alphabets; calculating, with a computer, a score indicative of the likelihood that the text is spam based on the amount of transitions; and labeling the text as spam based on the score.

Claims

1. A method of detecting spam in web sites, the method comprising:

obtaining a text entry;

detecting a number of character transitions between character sets in the text entry, wherein the character sets (i) each correspond to different alphabets and (ii) are different subsets of a character encoding that maps characters from multiple alphabets to respective binary numbers for use by computers;

calculating, with a computer, a score indicative of the likelihood that the text entry is spam based on the number of character transitions; and

labeling the text entry as spam based on the score.

2. The method of claim 1, wherein the score is based on the ratio of the number of character transitions between character sets and a number of characters in the text entry.

3. The method of claim 1, wherein detecting a number of character transitions between character sets in the text entry further comprises:

identifying a primary character set of characters in the text entry;

selecting a record of misleading characters corresponding to the primary character set, wherein a misleading character in a non-primary character set is visually similar to a character in the primary character set; and

weighting a count of the number of character transitions to or from characters in the record of misleading characters more than a count of the number of character transitions to or from characters that are not in the record of misleading characters.

4. The method of claim 1, wherein obtaining a text entry further comprises:

receiving a text and segmenting the text into a plurality of text entries.

5. The method of claim 4, wherein the plurality of text entries are reviews on a website having a plurality of user-provided reviews.

6. The method of claim 1, wherein detecting a number of character transitions between character sets in the text entry comprises:

obtaining from memory the subsets of the character encoding;

determining that a character in the text entry is encoded with a different subset of the character encoding than a previous character in the text entry; and

incrementing a transition count for the number of character transitions between character sets in the text entry.

7. The method of claim 6, wherein each subset of the character encoding specifies a range of values corresponding to an alphabet, and wherein determining that a character in the text entry is encoded with a different subset of the character encoding than a previous character in the text entry comprises determining that the character is encoded with a value that is outside the range of values corresponding to the subset of the character encoding with which the previous character was encoded.

8. The method of claim 1, wherein labeling the text entry as spam based on the score comprises adding the text entry to a set of suspicious texts for human review to confirm the spam labeling.

9. The method of claim 8, further comprising rendering the text entry and augmenting the display of the text entry to highlight transitions between character sets.

10. The method of claim 1, wherein obtaining a text entry comprises:

isolating visible text from a webpage to remove markup and scripting commands.

11. The method of claim 1, wherein calculating a score indicative of the likelihood that the text entry is spam based on the number of character transitions further comprises:

determining a number of languages in the text entry;

determining a ratio of the number of character transitions between character sets in the text entry and a number of characters in the text entry; and

multiplying the ratio by the number of languages to calculate the score.

12. The method of claim 1, comprising:

ranking the text entry relative to other text entries by down-ranking the text entry responsive to labeling the text entry as spam.

13. The method of claim 1, wherein labeling the text entry as spam based on the score further comprises:

identifying a primary character set in the text entry;

selecting a threshold based on the primary character set; and

determining that the score exceeds the threshold.

14. The method of claim 1, wherein labeling the text entry as spam based on the score further comprises:

determining that the text entry came from a particular website or portion of a website;

selecting a threshold based on the particular website or portion of the website; and

determining that the score exceeds the threshold.

15. The method of claim 14, wherein a threshold for text entries corresponding to a portion of a website containing business listings is lower than a threshold for text entries corresponding to a portion of a website corresponding to user reviews.

16. The method of claim 3, wherein a count of the number of character transitions between at least some Greek or Cyrillic characters and Latin characters are given greater weight than a count of the number of character transitions between characters encoded in other character sets.

17. The method of claim 1, wherein the character sets share at least some characters including at least some whitespace characters.

18. The method of claim 10, further comprising automatically removing the text entry from the webpage in response to labeling the text entry as spam.

19. A tangible, non-transitory, machine-readable medium storing instructions that when executed by a data processing apparatus cause the data processing apparatus to perform operations comprising:

obtaining a text entry;

detecting a number of character transitions between character sets in the text, wherein the character sets (i) each correspond to different alphabets and (ii) are different subsets of a character encoding that maps characters from multiple alphabets to respective binary numbers for use by computers;

calculating, with a computer, a score indicative of the likelihood that the text entry is spam based on the number of character transitions; and

labeling the text entry as spam based on the score.

20. A system, comprising:

one or more processors; and

memory storing instructions that when executed by the processors cause the processors to effectuate operations comprising;

obtaining a text entry;

detecting a number of character transitions between character sets in the text, wherein the character sets (i) each correspond to different alphabets and (ii) are different subsets of a character encoding that maps characters from multiple alphabets to respective binary numbers for use by computers;

calculating, with a computer, a score indicative of the likelihood that the text entry is spam based on the number of character transitions; and

labeling the text entry as spam based on the score.