METHODS AND SYSTEMS OF CLASSIFYING SPAM URL

• US 20140324741A1
• Filed: 04/29/2013
• Published: 10/30/2014
• Est. Priority Date: 04/29/2013
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Stewart, Allan, Zarakhovsky, Eugene, Palow, Christopher, Gowda, Chetan, Dorman, Brent
• Granted Patent: US 9,378,465 B2
• Time in Patent Office: Days
• US Class Current: 706/12
• CPC Class Codes: G06N 20/00 Machine learning

Abstract

A method of operation of a URL spam detection system includes: identifying a feature dimension of a user action on a social networking system to detect anomalies; extracting URL chunks from a content associated with the user action; aggregating a non-content feature of the user action along the feature dimension into a URL distribution store to produce a feature distribution for each of the URL chunks; determining whether the feature distribution of a particular URL chunk within the URL chunks exceeds an expectation threshold for the feature dimension; and classifying the particular URL chunk as an illegitimate URL when the feature distribution exceeds the expectation threshold to restrict access to a particular URL chunk on a social networking system.

Claims

1. A method comprising:

identifying a feature dimension of a user action on a social networking system to detect anomalies;

extracting URL chunks from content associated with the user action;

aggregating a non-content feature of the user action along the feature dimension into a URL distribution store to produce a feature distribution for each of the URL chunks;

determining whether the feature distribution of a particular URL chunk within the URL chunks exceeds an expectation threshold for the feature dimension; and

classifying the particular URL chunk as an illegitimate URL when the feature distribution exceeds the expectation threshold to restrict access to a particular URL chunk on a social networking system.

2. The method of claim 1, wherein identifying the feature dimension includes identifying the feature dimension of a content sharing action to disseminate the content.

3. The method of claim 1, wherein identifying the feature dimension includes identifying the feature dimension of an association action of a user account to associate with the content.

4. The method of claim 1, wherein identifying the feature dimension includes identifying the feature dimension of an indirect association action of a user account to associate with a social object affiliated with the content or the particular URL chunk.

5. The method of claim 1, wherein aggregating the non-content feature to produce the feature distribution includes aggregating along a time window wherein the feature distribution is a moving distribution along the feature dimension.

6. The method of claim 1, further comprising determining the expectation threshold by machine learning against known reliable URL chunks and known spam URL chunks.

7. The method of claim 1, further comprising determining the expectation threshold by machine learning against known spammer user accounts and known reliable user accounts.

8. The method of claim 1, wherein the feature distribution is a binomial distribution of whether the non-content feature exists for the user action.

9. The method of claim 1, wherein the feature distribution is a discrete distribution of enumerated states along the feature dimension.

10. The method of claim 1, wherein the feature distribution is a continuous distribution along the feature dimension.

11. The method of claim 1, wherein extracting the URL chunks includes extracting the URL chunks from an embedded URL and one or more redirects of the embedded URL, the URL chunks being one or more subsets of the URL delimited by one or more punctuations.

12. The method of claim 11, wherein classifying the particular URL chunk is based on classification of a related URL chunk in a sibling family tree of the particular URL chunk, the sibling family tree and the particular URL sharing a parent domain URL chunk.

13. A method comprising:

identifying a feature dimension of a user action on a social networking system, the feature dimension related to a user account responsible for the user action;

extracting URL chunks from content associated with the user action;

aggregating a sender feature of the user action along the feature dimension into a URL distribution store to produce a feature distribution for each of the URL chunks;

detecting an anomaly in the feature distribution of a particular URL chunk within the URL chunks as compared to an expected distribution along the feature dimension; and

raising a suspicion level of the particular URL chunk when the anomaly is detected.

14. The method of claim 13, wherein the expected distribution is a superset feature distribution of a superset URL chunk containing the particular URL chunk.

15. The method of claim 13, wherein the expected distribution is a white list feature distribution of known reliable URL chunks.

16. The method of claim 13, wherein raising the suspicion level includes raising the suspicion level when a pre-defined number of anomalies are detected along multiple feature dimensions.

17. The method of claim 13, wherein raising the suspicion level includes classifying the particular URL chunk under a specific type of illegitimate sharing channel.

18. The method of claim 13, wherein raising the suspicion level includes storing the suspicion level associated with the particular URL chunk in a classification table for a filter module restricting execution of the user action.

19. The method of claim 13, further comprising:

tracking the feature distribution to determine whether the anomaly subsides within an acceptable range similar to the expected distribution; and

lowering the suspicion level when the anomaly subsides.

20. A processor-based system comprising:

a feature collector module stored on a non-transitory memory, when executed by a processor is configured to;

identify a feature dimension of a user action on a social networking system, the feature dimension related to a user account responsible for the user action;

extract URL chunks from content associated with the user action;

aggregate a sender feature of the user action along the feature dimension into a URL distribution store to produce a feature distribution for each of the URL chunks; and

a URL classifier module stored on a non-transitory memory, when executed by a processor is coupled to the feature collection module via the URL distribution store and configured to;

detect an anomaly in the feature distribution of a particular URL chunk within URL chunks as compared to an expected distribution; and

raise a suspicion level of the particular URL chunk when the anomaly is detected.