Spam detection and prevention in a social networking system

• US 9,537,814 B2
• Filed: 12/14/2012
• Issued: 01/03/2017
• Est. Priority Date: 12/14/2012
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Rubinstein, Yigal Dan, Wiseman, Josh, Choi, Matthew Kai-Shing
• Primary Examiner(s): Nguyen, Tammy
• Application Number:
  US13/714,776
• Publication Number:
  US 20140172989A1
• Time in Patent Office: 1,481 Days
• Field of Search: 709/206, 709/220, 726/11
• US Class Current: 1/1
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: H04L 51/212 using filtering or selective blocking
• CPC Class Codes: H04L 51/52 for supporting social networking services

Abstract

A social networking system detects spam in comments posted by users of the social networking system. A spam prevention module of the social networking system retrieves content signals associated with a posted comment and analyzes the content signals to determine whether the posted comment contains spam content. The spam prevention module augments the content signal analysis by analyzing social signals of the posted comment. Based on the content signal analysis and the social signal analysis, the spam prevention module determines whether the posted comment is spam and takes remedial actions on detected spam, including blocking the comment and educating the user who posted the comment.

Claims

1. A computer implemented method, comprising:

receiving a comment by a user to a post posted by another user of a social networking environment;

analyzing one or more content signals of the comment, the content signals indicating whether the comment contains spam;

analyzing one or more social signals of the comment, wherein the social signals of the comment comprise information on social relations and social activities of the user in the social networking environment, and the social signals of the comment indicate whether the comment contains spam, wherein analyzing one or more social signals of the comment comprises;

identifying one or more common patterns of unacceptable behavior of the user who posted the comment and

generating one or more aggregate social signals associated with the comment based on the identified one or more common patterns of unacceptable behavior of the user;

determining whether the comment contains spam based on the analysis of the content signals and the analysis of the social signals associated with the comment; and

responsive to the determination that the comment is spam, taking remedial action regarding the comment.

2. The computer implemented method of claim 1, wherein the content signals comprise at least one of entropy of the comment, frequency of posting of the comment by the user and similarity of the comment with other comments posted by the user.

3. The computer implemented method of claim 1, wherein the social signals associated with the comment include at least one of:

a number of viewers that are directly connected to the user who posted the post;

a number of viewers that are indirectly connected to the user who posted the post;

an average length of connection between the user who posted the comment and the user who posted the post;

an average geographical distance between the user who posted the comment and the user who posted the post;

a number of “

likes”

on the comment among viewers of the comment;

a number of viewers who reported the comment as spam or annoying; and

a number of shared connections among the user who posted the post, the user who posted the comment on the post and viewers of the comment.

4. The computer implemented method of claim 1, wherein analyzing the content signals comprises:

applying a trained content feature extraction model to the content signals of the comment, the content feature extraction model being trained on a plurality of content signals of a corpus of comments received by the social networking system; and

determining whether the comment contains one or more common patterns of inappropriate or spam content based on content features of the comment.

5. The computer implemented method of claim 4, further comprising:

generating one or more aggregate content signals based on the analysis of the content signals, an aggregate content signal indicating whether the comment contains at least some inappropriate or spam content.

6. The computer implemented method of claim 1, wherein

the one or more common patterns of unacceptable behavior of the user who posted the comment are based on the number of “

likes”

from viewers of the comment.

7. The computer implemented method of claim 1, wherein the remedial action comprises at least one of:

blocking the comment that is spam;

adding a user account of the user who posted the comment that is spam to a black list;

quarantining the comment that is spam; and

communicating with the user who posted the comment or is about to post the comment to avoid posting spam content.

8. The computer implemented method of claim 1, wherein taking the remedial action comprises:

blocking the comment responsive to the comment having a link to a website known to compromise a visitor'"'"'s confidential information.

9. The computer implemented method of claim 1, wherein taking the remedial action further comprises:

responsive to the user who posted the comment using a fake user account, adding the fake user account to a black list, which contains a list of users whose comments are blocked.

10. The computer implemented method of claim 1, wherein taking the remedial action further comprises:

responsive to the user having posted identical or similar comments, presenting the user with a communication interface for the user to take a user action related to the comment.

11. A non-transitory computer-readable storage medium storing computer program instructions, executed by a computer processor, for detecting spam in a social networking system, the computer program instructions comprising instructions for:

receiving a comment by a user to a post posted by another user of a social networking environment;

analyzing one or more content signals of the comment, the content signals indicating whether the comment contains spam;

analyzing one or more social signals of the comment, wherein the social signals of the comment comprise information on social relations and social activities of the user, and the social signals of the comment indicate whether the comment contains spam, wherein analyzing one or more social signals of the comment comprises;

identifying one or more common patterns of unacceptable behavior of the user who posted the comment and

generating one or more aggregate social signals associated with the comment based on the identified one or more common patterns of unacceptable behavior of the user;

determining whether the comment contains spam based on the analysis of the content signals and the analysis of the social signals associated with the comment; and

responsive to the determination that the comment is spam, taking remedial action regarding the comment.

12. The computer-readable storage medium of claim 11, wherein the content signals comprise at least one of entropy of the comment, frequency of posting of the comment by the user and similarity of the comment with other comments posted by the user.

13. The computer-readable storage medium of claim 11, wherein the social signals associated with the comment include at least one of:

a number of viewers that are directly connected to the user who posted the post;

a number of viewers that are indirectly connected to the user who posted the post;

an average length of connection between the user who posted the comment and the user who posted the post;

an average geographical distance between the user who posted the comment and the user who posted the post;

a number of “

likes”

on the comment among viewers of the comment;

a number of viewers who reported the comment as spam or annoying; and

a number of shared connections among the user who posted the post, the user who posted the comment on the post and viewers of the comment.

14. The computer-readable storage medium of claim 11, wherein the computer program instructions for analyzing the content signals comprise computer program instructions for:

applying a trained content feature extraction model to the content signals of the comment, the content feature extraction model being trained on a plurality of content signals of a corpus of comments received by the social networking system; and

determining whether the comment contains one or more common patterns of inappropriate or spam content based on content features of the comment.

15. The computer-readable storage medium of claim 14, further comprising computer program instructions for:

generating one or more aggregate content signals based on the analysis of the content signals, an aggregate content signal indicating whether the comment contains at least some inappropriate or spam content.

16. The computer-readable storage medium of claim 11, wherein the one or more common patterns of unacceptable behavior of the user who posted the comment are based on the number of “

likes”

from viewers of the comment.

17. The computer-readable storage medium of claim 11, wherein the remedial action comprises at least one of:

blocking the comment that is spam;

adding a user account of the user who posted the comment that is spam to a black list;

quarantining the comment that is spam; and

communicating with the user who posted the comment or is about to post the comment to avoid posting spam content.

18. The computer-readable storage medium of claim 11, wherein the computer program instructions for taking the remedial action comprise computer program instruction for:

blocking the comment responsive to the comment having a link to a website known to compromise a visitor'"'"'s confidential information.

19. The computer-readable storage medium of claim 11, wherein the computer program instructions for taking the remedial action further comprise computer program instructions for:

responsive to the user who posted the comment using a fake user account, adding the fake user account to a black list, which contains a list of users whose comments are blocked.

20. The computer-readable storage medium of claim 11, wherein the computer program instructions for taking the remedial action further comprise computer program instructions for:

responsive to the user having posted identical or similar comments, presenting the user with a communication interface for the user to take a user action related to the comment.