Detecting content items violating policies of an online system using machine learning based model

• US 10,643,112 B1
• Filed: 03/27/2018
• Issued: 05/05/2020
• Est. Priority Date: 03/27/2018
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Mu, Yang, Olmedilla de la Calle, Daniel
• Primary Examiner(s): Krasnic, Bernard
• Application Number:
  US15/937,708
• Time in Patent Office: 770 Days
• Field of Search: None
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06V 30/19173 Classification techniques
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

An online system distributes content items provided by content providers. The online system determines a likelihood of a content item having deceptive information. The online system stores images showing faces of people in an image database. The online system extracts features from the content item, and provides the extracted features to a machine learning based model configured to generate score indicating whether a content item comprises deceptive information. The machine learning based model uses at least a feature based on matching of faces of users shown in the content item with faces of users shown in the images of the image database. If the online system determines that a content item is deceptive, the online system adds images comprising faces extracted from the content item to the image database to grow the image database.

Claims

1. A computer-implemented method comprising:

receiving, by an online system, a content item for distribution to users of the online system;

extracting features from the content item, the extracting comprising;

extracting a set of images from the content item, each image representing a face;

for each of the set of images, matching a face represented in the image with faces of images stored in an image database, the image database storing images comprising faces; and

setting a feature value based on matching of a face represented in at least one of the images from the set of images with a matching face in a particular image of the image database;

providing the extracted features to a machine learning based model configured to generate a score indicating whether the content item comprises deceptive information;

executing the machine learning based model to generate the score for the content item;

responsive to the generated score indicating that the content item comprises deceptive information;

determining whether the content item conforms to content policies of the online system; and

storing one or more images from the set of images in the image database;

determining a rate of distribution of the content item to users of the online system based on the determination of whether the content item conforms to content policies of the online system; and

sending the content item to users of the online system based on the determined rate of distribution.

2. The computer-implemented method of claim 1, wherein the image database stores facial features corresponding to faces represented in the stored images, wherein matching the face represented in the image with faces of images of the image database comprises:

identifying a first set of facial features of the face represented in the image;

comparing the first set of facial features with a second set of facial features of a face in an image stored in the image database; and

responsive to the comparison indicating that a number of facial features of the first set of facial features matching the facial features of the second set is more than a threshold, determining that the match is successful.

3. The computer-implemented method of claim 2, wherein comparing facial features of the face represented in the image with corresponding facial features of faces in an image stored in the image database comprises matching positions, sizes, shapes, biometrics, or skin textures of the identified facial features against corresponding facial features of faces in an image stored in the image database.

4. The computer-implemented method of claim 1, wherein the machine learning based model determines a weighted aggregate of feature scores corresponding to the extracted features from the content item.

5. The computer-implemented method of claim 1, wherein determining whether the content item conforms to content policies of the online system comprises:

comparing the generated score with a plurality of ranges of scores, each range of scores associated with an action that is performed by the online system in association with the content item; and

selecting an action corresponding to a range of scores that include the generated score; and

performing the selected action.

6. The computer-implemented method of claim 5, wherein the action is one or more of:

disabling or deleting the content item, or sending the content item for review.

7. The computer-implemented method of claim 1, further comprising, training the machine learning based model, the training comprising:

receiving a training dataset comprising a plurality of content items labeled as one of;

deceptive or not deceptive; and

training the model based on the training dataset.

8. The computer-implemented method of claim 1, wherein determining the rate of distribution of the content item to users of the online system based on the determination of whether the content item conforms to content policies of the online system comprises:

determining that the content item violates a content policy of the online system; and

reducing the rate of distribution of the content item, comprising, withholding the content item from users that satisfy certain criteria associated with the violated content policy and continuing to provide the content item to other users.

9. The computer-implemented method of claim 1, further comprising:

determining a deceptive information score for each of one or more images stored in the image database, the deceptive information score indicative of a likelihood that a content item including the image is deceptive; and

providing the deceptive information score of the particular image as another feature to the machine learning model.

10. The computer-implemented method of claim 9, wherein the deceptive information score for the particular image showing the matching face is based on a number of content items identified as being deceptive that show the matching face.

11. The computer-implemented method of claim 10, wherein the deceptive information score for the image showing the face is a weighted aggregate value based on the content items showing the face identified in the image, wherein each content item is weighted by a rate at which the content item is presented to users.

12. A non-transitory computer readable storage medium storing instructions which when executed by a computer processor, cause the computer processor to perform steps comprising:

receiving, by an online system, a content item for distribution to users of the online system;

extracting features from the content item, the extracting comprising;

extracting a set of images from the content item, each image representing a face;

for each of the set of images, matching a face represented in the image with faces of images stored in an image database, the image database storing images comprising faces; and

setting a feature value based on matching of a face represented in at least one of the images from the set of images with a matching face in a particular image of the image database;

providing the extracted features to a machine learning based model configured to generate a score indicating whether the content item comprises deceptive information;

executing the machine learning based model to generate the score for the content item;

responsive to the generated score indicating that the content item comprises deceptive information;

determining whether the content item conforms to content policies of the online system; and

storing one or more images from the set of images in the image database;

determining a rate of distribution of the content item to users of the online system based on the determination of whether the content item conforms to content policies of the online system; and

sending the content item to users of the online system based on the determined rate of distribution.

13. The non-transitory computer readable storage medium of claim 12, wherein the image database stores facial features corresponding to faces represented in the stored images, wherein instructions for matching the face represented in the image with faces of images of the image database comprise instructions for:

identifying a first set of facial features of the face represented in the image;

comparing the first set of facial features with a second set of facial features of a face in an image stored in the image database; and

responsive to the comparison indicating that a number of facial features of the first set of facial features matching the facial features of the second set is more than a threshold, determining that the match is successful.

14. The non-transitory computer readable storage medium of claim 12, wherein the machine learning based model determines a weighted aggregate of feature scores corresponding to the extracted features from the content item.

15. The non-transitory computer readable storage medium of claim 12, wherein instructions for determining whether the content item conforms to content policies of the online system comprise instructions for:

comparing the generated score with a plurality of ranges of scores, each range of scores associated with an action that is performed by the online system in association with the content item; and

selecting an action corresponding to a range of scores that include the generated score; and

performing the selected action.

16. The non-transitory computer readable storage medium of claim 12, wherein instructions for training the machine learning based model comprise instructions for:

receiving a training dataset comprising a plurality of content items labeled as one of;

deceptive or not deceptive; and

training the model based on the training dataset.

17. The non-transitory computer readable storage medium of claim 12, wherein instructions for determining the rate of distribution of the content item to users of the online system based on the determination of whether the content item conforms to content policies of the online system comprise instructions for:

determining that the content item violates a content policy of the online system; and

reducing the rate of distribution of the content item, comprising, withholding the content item from users that satisfy certain criteria associated with the violated content policy and continuing to provide the content item to other users.

18. The non-transitory computer readable storage medium of claim 12, further comprising:

determining a deceptive information score for each of one or more images stored in the image database, the deceptive information score indicative of a likelihood that a content item including the image is deceptive; and

providing the deceptive information score of the particular image as another feature to the machine learning model.

19. The non-transitory computer readable storage medium of claim 18, wherein the deceptive information score for the particular image showing the matching face is based on a number of content items identified as being deceptive that show the matching face.

20. The non-transitory computer readable storage medium of claim 19, wherein the deceptive information score for the image showing the face is a weighted aggregate value based on the content items showing the face identified in the image, wherein each content item is weighted by a rate at which the content item is presented to users.