Iterative Classifier Training on Online Social Networks

• US 20160155063A1
• Filed: 12/01/2014
• Published: 06/02/2016
• Est. Priority Date: 12/01/2014
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Rich, Mark Andrew
• Granted Patent: US 10,552,759 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/20 Ensemble learning
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06Q 50/01 Social networking

Abstract

In one embodiment, a method includes accessing a first set of objects associated with an online social network, each object being associated with one or more comments. The method also includes generating a second set of objects from the first set of objects by applying a first filtering criteria to the first set of objects and scoring each object in the second set of objects based on the comments associated with each object. The method further includes generating a training set of objects from the second set of objects by selecting each object from the second set of objects having a score greater than a first threshold score, each object in the training set being associated with a first object-classification. The method further includes determining an object-classifier algorithm for the first object-classification, the object-classifier algorithm being determined through an iterative training process performed one or more times.

Claims

1. A method comprising, by one or more computing devices:

accessing a first set of objects associated with an online social network, each object being associated with one or more comments;

generating a second set of objects from the first set of objects by applying a first filtering criteria to the first set of objects;

scoring each object in the second set of objects based on the comments associated with each object;

generating a training set of objects from the second set of objects by selecting each object from the second set of objects having a score greater than a first threshold score, each object in the training set being associated with a first object-classification; and

determining an object-classifier algorithm for the first object-classification, the object-classifier algorithm being determined through an iterative training process performed one or more times, each iteration of the iterative training process comprising;

training an initial object-classifier algorithm based on the comments associated with the objects in the training set of objects;

accessing a third set of objects associated with the online social network;

classifying, using the initial object-classifier algorithm, each object in the third set of objects based on an analysis of the comments associated with each object, one or more of the objects in the third set of objects being classified with the first object-classification;

training a revised object-classifier algorithm based on the comments associated with the objects in the third set of objects having the first object-classification;

accessing a fourth set of objects associated with the online social network, the fourth set of objects being generated by applying a second filtering criteria to a fifth set of objects associated with the online social network;

classifying, using the revised object-classifier algorithm, each object in the fourth set of objects based on an analysis of the comments associated with each object, one or more objects in the fourth set of objects being classified with the first object-classification; and

generating a sixth set of objects from the fourth set of objects by selecting each object from the fourth set of objects having a score greater than a second threshold score, each object in the sixth set of objects being associated with the first object-classification, wherein the sixth set of objects is to be used as the training set in a next iteration of the iterative training process.

2. The method of claim 1, further comprising:

accessing a social graph comprising a plurality of nodes and a plurality of edges connecting the nodes, each node corresponding to an object associated with the online social network.

3. The method of claim 2, wherein, for each object in the first set of objects, the object is a post by a first user of the online social network corresponding to a first node of the plurality of nodes, and wherein each comment associated with the object is a comment on the post by a second user of the online social network, each second user corresponding to a second node, each second node being within a single degree of separation from the first node in the social graph.

4. The method of claim 1, wherein the objects of the first set of objects are each of an object-type selected from a group consisting of:

photos, posts, pages, applications, events, locations, or user groups.

5. The method of claim 1, wherein:

applying the first filtering criteria to the first set of objects comprises comparing metadata associated with each object in the first set of objects to a set of filtering criteria; and

generating the second set of objects comprises adding objects from the first set of objects to the second set based on the comparison of the metadata to the filtering criteria.

6. The method of claim 5, wherein the metadata comprises one or more of:

social-graph information associated with the object, other objects associated with the object, a language associated with the object, a time or date associated with the object, or any combination thereof.

7. The method of claim 6, wherein the social-graph information comprises an affinity coefficient between first and second users of the online social network, the first user being associated with an object in the first set of objects.

8. The method of claim 1, wherein scoring each object in the second set of objects comprises determining a score for each comment associated with the object by comparing the comment to a set of regular expressions associated with the first object-classification.

9. The method of claim 1, wherein scoring each object in the second set of objects comprises:

determining, for each object in the second set of objects, a score for each comment associated with the object; and

combining, for each object in the second set of objects, the scores for the comments to produce an object score.

10. The method of claim 9, wherein the object score is an average of the scores for the comments.

11. The method of claim 1, wherein the first threshold score is greater than 90% of the scores of the objects in the second set of objects.

12. The method of claim 1, wherein training the initial object-classifier algorithm comprises:

analyzing the comments associated with the objects in the training set to determine a set of features;

scoring each feature in the set of features based on a frequency of occurrence of the feature in the comments associated with the objects in the training set; and

modifying the initial object-classifier algorithm to associate at least in part the first object-classification with one or more of the features in the set of features having a score greater than a threshold feature score.

13. The method of claim 1, wherein classifying each object in the third set of objects comprises comparing the comments associated with an object in the third set to features associated with the initial object-classifier algorithm to determine whether the object is classified with the first object-classification.

14. The method of claim 1, wherein training the revised object-classifier algorithm comprises:

analyzing the comments associated with the objects in the third set of objects having the first object-classification to determine a set of revised features;

scoring each revised feature in the set of revised features based on a frequency of occurrence of the feature in the comments associated with the objects in the third set of objects having the first object-classification; and

modifying the revised object-classifier algorithm to associate at least in part the first object-classification with one or more of the features in the set of revised features having a score greater than a threshold feature score.

15. The method of claim 1, wherein classifying each object in the fourth set of objects comprises comparing the comments associated with an object in the fourth set to features associated with the revised object-classifier algorithm to determine whether the object is classified with the first object-classification.

16. The method of claim 1, wherein training the initial object-classifier algorithm is further based on the revised object-classifier algorithm trained in a prior iteration of the iterative training process.

17. The method of claim 1, wherein one or more of the first, third, or fifth sets of objects each comprise objects that are randomly selected from objects associated with the online social network.

18. The method of claim 1, wherein the first, third, and fifth sets of objects each comprise greater than or equal to one million objects.

19. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

access a first set of objects associated with an online social network, each object being associated with one or more comments;

generate a second set of objects from the first set of objects by applying a first filtering criteria to the first set of objects;

score each object in the second set of objects based on the comments associated with each object;

generate a training set of objects from the second set of objects by selecting each object from the second set of objects having a score greater than a first threshold score, each object in the training set being associated with a first object-classification; and

determine an object-classifier algorithm for the first object-classification, the object-classifier algorithm being determined through an iterative training process performed one or more times, each iteration of the iterative training process comprising;

train an initial object-classifier algorithm based on the comments associated with the objects in the training set of objects;

access a third set of objects associated with the online social network;

classify, using the initial object-classifier algorithm, each object in the third set of objects based on an analysis of the comments associated with each object, one or more of the objects in the third set of objects being classified with the first object-classification;

train a revised object-classifier algorithm based on the comments associated with the objects in the third set of objects having the first object-classification;

access a fourth set of objects associated with the online social network, the fourth set of objects being generated by applying a second filtering criteria to a fifth set of objects associated with the online social network;

classify, using the revised object-classifier algorithm, each object in the fourth set of objects based on an analysis of the comments associated with each object, one or more objects in the fourth set of objects being classified with the first object-classification; and

generate a sixth set of objects from the fourth set of objects by selecting each object from the fourth set of objects having a score greater than a second threshold score, each object in the sixth set of objects being associated with the first object-classification, wherein the sixth set of objects is to be used as the training set in a next iteration of the iterative training process.

20. A system comprising:

one or more processors; and

a memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to;

access a first set of objects associated with an online social network, each object being associated with one or more comments;

generate a second set of objects from the first set of objects by applying a first filtering criteria to the first set of objects;

score each object in the second set of objects based on the comments associated with each object;

generate a training set of objects from the second set of objects by selecting each object from the second set of objects having a score greater than a first threshold score, each object in the training set being associated with a first object-classification; and

determine an object-classifier algorithm for the first object-classification, the object-classifier algorithm being determined through an iterative training process performed one or more times, each iteration of the iterative training process comprising;

train an initial object-classifier algorithm based on the comments associated with the objects in the training set of objects;

access a third set of objects associated with the online social network;

classify, using the initial object-classifier algorithm, each object in the third set of objects based on an analysis of the comments associated with each object, one or more of the objects in the third set of objects being classified with the first object-classification;

train a revised object-classifier algorithm based on the comments associated with the objects in the third set of objects having the first object-classification;

access a fourth set of objects associated with the online social network, the fourth set of objects being generated by applying a second filtering criteria to a fifth set of objects associated with the online social network;

classify, using the revised object-classifier algorithm, each object in the fourth set of objects based on an analysis of the comments associated with each object, one or more objects in the fourth set of objects being classified with the first object-classification; and

generate a sixth set of objects from the fourth set of objects by selecting each object from the fourth set of objects having a score greater than a second threshold score, each object in the sixth set of objects being associated with the first object-classification, wherein the sixth set of objects is to be used as the training set in a next iteration of the iterative training process.