Online Active Learning in User-Generated Content Streams

• US 20130111005A1
• Filed: 10/26/2011
• Published: 05/02/2013
• Est. Priority Date: 10/26/2011
• Status: Active Grant

• Current Assignee: Yahoo Assets LLC
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Zinkevich, Martin, Li, Lihong, Thomas, Achint Oommen, Tseng, Belle, Chu, Wei
• Granted Patent: US 9,967,218 B2
• Time in Patent Office: Days
• US Class Current: 709/224
• CPC Class Codes: G06F 11/00 Error detection; Error correction; Monitoring error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00
• CPC Class Codes: G06F 15/16 Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs coordinating program control therefor G06F9/52; in regulating and control system G05B
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 7/00 Computing arrangements based on specific mathematical models
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 50/20 Education
• CPC Class Codes: H04L 51/212 using filtering or selective blocking

Abstract

Software for online active learning receives content posted to an online stream at a website. The software converts the content into an elemental representation and inputs the elemental representation into a probit model to obtain a predictive probability that the content is abusive. The software also calculates an importance weight based on the elemental representation. And the software updates the probit model using the content, the importance weight, and an acquired label if a condition is met. The condition depends on an instrumental distribution. The software removes the content from the online stream if a condition is met. The condition depends on the predictive probability, if an acquired label is unavailable.

Claims

1. A method for monitoring user generated content, comprising the operations of:

receiving content posted to an online stream at a website;

converting the content into an elemental representation;

inputting the elemental representation into a probit model to obtain a predictive probability that the content is abusive;

calculating an importance weight based at least in part on the elemental representation;

updating the probit model using the elemental representation, an importance weight, and an acquired label, wherein the updating occurs if a condition is met and wherein the condition depends at least in part on an instrumental distribution; and

removing the content from the online stream, based at least in part on the predictive probability if an acquired label is unavailable, wherein each operation of the method is executed by one or more processors.

2. The method of claim 1, wherein the condition depends at least in part on a randomized value.

3. The method of claim 1, wherein the content is text and wherein the elemental representation is based at least in part on a bag of words model.

4. The method of claim 1, wherein the content is an image and wherein the elemental representation is based at least in part on a bag of features model.

5. The method of claim 1, wherein the instrumental distribution is based on entropy.

6. The method of claim 1, wherein the instrumental distribution is based on function values.

7. The method of claim 1, wherein the acquired label is received from an editor who pulled the content from a probabilistic queue.

8. The method of claim 1, wherein the probit model includes a memory loss factor.

9. The method of claim 1, wherein updating the probit model includes updating a mean and updating a covariance matrix.

10. A computer-readable storage medium persistently storing a program, wherein the program, when executed, instructs a processor to perform the following operations:

receive content posted to an online stream at a website;

convert the content into an elemental representation;

input the elemental representation into a probit model to obtain a predictive probability that the content is abusive;

calculate an importance weight based at least in part on the elemental representation;

update the probit model with the elemental representation, the importance weight, and an acquired label, wherein the update occurs if a condition is met and wherein the condition depends at least in part on an instrumental distribution; and

remove the content from the online stream, based at least in part on the predictive probability if an acquired label is unavailable.

11. The computer-readable storage medium of claim 10, wherein the condition depends at least in part on a randomized value.

12. The computer-readable storage medium of claim 10, wherein the content is text and wherein the elemental representation is based at least in part on a bag of words model.

13. The computer-readable storage medium of claim 10, wherein the content is an image and wherein the elemental representation is based at least in part on a bag of features model.

14. The computer-readable storage medium of claim 10, wherein the importance weight is based on entropy.

15. The computer-readable storage medium of claim 10, wherein the importance weight is based on function values.

16. The computer-readable storage medium of claim 10, wherein the acquired label is received from an editor who pulled the content from a probabilistic queue.

17. The computer-readable storage medium of claim 10, wherein the probit model includes a memory loss factor.

18. The computer-readable storage medium of claim 10, wherein updating the probit model includes updating a mean and updating a covariance matrix.

19. A method for displaying user generated content, comprising the operations of:

receiving content posted to an online stream at a website;

converting the content into an elemental representation;

inputting the elemental representation into a probit model to obtain a predictive probability that the content is interesting;

calculating an importance weight based at least in part on the elemental representation;

updating the probit model with the elemental representation, the importance weight, and an acquired label, wherein the updating occurs if a condition is met and wherein the condition depends at least in part on an instrumental distribution; and

relocating the content in the online stream, based at least in part on the predictive probability if an acquired label is unavailable, wherein each operation of the method is executed by one or more processors.

20. The method of claim 19, wherein the probit model includes a memory loss factor.