Online active learning in user-generated content streams

• US 9,967,218 B2
• Filed: 10/26/2011
• Issued: 05/08/2018
• Est. Priority Date: 10/26/2011
• Status: Active Grant

• Current Assignee: Yahoo Assets LLC
• Original Assignee: Oath Inc. (Verizon Communications Inc.)
• Inventors: Chu, Wei, Zinkevich, Martin, Li, Lihong, Thomas, Achint Oommen, Tseng, Belle
• Primary Examiner(s): Mejia, Anthony
• Assistant Examiner(s): Shah, Mehulkumar
• Application Number:
  US13/282,285
• Publication Number:
  US 20130111005A1
• Time in Patent Office: 2,386 Days
• Field of Search: 709224, 709223, 370252, 714 39
• CPC Class Codes: G06F 11/00 Error detection; Error correction; Monitoring error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00
• CPC Class Codes: G06F 15/16 Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs coordinating program control therefor G06F9/52; in regulating and control system G05B
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 7/00 Computing arrangements based on specific mathematical models
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 50/20 Education
• CPC Class Codes: H04L 51/212 using filtering or selective blocking

Abstract

Software for online active learning receives content posted to an online stream at a website. The software converts the content into an elemental representation and inputs the elemental representation into a probit model to obtain a predictive probability that the content is abusive. The software also calculates an importance weight based on the elemental representation. And the software updates the probit model using the content, the importance weight, and an acquired label if a condition is met. The condition depends on an instrumental distribution. The software removes the content from the online stream if a condition is met. The condition depends on the predictive probability, if an acquired label is unavailable.

Claims

1. A method for delivering modified user generated content for display on client devices, comprising the operations of:

receiving, at one or more servers over a network, content that is user generated content from an online stream at a website, the content including text;

converting, by a machine process executed at the one or more servers, the content into an elemental representation using a bag of words model;

applying a probit model to the elemental representation to obtain a predictive probability that the content is abusive or not abusive, the machine process further includes, calculating an importance weight for the probit model based on the elemental representation, the importance weight is modeled as a multivariate Gaussian distribution with a mean and a covariance matrix;

creating a probabilistic queue for delivering the content to a human labeler for acquiring a label for the content, wherein placement of the content within the probabilistic queue depends on the predictive probability that the content is abusive or not abusive;

updating the probit model using the elemental representation, the importance weight, and the label acquired from the human labeler, the updating the probit model includes calculating an updated mean and an updated covariance matrix for the multivariate Gaussian distribution of the importance weight based on the label;

receiving, at the one or more servers, a request from a client device for the online stream at the website, the online stream including the content;

applying the probit model having been updated to the content and removing the content from the online stream to produce a modified online stream, the removing is based on the predictive probability that the content is abusive as calculated by the probit model having been updated; and

sending, from the one or more servers, the modified online stream to the client device for display.

2. The method of claim 1, wherein the content further includes an image and wherein the elemental representation is further based at least in part on a bag of features model.

3. The method of claim 1, wherein the importance weight is further based on entropy or on function values.

4. The method of claim 1, wherein the probit model includes a memory loss factor.

5. The method of claim 1, wherein when the abusive content includes spam, fraudulent offers, illegal offers, offensive language, threatening language, or treasonous language.

6. The method of claim 1, wherein the delivering the content to the human labeler includes pushing the content to the human labeler or having the human labeler pull the content.

7. A method for preventing spread of abusive content to client devices, comprising:

receiving content that is user generated from an online stream at a website, the content including text;

converting the content into elemental representation;

applying a probit model to the elemental representation to obtain a predictive probability that the content is abusive or not abusive, the applying the probit model includes calculating an importance weight based on the elemental representation, the probit model includes a memory loss factor associated with the importance weight, the importance weight is modeled as a multivariate Gaussian distribution with a mean and a covariance matrix;

creating a probabilistic queue for delivering the content to a human labeler for acquiring a label for the content indicative of whether the content is abusive or not abusive, the content is inserted into the probabilistic queue depending on the predictive probability that the content is abusive or not abusive;

updating the probit model using the elemental representation, the importance weight, the memory loss factor, and the label acquired from the human labeler, the updating includes calculating an updated mean and an updated covariance matrix for the multivariate Gaussian distribution of the importance weight based on the label;

receiving a request from a client device for the online stream at the website, the online stream including the content;

applying the probit model having been updated to the content, the applying the probit model having been updated includes,

removing the content from the online stream if the predictive probability is determined to be abusive as calculated by the probit model having been updated, or

not removing the content from the online stream if the predictive probability is determined to be not abusive as calculated by the probit model having been updated; and

sending the online stream to the client device.

8. The method of claim 7, wherein the converting the content into the elemental representation includes using a bag of words model, wherein words of the content are represented as uni-grams, bi-grams, n-grams, or skip-grams in the bag of words model.

9. The method of claim 7, wherein the converting the content into the elemental representation includes generating a vector using term frequency-inverse document frequency.

10. The method of claim 7, wherein when the abusive content includes spam, fraudulent offers, illegal offers, offensive language, threatening language, or treasonous language.

11. The method of claim 7, wherein the delivering the content to the human labeler includes pushing the content to the human labeler or having the human labeler pull the content.

12. A computer-readable storage medium that is non-transitory and that stores a program for delivering modified user generated content for display on client devices, wherein the program, when executed, instructs a processor to perform the following operations:

receive, at one or more servers, content posted to an online stream at a website;

convert, by a machine process, the content into an elemental representation using a bag of words model;

apply the probit model to the elemental representation to obtain a predictive probability that the content is abusive, the machine process further includes,

calculate an importance weight for the probit model based on the elemental representation, the importance weight is defined by a multivariate Gaussian distribution defined by a mean and a covariance matrix;

create a probabilistic queue for delivering the content to a human labeler for acquiring a label for the content, wherein placement of the content within the probabilistic queue depends on the predictive probability that the content is abusive or not abusive;

update the probit model with the elemental representation and the importance weight, and the label acquired from the human labeler, said update the probit model includes calculating, based on the label, an updated mean and an updated covariance matrix that define the multivariate Gaussian distribution that defines the importance weight;

receive, at the one or more servers, a request from a client device for the online stream at the website, the online stream including the content;

apply the probit model having been updated to the content to remove the content from the online stream to produce a modified online stream, said remove the content is based on the predictive probability that the content is abusive as calculated by the probit model having been updated; and

send, from the one or more servers, the modified online stream to the client device for display.

13. The computer-readable storage medium of claim 12, wherein the content further includes an image and wherein the elemental representation is further based at least in part on a bag of features model.

14. The computer-readable storage medium of claim 12, wherein the importance weight is further based on entropy or on function values.

15. The computer-readable storage medium of claim 12, wherein the probit model includes a memory loss factor.

16. The computer-readable storage medium of claim 12, wherein when the abusive content includes spam, fraudulent offers, illegal offers, offensive language, threatening language, or treasonous language.

17. The computer readable-storage medium of claim 12, wherein the delivering the content to the human labeler includes pushing the content to the human labeler or having the human labeler pull the content.