System and Method of Detecting Offensive Content Sent or Received on a Portable Electronic Device

• US 20170061248A1
• Filed: 09/02/2016
• Published: 03/02/2017
• Est. Priority Date: 09/02/2015
• Status: Active Grant

• Current Assignee: PocketGuardian, LLC
• Original Assignee: James Ronald Ryan, Jr
• Inventors: Ryan, James Ronald Jr., France, Jason Otis, Etter, David Lee
• Granted Patent: US 10,198,667 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/2148 characterised by the process organisation or structure, e.g. boosting cascade
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06F 18/2415 based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/774 Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]
• CPC Class Codes: H04L 51/212 using filtering or selective blocking
• CPC Class Codes: H04L 51/52 for supporting social networking services
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/55 Push-based network services
• CPC Class Codes: H04W 12/02 Protecting privacy or anonymity, e.g. protecting personally identifiable information [PII]

Abstract

The present invention is directed at a system, method and device for detecting offensive content on a portable electronic device, by monitoring communications sent, received or stored on the portable electronic device, and wherein monitoring comprises collecting content data, classifying content data by calculating an alert score for content data wherein an alert score corresponds to offensive content detected, and sending an alert notification to a second portable electronic device to alert the detection of offensive content on the first portable electronic device.

Claims

1. A system for monitoring wireless communications on a portable electronic device to detect offensive content received thereon, comprising:

a first portable electronic device configured with a monitoring mode;

a training module residing on a cloud server in communication with the monitoring mode of the first portable electronic device;

a classification module residing on a classification server configured for classifying content on the first portable electronic device and assigning one of an alert score or non-alert score to content classified; and

a notification module in communication with the classification module and configured to alert a second portable electronic device in response to an alert score.

2. A method for detecting offensive content received on a first portable electronic device and from external data sources, comprising:

identifying content data of a communication sent or received on a first portable electronic device;

sending the content data identified from the first portable electronic device to a classification server;

analyzing the content data against a training data set;

calculating a score of the content data in order to determine an alert score or non-alert score of the content data;

sending, in response to an alert score calculated, an alert notification via a notification module in communication with the classification server to a second portable electronic device, wherein the notification comprises a text message or email message, and wherein the alert score corresponds to the identification of offensive content associated with the content data of the first portable electronic device.

3. The method of claim 2, further comprising:

capturing a candidate set of training content from a plurality of data sources;

filtering the content;

assembling a training data set and extracting one or more of feature vectors from each training data set using a combination of one or more stop words, unigrams and bigrams, or feature count from a pixel-level from each data set using counts based on one or more of color, shape, and texture from the candidate set to generate a matrix representation;

calculating either the probability that a given feature vector is associated to a known sub-category from the training data or a probability estimation based on feature count, to learn a weighted feature vector that separates the positive and negative candidate images;

storing the feature vector and/or probability into a mapping that separates each of the sub-categories from one another, wherein the mapping comprises values identifying an alert score and a non-alert score.

4. The method of claim 2, wherein training content comprises text, video or digital photos.

5. The method of claim 2, further comprising:

upon receiving a new content sample, transforming the new content sample, creating one or more new feature vectors;

retrieving sub-category probability per feature vector, wherein the content corresponds to a particular content; and

for each alert score calculated, sending an alert to a second portable electronic device comprising a notification of the offensive content.

6. The method of claim 2, further comprising:

upon receiving a previously unseen image, the steps of extracting image features, and then performing matrix multiplication to calculate the product of the weighted model feature vector and the input image feature vector; and

for each category of offensive content detected, sending an alert to a second portable electronic device comprising a notification of the offensive content.

7. The method of claim 2, further comprising:

upon receiving a previously unseen video, extracting video frame features, and then performing matrix multiplication to calculate the product of the weighted model feature vector and the input video frame feature vector.

8. The method of claim 2, wherein capturing comprises using one or more of metadata key words, keywords and/or data source as the initial selection criteria

9. A non-transitory computer-readable medium having recorded thereon a program that causes a portable electronic device to execute a method for monitoring content received on the portable electronic device, the method comprising the steps of:

receiving a communication comprising content on a first portable electronic device;

transmitting the content to a classification server;

comparing, on the classification server, the content received on the portable electronic device against a training data set, the training data set containing a collection of content classified as one of an alert score content and non-alert score content;

calculating a score of the content received on the portable electronic device; and

generating an alert notification in response to an alert score calculation, wherein the alert notification is an electronic message sent to a second portable electronic device with a notification of the detection of content corresponding to an alert score.