System and method of detecting offensive content sent or received on a portable electronic device

• US 10,198,667 B2
• Filed: 09/02/2016
• Issued: 02/05/2019
• Est. Priority Date: 09/02/2015
• Status: Active Grant

• Current Assignee: PocketGuardian, LLC
• Original Assignee: PocketGuardian, LLC
• Inventors: Ryan, Jr., James Ronald, France, Jason Otis, Etter, David Lee
• Primary Examiner(s): Lefkowitz, Sumati
• Assistant Examiner(s): Perlman, David
• Application Number:
  US15/255,918
• Publication Number:
  US 20170061248A1
• Time in Patent Office: 886 Days
• Field of Search: None
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/2148 characterised by the process organisation or structure, e.g. boosting cascade
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06F 18/2415 based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/774 Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]
• CPC Class Codes: H04L 51/212 using filtering or selective blocking
• CPC Class Codes: H04L 51/52 for supporting social networking services
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/55 Push-based network services
• CPC Class Codes: H04W 12/02 Protecting privacy or anonymity, e.g. protecting personally identifiable information [PII]

Abstract

The present invention is directed at a system, method and device for detecting offensive content on a portable electronic device, by monitoring communications sent, received or stored on the portable electronic device, and wherein monitoring comprises collecting content data, classifying content data by calculating an alert score for content data wherein an alert score corresponds to offensive content detected, and sending an alert notification to a second portable electronic device to alert the detection of offensive content on the first portable electronic device.

Claims

1. A system for monitoring wireless communications on a first portable electronic device to detect offensive content received thereon, consisting essentially of:

at least one server comprising a non-transitory computer readable medium tangibly storing training data received from publicly available sources;

a training module consisting of a non-transitory computer-readable medium tangibly storing computer program instructions, which when executed by a processor cause the processor to receive the training data, transform the training data into a normalized representation of the training data, extract at least one first feature vector from the normalized representation and classify the at least one first feature vector as offensive or non-offensive;

a classification module in communication with the first portable electronic device and the training module, consisting of a non-transitory computer-readable medium tangibly storing computer program instructions, which when executed by a processor cause the processor to receive deleted or undeleted user data from the first portable electronic device, extract at least one second feature vector from the user data, classify the at least one second feature vector in correspondence with the at least one first feature vector as offensive or non-offensive, and calculate a score;

an alert module in communication with the classification module and a second portable electronic device, consisting of a non-transitory computer-readable medium tangibly storing computer program instructions, which when executed by a processor cause the processor to receive the score from the classification module and communicate the score to the second portable electronic device;

wherein the first portable electronic device comprises a non-transitory, computer-readable medium tangibly storing computer program instructions, which when executed by a processor cause the processor to transmit the user data to the classification module; and

the second portable electronic device comprises a non-transitory, computer-readable medium tangibly storing computer program instructions, which when executed by a processor cause the processor to receive an alert from the alert module when the score received by the alert module meets a user-determined threshold;

wherein the normalized representation of the training data is a matrix representation compiled by;

iteratively transforming the training data into a group of features selected from the group consisting of words, unigrams, bigrams, stop words, metadata key words, pixel color, pixel shape, and pixel texture; and

iteratively mapping the training data according to at least one category of offensive content under human supervision; and

wherein the matrix representation comprises at least one comparison matrix consisting of a difference comparison of the distance between each feature of the training data from every other feature;

the at least one first feature vector is multiplied by the at least one comparison matrix to generate a first vector-matrix product;

the classification of the at least one first feature vector as offensive or non-offensive is based on value of the first vector-matrix product; and

the at least one second feature vector is multiplied by the at least one comparison matrix to generate a second vector-matrix product; and

the score is the difference between the first and second vector-matrix products.

2. The system of claim 1, wherein the at least one category of offensive content comprises cyberbullying, sexting, hate speech, references to drug or alcohol use, or reference to symptoms of depression.