System and method for automated pattern based alert generation

• US 10,452,984 B2
• Filed: 03/01/2016
• Issued: 10/22/2019
• Est. Priority Date: 03/01/2016
• Status: Active Grant

• Current Assignee: Pearson Education Incorporated (Pearson plc)
• Original Assignee: Pearson Education Incorporated (Pearson plc)
• Inventors: Ramachandran, Lakshmi, Foltz, Peter W., Cheng, Jian
• Primary Examiner(s): Holmes, Michael B
• Application Number:
  US15/058,035
• Publication Number:
  US 20170255867A1
• Time in Patent Office: 1,330 Days
• Field of Search: 706 12
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/353 into predefined classes
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/047 Pattern matching networks; Rete networks

Abstract

Systems and methods for alerting a third party device according to a result of automated evaluation of a received response packet according to at least one pattern in the data packet are disclosed herein. The system can include a memory having a pattern database and a model database. The system can further include a user device and a content management server. The content management server can receive a response packet and identify one or several patterns within the response packet. Based on the presence or absence of these one or several patterns in the response packet, the content management server can generate an evaluation of the response packet and generate and send an alert with the results of the evaluation.

Claims

1. A system for alerting a third party device according to a result of automated evaluation of a received response packet according to at least one pattern in the received response packet, the system comprising:

memory comprising;

a pattern database containing data defining at least two data patterns of tokens;

a model database comprising a model for generating an evaluative output for the received response packet;

a user device comprising;

a network interface configured to exchange data with a content management server via a communication network; and

an I/O subsystem configured to convert electrical signals to user interpretable outputs user interface;

the content management server, wherein the content management server is configured to;

receive a response packet from the user device, wherein the response packet comprises a user generated text string forming a plurality of words;

receive data from the pattern database, wherein the data from the pattern database defines at least two data patterns of tokens that correspond to one or more partitions of an evaluation schedule;

automatically generate a binary value for each of the at least two data patterns of tokens, wherein the binary value automatically generated for each of the at least two data patterns of tokens comprises one of;

a first binary value indicative of a presence of the data pattern in the response packet; and

a second binary value indicative of an absence of the data pattern in the response packet;

automatically generate an evaluation of the response packet based on the binary value generated for each of the at least two data patterns of tokens by applying the automatically generated binary values to the model; and

generate and send an alert to a supervisor device when the evaluation is below a threshold value; and

a supervisor device comprising;

a network interface configured to exchange data with the content management server via the communication network; and

an I/O subsystem configured to convert electrical signals to user interpretable outputs user interface;

wherein the supervisor device is configured to receive the alert from the content management server and automatically provide alert data to the user via the I/O subsystem in response to receipt of the alert, wherein the alert data identifies at least one aspect of the response packet and the user of the user device.

2. The system of claim 1, wherein the content management server is further configured to train the model.

3. The system of claim 2, wherein the memory further comprises a training database, wherein the training database comprises a plurality of response packets associated with a data packet.

4. The system of claim 3, wherein the training database further comprises an evaluation matrix, wherein the evaluation matrix identifies a plurality of performance levels and indicia associated with the performance levels.

5. The system of claim 4, wherein the indicia identify one or more attributes of a response packet of one or more performance levels.

6. The system of claim 5, wherein the training the model comprises:

receiving a training set from the training database, wherein the training set comprises some or all of the plurality of response packets associated with the data packet;

retrieving score data for the response packets in the training set;

dividing the training set into a plurality of performance groups, wherein each of the performance groups comprises a subset of the training set, wherein the training set is divided into performance groups according to the retrieved score data; and

identifying patterns in each of the performance groups.

7. The system of claim 6, wherein the identifying patterns in each of the performance groups comprises:

selecting a response packet;

tokenizing the text string of the response packet, wherein the text string is tokenized such that a token is associated with a word; and

extracting pattern data from the tokenized text string.

8. The system of claim 7, wherein the pattern data is stored in the pattern database.

9. The system of claim 8, wherein the alert comprises code to direct the supervisor device to provide an indicator of the received alert via the I/O subsystem.

10. The system of claim 9, wherein the indicator of the received alert comprises at least one of:

an aural indicator;

a tactile indicator;

or a visual indicator.

11. A method for alerting a third party device according to a result of automated evaluation of a received response packet according to at least one pattern in the received response packet, the method comprising:

receiving a response packet at a content management server from a user device via a communications network, wherein the response packet comprises a user generated text string forming a plurality of words;

receiving data from a pattern database, wherein the data from the pattern database defines at least two data patterns of tokens that correspond to one or more partitions of an evaluation schedule;

automatically generating a binary value for each of the at least two data patterns of tokens with the content management server, wherein the binary value automatically generated for each of the at least two data patterns of tokens comprises one of;

a first binary value indicative of a presence of the data pattern in the response packet; and

a second binary value indicative of an absence of the data pattern in the response packet;

automatically generating an evaluation of the response packet based on the binary value generated for each of the at least two data patterns of tokens by applying the automatically generated binary values to a model for generating an evaluative output for the received response packet; and

generating and sending an alert to a supervisor device when the evaluation triggers an alert status.

12. The method of claim 11, further comprising training the model.

13. The method of claim 12, wherein the training the model comprises:

receiving a training set from a training database, wherein the training set comprises some or all of a plurality of response packets associated with the response packet;

retrieving score data for the response packets in the training set;

dividing the training set into a plurality of performance groups, wherein each of the performance groups comprises a subset of the training set, wherein the training set is divided into performance groups according to the retrieved score data; and

identifying patterns in each of the performance groups.

14. The method of claim 13, wherein the identifying patterns in each of the performance groups comprises:

selecting a response packet;

tokenizing the text string of the response packet, wherein the text string is tokenized such that a token is associated with a word; and

extracting pattern data from the tokenized text string.

15. The method of claim 14, further comprising storing the pattern data in a pattern database.

16. The method of claim 15, wherein the alert comprises code to direct the supervisor device to provide an indicator of having received the alert via an I/O subsystem.

17. The method of claim 16, wherein the indicator of having received the alert comprises at least one of:

an aural indicator;

a tactile indicator;

or a visual indicator.

18. One or more non-transitory, computer-readable media having computer-readable instructions thereon which, when executed by one or more processors, cause the one or more processors to performing actions including:

processing a response packet received from a user device via a communications network, wherein the response packet comprises a user generated text string forming a plurality of words;

receiving data from a pattern database, wherein the data from the pattern database defines at least two data patterns of tokens that correspond to one or more partitions of an evaluation schedule;

automatically generating a binary value for each of the at least two data patterns of tokens with a content management server, wherein the binary value automatically generated for each of the at least two data patterns of tokens comprises one of;

a first binary value indicative of a presence of the data pattern in the response packet; and

a second binary value indicative of an absence of the data pattern in the response packet;

automatically generating an evaluation of the response packet based on the binary value generated for each of the at least two data patterns of tokens by applying the automatically generated binary values to a model for generating an evaluative output for the received response packet; and

generating and sending an alert to a supervisor device when the evaluation triggers an alert status.

19. The one or more non-transitory, computer-readable media of claim 18, where the actions further comprise training the model.

20. The one or more non-transitory, computer-readable media of claim 19, wherein the training the model comprises:

processing a training set from a training database, wherein the training set comprises some or all of a plurality of response packets associated with the response packet;

retrieving score data for the response packets in the training set;

dividing the training set into a plurality of performance groups, wherein each of the performance groups comprises a subset of the training set, wherein the training set is divided into performance groups according to the retrieved score data; and

identifying patterns in each of the performance groups.