Data item clustering and analysis

• US 9,202,249 B1
• Filed: 08/29/2014
• Issued: 12/01/2015
• Est. Priority Date: 07/03/2014
• Status: Active Grant

• Current Assignee: Palantir Technologies Incorporated
• Original Assignee: Palantir Technologies Incorporated
• Inventors: Cohen, David, Ma, Jason, Grossman, Jack, Thompson, James, Sprague, Matthew, Menon, Parvathy, Kross, Michael, Harris, Michael, Borochoff, Adam, Fu, Bing Jie, Nepomnyashchiy, Ilya, Berler, Steven, Smaliy, Alex, Boortz, Julia
• Primary Examiner(s): Hamilton, Lalita M
• Application Number:
  US14/473,552
• Time in Patent Office: 459 Days
• Field of Search: 705/35, 705/36.R, 705/37, 705/38
• US Class Current: 1/1
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06Q 40/12 Accounting
• CPC Class Codes: H04L 63/1408 by monitoring network traffic monitoring network traffic per se H04L43/00
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection
• CPC Class Codes: H04L 63/145 the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms

Abstract

Embodiments of the present disclosure relate to a data analysis system that may automatically generate memory-efficient clustered data structures, automatically analyze those clustered data structures, and provide results of the automated analysis in an optimized way to an analyst. The automated analysis of the clustered data structures (also referred to herein as data clusters) may include an automated application of various criteria or rules so as to generate a compact, human-readable analysis of the data clusters. The human-readable analyzes (also referred to herein as “summaries” or “conclusions”) of the data clusters may be organized into an interactive user interface so as to enable an analyst to quickly navigate among information associated with various data clusters and efficiently evaluate those data clusters in the context of, for example, a fraud investigation. Embodiments of the present disclosure also relate to automated scoring of the clustered data structures.

Claims

1. A computer system for efficiently analyzing large amounts of malfeasance-related data, the computer system comprising:

one or more computer readable storage devices configured to store;

a plurality of computer executable instructions;

one or more software modules including the plurality of computer executable instructions, the one or more software modules including a cluster engine module, a user interface engine module, and a workflow engine module;

a plurality of data clustering strategies based on rules generated by the cluster engine module; and

a plurality of data cluster types, each data cluster type of the plurality of data cluster types associated with a data clustering strategy of the plurality of data clustering strategies;

one or more cluster data sources configured to store malfeasance-related data items, the malfeasance-related data items including at least one of;

electronic documents, accounts, profiles, alerts, records, or communications; and

one or more hardware computer processors in communication with the one or more computer readable storage devices and the one or more cluster data sources, and configured to execute the one or more software modules in order to cause the computer system to;

designate, by the cluster engine module, one or more seeds by;

accessing, from the one or more cluster data sources, the malfeasance-related data items; and

selecting one or more data items from the accessed malfeasance-related data items based at least on a property value of at least one electronic document, account, profile, alert, record, or communication of the one or more data items evidencing suspicious behavior or a likelihood of fraud or criminal activity, and designating each data item of the one or more data items as a seed;

for each data item seed of the one or more designated seeds;

select, by the cluster engine module, a particular data clustering strategy from the plurality of data clustering strategies;

identify, by the cluster engine module, one or more malfeasance-related data items based at least on the particular data clustering strategy, wherein the particular data clustering strategy queries the one or more cluster data sources to determine the one or more malfeasance-related data items associated with the data item seed; and

generate, by the cluster engine module, a data cluster based at least on the data item seed, wherein generating the data cluster comprises;

adding the data item seed to the data cluster;

adding the one or more malfeasance-related data items identified as being associated with the data item seed to the data cluster;

identifying an additional one or more data items associated with any data items of the data cluster;

adding the additional one or more data items to the data cluster; and

storing the generated data cluster in the one or more computer readable storage devices;

generate, by the user interface engine module, at least one human-readable conclusion associated with at least one generated data cluster, wherein generating the at least one human-readable conclusion comprises;

determining a particular data cluster type from the plurality of cluster types based at least on the particular data clustering strategy;

identifying one or more human-readable templates comprising pre-generated text, wherein identifying the human-readable templates is based at least on predefined associations between respective human-readable templates and data cluster types, and the particular data cluster type;

automatically analyzing the at least one generated data cluster to generate summary data according to rules, scoring algorithms, or other criteria; and

populating the identified one or more human-readable templates with data from the at least one generated data cluster or summary data of the at least one generated data cluster;

cause presentation, by the user interface engine module, of the at least one generated data cluster and the at least one human-readable conclusion including the populated pre-generated text data, in a user interface of a client computing device; and

generate, by the workflow engine and the user interface engine, an interactive workflow process to allow the user to perform at least one of;

select new seeds, operate on existing seeds, generate new data clusters, or regenerate existing clusters.

2. The computer system of claim 1, wherein the one or more hardware computer processors are further configured to execute the one or more software modules in order to cause the computer system to:

for each generated data cluster;

determine a data cluster type associated with the generated data cluster;

analyze the generated data cluster based at least on data cluster analysis rules associated with the data cluster type; and

determine an alert score for the generated data cluster based on the analysis of the generated data cluster, wherein the alert score indicates a degree of correlation between characteristics of the generated data cluster and the data cluster analysis rules.

3. The computer system of claim 2, wherein the degree of correlation is based on both an assessment of risk associated with the generated data cluster and a confidence level in accuracy of the assessment of risk.

4. The computer system of claim 2, wherein a relatively higher alert score indicates a data cluster that is relatively more important for a human analyst to evaluate, and a relatively lower alert score indicates a data cluster that is relatively less important for the human analyst to evaluate.

5. The computer system of claim 2, wherein each alert score for respective data clusters is assigned to a category indicating a high degree of correlation, a medium degree of correlation, or a low degree of correlation.

6. The computer system of claim 5, wherein the high degree of correlation is associated with a first color, the medium degree of correlation is associated with a second color, and the low degree of correlation is associated with a third color.

7. The computer system of claim 2, wherein the one or more hardware computer processors are further configured to execute the one or more software modules in order to cause the computer system to, for each generated data cluster:

generate one or more human-readable conclusions regarding the generated data cluster;

generate an alert, the alert comprising the alert score, the one or more human-readable conclusions, one or more data items associated with the generated data cluster, and metadata associated with each of the one or more data items.

8. The computer system of claim 7, wherein the one or more hardware computer processors are further configured to execute the one or more software modules in order to cause the computer system to:

generate a second user interface including a list of user-selectable alert indicators, an alert indicator being provided for each of the generated alerts, each of the alert indicators providing a summary of information associated with respective generated alerts.

9. The computer system of claim 8, wherein the one or more hardware computer processors are further configured to execute the one or more software modules in order to cause the computer system to:

in response to a selection of an alert indicator by a human analyst;

generate an alert display, the alert display including at least an indication of the alert score and a list of the one or more human-readable conclusions.

10. The computer system of claim 9, wherein the alert display further includes a table of information associated with the one or more data items and associated metadata of the generated data cluster, and wherein the table of information includes a mixture of information of various types.

11. The computer system of claim 10, wherein the table of information includes one or more selectable user interface controls in order to filter according information type and/or time period.

12. The computer system of claim 9, wherein the one or more hardware computer processors are further configured to execute the one or more software modules in order to cause the computer system to, for each generated data cluster:

determine whether any other data cluster having a respective data cluster type different from the data cluster type of the generated data cluster is associated with a data item that is also associated with the generated data cluster.

13. The computer system of claim 12, wherein the alert display further indicates and provides a link to any of the other data clusters having the respective data cluster type different from the data cluster type of the generated data cluster and determined to be associated with the data item that is also associated with the generated data cluster of the alert display.

14. The computer system of claim 9, wherein a notification is provided to a human analyst via the alert display when the generated data cluster associated with the alert display has been regenerated such that the generated data cluster is changed.