Visual analysis of data using sequenced dataset reduction

• US 10,545,975 B1
• Filed: 12/12/2016
• Issued: 01/28/2020
• Est. Priority Date: 06/22/2016
• Status: Active Grant

• Current Assignee: Palantir Technologies Incorporated
• Original Assignee: Palantir Technologies Incorporated
• Inventors: Mahdy, Mohammady, Carter, Joseph
• Primary Examiner(s): Gortayo, Dangelino N
• Application Number:
  US15/375,894
• Time in Patent Office: 1,142 Days
• Field of Search: 707603, 707736, 707737, 707754, 707771, 707776, 707797
• CPC Class Codes: G06F 16/2423 Interactive query statement specification based on a database schema
• CPC Class Codes: G06F 16/2425 Iterative querying; Query formulation based on the results of a preceding query
• CPC Class Codes: G06F 16/2428 Query predicate definition using graphical user interfaces, including menus and forms G06F16/2423 takes precedence
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems
• CPC Class Codes: G06F 16/252 between a Database Management System and a front-end application
• CPC Class Codes: G06F 16/26 Visual data mining; Browsing structured data
• CPC Class Codes: G06F 16/283 Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 40/221 Parsing markup language streams streaming G06F40/149

Abstract

Systems and methods for implementing sequenced filter templates to intelligently reduce a dataset to find useful patterns and source data are disclosed. An expert investigative user may configure a filter template comprising a series of filters organized in a sequence desired by the expert user. The filter template can be customized by an end user to reduce a dataset and perform guide investigation of the reduced dataset.

Claims

1. A system comprising:

one or more processors; and

a memory comprising instructions which, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

receiving, from a first client device, specification of a sequenced reducing filter template comprising a plurality of reducing filters that specify parameter refinements of a dataset, the sequenced reducing filter template specifying a sequence in which to apply the plurality of reducing filters and further specifying parameters available for each of the plurality of reducing filters;

causing, on a second client device, a presentation of the sequenced reducing filter template;

receiving, from the second client device, a selection of one or more parameter refinements available in the sequenced filter template;

responsive to the selection, generating a query configured to apply the plurality of reducing filters and the one or more parameter refinements to reduce the dataset according to the sequenced reducing filter template;

transmitting the query to a database server;

receiving, from the database server, a reduced dataset generated by applying, to the dataset, the plurality of reducing filters with the one or more parameter refinements in the sequence specified by the sequenced filter template;

storing the reduced dataset generated from the dataset according to the query; and

causing, on the second client device, a display including a visualization generated from the reduced dataset.

2. The system of claim 1, the operations further comprising:

generating the reduced dataset by applying, to the dataset, the plurality of reducing filters in the sequence specified by the sequenced reducing filter template.

3. The system of claim 1, the operations further comprising:

receiving, at the system, customization data for customizable parameters in the plurality of reducing filters of the sequenced reducing filter template.

4. The system of claim 3, wherein the generating of the query comprises generating the query such that the query is configured to apply the plurality of reducing filters in the sequence and according to the customizable parameters.

5. The system of claim 1, wherein causing the display of the visualization comprises displaying, on the display device, the reduced dataset as at least one of:

a bar graph, a histogram, a distribution plot, a table comprising columns and rows, or a network graph comprising nodes connected by edges.

6. The system of claim 1, the operations further comprising:

determining that web pages of a website are parsable using a parse template;

generating a parse user interface comprising input fields based on the parse template, at least a portion of the input fields prepopulated with data parsed from a webpage of the website; and

storing, through the parse user interface, the data from the input fields in the dataset.

7. A method comprising:

receiving, from a first client device, specification of a sequenced reducing filter template comprising a plurality of reducing filters that specify parameter refinements of a dataset, the sequenced reducing filter template specifying a sequence in which to apply the plurality of reducing filters and further specifying parameters available for each of the plurality of reducing filters;

causing, on a second client device, a presentation of the sequenced reducing filter template;

receiving, from the second client device, a selection of one or more parameter refinements available in the sequenced filter template;

responsive to the selection, generating a query configured to apply the plurality of reducing filters and the one or more parameter refinements to reduce the dataset according to the sequenced reducing filter template;

transmitting the query to a database server;

receiving, from the database server, a reduced dataset generated by applying, to the dataset, the plurality of reducing filters with the one or more parameter refinements in the sequence specified by the sequenced filter template;

storing the reduced dataset generated from the dataset according to the query; and

causing, on the second client device, a display including a visualization generated from the reduced dataset.

8. The method of claim 7, further comprising:

generating the reduced dataset by applying, to the dataset, the plurality of reducing filters in the sequence specified by the sequenced reducing filter template.

9. The method of claim 7, wherein the parameter refinements available include range limits for a data type in a reducing filter of the plurality of filters, the range limits of the reducing filter including a first range of values and a second range of other values that do not overlap with the first range of values.

10. The method of claim 9, wherein the range limits include or more of:

ranges of year values, ranges of physical distance values.

11. The method of claim 7, wherein causing the display of the visualization comprises displaying, on the display device, the reduced dataset as at least one of:

a bar graph, a histogram, a distribution plot, a table comprising columns and rows, or a network graph comprising nodes connected by edges.

12. The method of claim 7, further comprising:

determining that web pages of a website are parsable using a parse template;

generating a parse user interface comprising input fields based on the parse template, at least a portion of the input fields prepopulated with data parsed from a webpage of the website; and

storing, through the parse user interface, the data from the input fields in the dataset.

13. A computer-readable storage device embodying instructions that, when executed by a machine, cause the machine to perform operations comprising:

receiving, from a first client device, specification of a sequenced reducing filter template comprising a plurality of reducing filters that specify refinements of a dataset, the sequenced reducing filter template specifying a sequence in which to apply the plurality of filters and further specifying parameters available for each of the plurality of reducing filters;

causing, on a second client device, a presentation of the sequenced reducing filter template;

receiving, from the second client device, a selection of the one or more parameter refinements available in the sequenced filter template;

responsive to the selection, generating a query configured to apply the plurality of reducing filters and the one or more parameter refinements to reduce the dataset according to the sequenced reducing filter template;

transmitting the query to a database server;

receiving, from the database server, a reduced dataset generated by applying, to the dataset, the plurality of reducing filters with the one or more parameter refinements in the sequence specified by the sequenced filter template;

storing the reduced dataset generated from the dataset according to the query; and

causing, on the second client device, a display including a visualization generated from the reduced dataset.

14. The computer-readable storage medium of claim 13, wherein the instructions further cause the machine to perform operations comprising:

generating the reduced dataset by applying, to the dataset, the plurality of reducing filters in the sequence specified by the sequenced reducing filter template.

15. The computer-readable storage medium of claim 13, wherein the parameter refinements available include range limits for a data type in a reducing filter of the plurality of filters, the range limits of the reducing filter including a first range of values and a second range of other values that do not overlap with the first range of values.

16. The computer-readable storage medium of claim 15, wherein the range limits include or more of:

ranges of year values, ranges of physical distance values.

17. The computer-readable storage medium of claim 13, wherein the instructions further cause the machine to perform operations comprising:

determining that web pages of a website are parsable using a parse template;

generating a parse user interface comprising input fields based on the parse template, at least a portion of the input fields prepopulated with data parsed from a webpage of the website; and

storing, through the parse user interface, the data from the input fields in the dataset.