SCALABLE TOPOLOGICAL DATA ANALYSIS USING TOPOLOGICAL SUMMARIES OF SUBSETS

• US 20180025073A1
• Filed: 07/21/2017
• Published: 01/25/2018
• Est. Priority Date: 07/21/2016
• Status: Abandoned Application

• Current Assignee: Ayasdi Al LLC
• Original Assignee: Ayasdi, Inc. (Fortive Corp.)
• Inventors: Singh, Gurjeet, Hsu, Ryan, Carlsson, Gunnar
• Application Number:
  US15/656,995
• Publication Number:
  US 20180025073A1
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/26 Visual data mining; Browsing structured data
• CPC Class Codes: G06F 16/287 Visualization; Browsing
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs

Abstract

A method comprises dividing a set of data points into a structure subset and boost subsets, adding the data points in structure subset into each boost subset, analyzing the structure subset using topological data analysis (TDA) to identify nodes of a structure graph, boost graph, and modified graph, analyze each of the boost subsets using the TDA to identify additional nodes of boost graph, for each node in each of the plurality of boost graphs that do not share at least one data point with a node in the structure graph, adding the node of a particular boost subset including data points that are members of the node, to the modified graph, and generating report indicating relationships between data points of the set of data points based on the nodes of the modified graph.

Claims

1. A method comprising:

dividing a set of data points into a structure subset and a plurality of boost subsets;

adding the data points in the structure subset into each of the plurality of boost subsets to create a plurality of combination subsets;

receiving a lens function identifier, a metric function identifier, and a resolution function identifier;

mapping data points of the structure subset to a reference space utilizing a lens function identified by the lens function identifier;

generating a cover of reference space using a resolution function identified by the resolution identifier;

clustering the data points of the structure subset using the cover and a metric function identified by the metric function identifier to determine each node of a plurality of nodes of a structure graph;

generating a plurality of nodes for a modified graph, each of the plurality of nodes of the modified graph corresponding to each of the plurality of nodes in the structure graph;

for each of the plurality of combination subsets;

mapping data points of a particular combination subset to the reference space utilizing the lens function;

generating the cover of reference space using the resolution function; and

clustering the data points of the particular combination subset using the cover and the metric function to determine each node of a plurality of nodes to add to a particular boost graph of the plurality of boost graphs; and

for each node in each of the plurality of boost graphs that do not share at least one data point with a node in the structure graph, adding the node of a particular boost subset including data points that are members of the node, to the modified graph; and

generating report indicating relationships between data points of the set of data points based on the nodes of the modified graph.

2. The method of claim 1, wherein each data point in the set of data points is a member of a structure subset or one of the plurality of boost subsets.

3. The method of claim 1, wherein dividing the set of data points into the structure subset comprises selecting data points from the set of data points at random.

4. The method of claim 1, wherein generating the report indicating the relationships between the data points of the set of data points based on the nodes of the modified graph comprises generating a visualization of the modified graph including the nodes of the modified graph and a plurality of edges, wherein each of the edges of the plurality of edges connects two nodes of the modified graph that share at least one data point as members.

5. The method of claim 1, further comprising:

for each node in each of the plurality of boost graphs shares at least one data point with a node in the structure graph;

determining a node in the structure graph with the greatest intersection of data points with the node of the particular boost graph;

determining a corresponding node in the modified subset to the node in the structure graph with the greatest intersection of data points, the corresponding node in the modified subset sharing the greatest number of data points with the node in the structure graph relative to other nodes in the modified subset; and

adding data points from the node of the particular boost graph to the corresponding node.

6. The method of claim 5, wherein if there is a first node and a second in the structure graph that share an equal number of data points with the node of the particular boost graph, determining the corresponding node in the modified subset comprises:

determining a first corresponding node in the modified graph that corresponds to the first node in the structure graph;

determining a second corresponding node in the modified graph that corresponds to the second node in the structure graph; and

adding half the data points of the node of the particular boost graph to each the first corresponding node and the second corresponding node.

7. The method of claim 6 wherein individual data points of the node of the particular boost graph are divided between the first and second corresponding nodes at random.

8. The method of claim 5, wherein generating the report indicating the relationships between the data points of the set of data points based on the nodes of the modified graph comprises generating a visualization of the modified graph including the nodes of the modified graph and a plurality of edges, wherein each of the edges of the plurality of edges connects two nodes of the modified graph that share at least one data point as members.

9. The method of claim 1, further comprising generating edges between nodes of the modified graph if the nodes share at least one data point.

10. The method of claim 5, wherein determining the node in the structure graph with the greatest intersection of data points with the node of the particular boost graph comprises determining the node in the structure graph that shares the greatest number of data points with the node of the particular boost graph in proportion to a total number of data points that are members of the node in the structure graph.

11. A non-transitory computer readable medium comprising instructions executable by a processor to perform a method, the method comprising:

dividing a set of data points into a structure subset and a plurality of boost subsets;

adding the data points in the structure subset into each of the plurality of boost subsets to create a plurality of combination subsets;

receiving a lens function identifier, a metric function identifier, and a resolution function identifier;

mapping data points of the structure subset to a reference space utilizing a lens function identified by the lens function identifier;

generating a cover of reference space using a resolution function identified by the resolution identifier;

clustering the data points of the structure subset using the cover and a metric function identified by the metric function identifier to determine each node of a plurality of nodes of a structure graph;

generating a plurality of nodes for a modified graph, each of the plurality of nodes of the modified graph corresponding to each of the plurality of nodes in the structure graph;

for each of the plurality of combination subsets;

mapping data points of a particular combination subset to the reference space utilizing the lens function;

generating the cover of reference space using the resolution function; and

clustering the data points of the particular combination subset using the cover and the metric function to determine each node of a plurality of nodes to add to a particular boost graph of the plurality of boost graphs; and

for each node in each of the plurality of boost graphs that do not share at least one data point with a node in the structure graph, adding the node of a particular boost subset including data points that are members of the node, to the modified graph; and

generating report indicating relationships between data points of the set of data points based on the nodes of the modified graph.

12. The non-transitory computer readable medium of claim 11, wherein each data point in the set of data points is a member of a structure subset or one of the plurality of boost subsets.

13. The non-transitory computer readable medium of claim 11, wherein dividing the set of data points into the structure subset comprises selecting data points from the set of data points at random.

14. The non-transitory computer readable medium of claim 11, wherein generating the report indicating the relationships between the data points of the set of data points based on the nodes of the modified graph comprises generating a visualization of the modified graph including the nodes of the modified graph and a plurality of edges, wherein each of the edges of the plurality of edges connects two nodes of the modified graph that share at least one data point as members.

15. The non-transitory computer readable medium of claim 11, the method further comprising:

for each node in each of the plurality of boost graphs shares at least one data point with a node in the structure graph;

determining a node in the structure graph with the greatest intersection of data points with the node of the particular boost graph;

determining a corresponding node in the modified subset to the node in the structure graph with the greatest intersection of data points, the corresponding node in the modified subset sharing the greatest number of data points with the node in the structure graph relative to other nodes in the modified subset; and

adding data points from the node of the particular boost graph to the corresponding node.

16. The non-transitory computer readable medium of claim 15, wherein if there is a first node and a second in the structure graph that share an equal number of data points with the node of the particular boost graph, determining the corresponding node in the modified subset comprises:

determining a first corresponding node in the modified graph that corresponds to the first node in the structure graph;

determining a second corresponding node in the modified graph that corresponds to the second node in the structure graph; and

adding half the data points of the node of the particular boost graph to each the first corresponding node and the second corresponding node.

17. The non-transitory computer readable medium of claim 16, wherein individual data points of the node of the particular boost graph are divided between the first and second corresponding nodes at random.

18. The non-transitory computer readable medium of claim 15, wherein generating the report indicating the relationships between the data points of the set of data points based on the nodes of the modified graph comprises generating a visualization of the modified graph including the nodes of the modified graph and a plurality of edges, wherein each of the edges of the plurality of edges connects two nodes of the modified graph that share at least one data point as members.

19. The non-transitory computer readable medium of claim 11, further comprising generating edges between nodes of the modified graph if the nodes share at least one data point.

20. The non-transitory computer readable medium of claim 15, wherein determining the node in the structure graph with the greatest intersection of data points with the node of the particular boost graph comprises determining the node in the structure graph that shares the greatest number of data points with the node of the particular boost graph in proportion to a total number of data points that are members of the node in the structure graph.

21. A system comprising:

one or more processors; and

memory containing instructions executable by at least one of the one or more processors to;

divide a set of data points into a structure subset and a plurality of boost subsets;

add the data points in the structure subset into each of the plurality of boost subsets to create a plurality of combination subsets;

receive a lens function identifier, a metric function identifier, and a resolution function identifier;

map data points of the structure subset to a reference space utilizing a lens function identified by the lens function identifier;

generate a cover of reference space using a resolution function identified by the resolution identifier;

cluster the data points of the structure subset using the cover and a metric function identified by the metric function identifier to determine each node of a plurality of nodes of a structure graph;

generate a plurality of nodes for a modified graph, each of the plurality of nodes of the modified graph corresponding to each of the plurality of nodes in the structure graph;

for each of the plurality of combination subsets;

map data points of a particular combination subset to the reference space utilizing the lens function;

generate the cover of reference space using the resolution function; and

cluster the data points of the particular combination subset using the cover and the metric function to determine each node of a plurality of nodes to add to a particular boost graph of the plurality of boost graphs; and

for each node in each of the plurality of boost graphs that do not share at least one data point with a node in the structure graph, add the node of a particular boost subset including data points that are members of the node, to the modified graph; and

generate report indicating relationships between data points of the set of data points based on the nodes of the modified graph.