Data mining platform for knowledge discovery from heterogeneous data types and/or heterogeneous data sources

• US 7,921,068 B2
• Filed: 10/30/2007
• Issued: 04/05/2011
• Est. Priority Date: 05/01/1998
• Status: Expired due to Fees

• Current Assignee: Curtis Anderson, Health Discovery Corporation, James Roberts, Joe Mckenzie, Jules B. Paderewski, Julian N. Stern, Memorial Health Systems Incorporated, Timothy P. O'Hayer
• Original Assignee: Health Discovery Corporation
• Inventors: Reiss, Edward P., Doursat, René, Guyon, Isabelle, Weston, Jason Aaron Edward
• Primary Examiner(s): HOLMES, MICHAEL B
• Application Number:
  US11/928,606
• Publication Number:
  US 20080097938A1
• Time in Patent Office: 1,253 Days
• Field of Search: 706/45
• US Class Current: 706/45
• CPC Class Codes: G06F 18/2115 by evaluating different subsets according to an optimisation criterion, e.g. class separability, forward selection or backward elimination
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G16B 25/00 ICT specially adapted for hybridisation; ICT specially adapted for gene or protein expression
• CPC Class Codes: G16B 25/10 Gene or protein expression profiling; Expression-ratio estimation or normalisation
• CPC Class Codes: G16B 40/00 ICT specially adapted for biostatistics; ICT specially adapted for bioinformatics-related machine learning or data mining, e.g. knowledge discovery or pattern finding
• CPC Class Codes: G16B 40/20 Supervised data analysis
• CPC Class Codes: G16B 40/30 Unsupervised data analysis
• CPC Class Codes: G16B 50/00 ICT programming tools or database systems specially adapted for bioinformatics
• CPC Class Codes: G16B 50/30 Data warehousing; Computing architectures

Abstract

The data mining platform comprises a plurality of system modules, each formed from a plurality of components. Each module has an input data component, a data analysis engine for processing the input data, an output data component for outputting the results of the data analysis, and a web server to access and monitor the other modules within the unit and to provide communication to other units. Each module processes a different type of data, for example, a first module processes microarray (gene expression) data while a second module processes biomedical literature on the Internet for information supporting relationships between genes and diseases and gene functionality. In the preferred embodiment, the data analysis engine is a kernel-based learning machine, and in particular, one or more support vector machines (SVMs). The data analysis engine includes a pre-processing function for feature selection, for reducing the amount of data to be processed by selecting the optimum number of attributes, or “features”, relevant to the information to be discovered.

Claims

1. A data mining platform for generating an output comprising knowledge discovered from analysis of a plurality of data sets comprising heterogeneous data types or data from heterogeneous data sources, wherein the data points within the data sets comprise a plurality of descriptive features of varied relevance to knowledge discovery, the platform comprising:

a computer system programmed to implement a plurality of modules stored within a system memory, each module adapted for processing one data type of the plurality of heterogeneous data types, each module comprising;

(i) an input data source;

(ii) a data analysis engine;

(iii) a data output; and

(iv) a server connection for the input data source, the data analysis engine and the data output, wherein the data analysis engine comprises at least one processor for executing one or more support vector machines for generating a plurality of classes of data, and one or more feature subset ranking algorithms for ranking feature relevance to knowledge discovery from the plurality of data sets, wherein the at least one processor executes multiple runs of feature subset ranking on a plurality of data sets comprising one or more of sub-samples of the same data set, multiple data sets of heterogeneous data types, and heterogeneous data sources, to produce ranked lists of subsets of features with features having more relevance being ranked higher than features having less relevance, and wherein the at least one processor further validates an analysis obtained with one data type with the analysis obtained with another data type;

a server connected to the server connection for communicating with each of the input data source, the data analysis engine and the data output and for providing means for monitoring one or more of the input data source, the data analysis engine, and the data output;

a combined data analysis engine in communication with the server for combining the data output from the plurality of modules to generate a single output representing knowledge obtained from analyzing the plurality of heterogeneous data types; and

a graphical user interface for receiving the results of the feature subset ranking and generating a display of organized results of the feature subset ranking

2. The data mining platform of claim 1, wherein the at least one processor further executes at least one pre-processing algorithm for pre-processing the input data.

3. The data mining platform of claim 1, wherein the feature subset ranking algorithm comprises recursive feature elimination.

4. The data mining platform of claim 1, wherein the data analysis engine includes a visualization tool for displaying the results.

5. The data mining platform of claim 2, wherein the pre-processing algorithm comprises one or more data-cleaning algorithm for removing data outliers.

6. The data mining platform of claim 1, wherein the feature subset ranking algorithm comprises determining a distance from a margin between classes.

7. The data mining platform of claim 1, wherein the at least one processor executes an algorithm to merge into a single graph a structure of features previously obtained, including ranked lists of subsets of features, ranked lists of features, or trees of features.

8. The data-mining platform of claim 7, wherein the graph is a tree.

9. The data-mining platform of claim 1, wherein the at least one processor further executes an algorithm to merge ranked lists of subsets of features into a single ranked list of subset of features.

10. The data-mining platform of claim 1, wherein the at least one processor executes multiple feature ranking algorithms on multiple data sets including heterogeneous data types or from heterogeneous data sources to produce ranked lists of features and further executes an algorithm to merge the ranked lists of features into a single ranked list of features.

11. The data-mining platform of claim 1, wherein the one data type comprises knowledge contained in published literature.

12. The data mining platform of claim 1, wherein the one data type comprises gene expression data.

21. The data-mining platform of claim 8, wherein the tree includes alternate subsets of complementary child features corresponding to the ranked features.

22. The data-mining platform of claim 21, wherein the complementary child features in each alternate subset are ranked according to a joint probability of a parent ranked feature and the complementary child feature.

23. The data-mining platform of claim 1, wherein data from heterogeneous data sources are ranked separately according to the data type and the separate rankings are combined by addition or multiplication to generate the feature subset ranking.

24. The data-mining platform of claim 1, wherein the one data type comprises knowledge obtained from an internet search.

26. The computer system of claim 24, wherein the complementary child features in each alternate subset are ranked according to a joint probability of a parent ranked feature and the complementary child feature.

13. A computer system for generating an output comprising knowledge discovered from analysis of a plurality of data sets comprising heterogeneous data types or data from heterogeneous data sources, wherein the data points within the data sets comprise a plurality of descriptive features of varied relevance to knowledge discovery, the system comprising:

a plurality of modules, each module comprising software for processing one data type of the plurality of heterogeneous data types, each module comprising an input data source, a data analysis engine, a data output and a web server connection for each of the input data source, the data analysis engine and the data output, wherein the data analysis engine comprises at least one processor for executing one or more support vector machines for generating a plurality of classes of data, and one or more feature ranking algorithms, wherein the at least one processor executes multiple runs of feature ranking on a plurality of data sets comprising one or more of sub-samples of the same data set, multiple data sets of heterogeneous data types, and heterogeneous data sources, to produce ranked lists of features, wherein the at least one processor further executes an algorithm for organizing results of the feature ranking into a graph or map of features, and wherein the at least one processor executes multiple feature ranking algorithms on multiple data sets including heterogeneous data types or from heterogeneous data sources to produce ranked lists of features and further executes an algorithm to merge the ranked lists of features into a single ranked list of features;

a web server connected to the web server connection for communicating with each of the input data source, the data analysis engine and the data output and for providing means for monitoring one or more of the input data source, the data analysis engine, and the data output;

a combined data analysis engine in communication with the web server for combining the data output from the plurality of modules to generate a single output representing results obtained from analyzing the plurality of heterogeneous data types; and

a graphical user interface for receiving the results of the feature ranking and generating a display of organized results of the feature ranking.

14. The computer system of claim 13, wherein the at least one processor further executes at least one pre-processing algorithm for pre-processing the input data.

15. The computer system of claim 13, wherein the feature ranking algorithm comprises recursive feature elimination.

16. The computer system of claim 14, wherein the pre-processing algorithm comprises one or more data-cleaning algorithm for removing data outliers.

17. The computer system of claim 13, wherein the feature ranking algorithm comprises determining a distance from a margin between classes.

18. The computer system of claim 13, wherein the at least one processor executes an algorithm to merge into a single graph a structure of features previously obtained, including ranked lists of subsets of features, ranked lists of features, or trees of features.

19. The computer system of claim 13, wherein the at least one processor further executes an algorithm to merge ranked lists of subsets of features into a single ranked list of subsets of features.

20. The computer system of claim 13, wherein the one data type comprises knowledge contained in published literature.

25. The computer system of claim 18, wherein the trees of features include alternate subsets of complementary child features corresponding to the features in the ranked subsets of features.

27. The computer system of claim 13, wherein the lists of features are merged by addition or multiplication to generate the single ranked list of features.

28. The computer system of claim 13, wherein the one data type comprises knowledge obtained from an internet search.