System and method for integrating data mining into a relational database management system

• US 6,629,095 B1
• Filed: 10/13/1998
• Issued: 09/30/2003
• Est. Priority Date: 10/14/1997
• Status: Expired due to Term

• Current Assignee: SAP SE
• Original Assignee: International Business Machines Corporation
• Inventors: Schneider, Donovan A., Bunger, Craig J., Wagstaff, William M., Koehler, Ann M., Cole, Richard L.
• Primary Examiner(s): JUNG, DAVID YIUK
• Application Number:
  US09/170,990
• Time in Patent Office: 1,813 Days
• Field of Search: 706/12, 706/18, 706/6, 706/16, 706/45, 705/10, 707/1-10, 707/100-104, 707/200-206, 711/100
• US Class Current: 1/1
• CPC Class Codes: G06F 16/284 Relational databases
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99936 Pattern matching access

Abstract

An integrated data mining and relational database management system is provided on a server. The data mining engine determines characteristics of relationships between input data values and an output data value that are obtained from a relational database (managed by the relational database management system). The integration allows direct conversion of data values from the relational database to data mining identifiers used for data mining operations by the data mining engine, and vice-versa. The integrated server also makes patterns uncovered during data mining (e.g., “understand” and “predict” information) available in virtual relational database tables that can be queried.

Claims

1. A computer program product, implemented on a machine readable medium, comprising instructions operable to:

integrate a relational database with a data mining engine; and

define a data mining model as a relational table, the data mining model providing multiple relationships between inputs and outputs and indicia of the importance of each relationship between an output and one or more inputs.

2. The computer program product of claim 1, further comprising instructions operable to construct rows corresponding to the relationships in the data mining model in response to user queries.

3. The computer program product of claim 1, further comprising instructions to define one or more columns from a relational database as the inputs to the data mining model relational table.

4. The computer program product of claim 3, further comprising instructions to integrate the data mining model relational table with the relational database on a server.

5. The computer program product of claim 1, further comprising instructions to define a column from a relational database as the output of the data mining model relational table.

6. The computer program product of claim 5, further comprising instructions to integrate the data mining model relational table with the relational database on a server.

7. The computer program product of claim 1, further comprising instructions to indicate the importance of the relationships between the inputs and the output of the data mining model in one or more columns of the data mining model relational table.

8. The computer program product of claim 7, further comprising instructions to indicate occurrence counts for the relationships between the inputs and the output of the data mining model in a column of the data mining model relational table.

9. The computer program product of claim 7, further comprising instructions to to indicate occurrence frequencies for the relationships between the inputs and the output of the data mining model in a column of the data mining model relational table.

10. The computer program product of claim 7, further comprising instructions to indicate correlations for the relationships between the inputs and the output of the data mining model in a column of the data mining model relational table.

11. The computer program product of claim 1, further comprising instructions to define a data mining model relational table having multiple rows, each row providing a relationship between input data values and an output data value, and each row also providing correlations between the output data value and the input data values.

12. The computer program product of claim 11, further comprising instructions to obtain the input data values and the output data value in a row of the data mining model from a relational database, and to calculate correlations between the input data values and the output data value with a data mining engine.

13. The computer program product of claim 1, further comprising instructions to define a look up table to translate between data mining identifiers used by the data mining engine and input and output values used by the data mining model relational table.

14. The computer program product of claim 13, further comprising instructions to define one or more look-up table indexes, including a forward translation index configured to translate data values from data stored in the relational database to data mining identifiers.

15. The computer program product of claim 13, further comprising instructions to define one or more look-up table indexes, including a reverse translation index configured to translate data mining identifiers to corresponding data values of data stored in the relational database.

16. The computer computer program product of claim 1, further comprising instructions to convert between data values of data stored in a relational database managed by the relational database management system and data mining identifiers representing the data values that are used by the data mining engine.

17. The computer program product of claim 1, wherein the instruction to define a data mining model further includes instructions to define one or more data mining sub-tables.

18. The computer program product of claim 1, wherein the data mining capabilities perform calculations without transferring large blocks of data from a remote database.

19. The computer program product of claim 1, wherein the instruction to define a data mining model relational table further comprises instructions to define a relational predict table, the relational predict table providing multiple relationships between inputs and an output and configured to return a predicted output in response to a query containing one or more inputs.

20. The computer program product of claim 19, wherein the instruction to define a relational predict table, further comprises instructions to indentify a column of the relational predict table to hold a confidence value for the output returned in response to the query.

21. The computer program product of claim 19, further comprising instructions to construct one or more rows in the relational predict table in response to one or more corresponding user queries.

22. The computer program product of claim 19, further comprising instructions to identify one or more columns of the relational predict table to receive user specified inputs from a relational database.

23. The computer program product of claim 19, further comprising instructions to identify a column of the relational predict table to receive output from a data mining engine.

24. The computer program product of claim 19, further comprising instructions to identify one or more columns of the relational predict table to receive a value indicating the confidence in the relationship between one or more user specified inputs and the output of the data mining engine.

25. The computer program product of claim 24, further comprising instructions to identify one or more columns of the relational predict table to receive values discriminating the relationship between one or more user specified inputs and the output of the data mining engine.

26. The computer program product of claim 19, wherein the instruction to define the relational predict table further comprises instructions to define multiple rows for the relational predict table, each row providing a relationship between input data values and an output data value and each row also providing a discrimination associated with the relationship.

27. The computer program product of claim 26, further comprising instructions to obtain the input data values and the output data value in a row from a relational database, and to calculate the discrimination of the relationship between the input data values and the output data value with a data mining engine.

28. A method of evaluating a query on a predict table in a data mining model, the predict table providing multiple relationships between input values and output values and configured to return a predicted output value in response to a query containing one or more input values, the method comprising:

(a) identifying an expression or sub-expression in a predicate of the query;

(b) identifying a start/stop condition from the expression or sub-expression;

(c) binding the start/stop condition to a column of the predict table; and

(d) passing column values constrained by the start/stop condition to data mining engine.

29. The method of claim 28, further comprising:

identifying a second sub-expression in the predicate; and

determining that the second sub-expression will be evaluated after receiving results of a data mining operation from the data mining engine.

30. The method of claim 29, wherein the second sub-expression is saved for binding to results of a data mining operation performed by the data mining engine on column values passed to it.

31. The method of claim 30, wherein the determining whether the expression or sub-expression is an atomic conjunctive expression or sub-expression, further comprises determining whether the expression or sub-expression compares a bindable value to a model input.

32. The method of claim 28, further comprising determining whether the expression or sub-expression is an atomic conjunctive expression or sub-expression, prior to identifying the start/stop condition.

33. A computer system comprising:

a relational database management system;

a data mining engine integrated with the relational database management system such that data values from a relational database managed by the relational database management system are directly converted to data mining identifiers used for data mining operations by the data mining engine, wherein said data mining engine determines characteristics of relationships between input data values and an output data value and wherein the input and output data values are obtained from the relational database.

34. The computer system of claim 33, further comprising a data mining model including a model table specifying input columns and an output column for rows of values from the relational database, which rows specify the relationships between the input values and the output value.

35. The computer system of claim 34, wherein the data mining model includes

a definition of one or more data mining sub-tables which are not populated with data values; and

one or more data model look up indexes specifying translations between values in the relational database and data mining identifiers used by the data mining engine to represent the relational database values.

36. The computer system of claim 35, wherein the one or more data model indexes include a forward translation index configured to translate data values from the relational database to data mining identifiers.

37. The computer system of claim 35, wherein the one or more data model indexes include a reverse translation index configured to translate data mining identifiers to corresponding data values from the relational database.

38. A computer program product, implemented on a machine readable medium, comprising instructions operable to:

receive a create model statement at a relational database with data mining capabilities; and

define a data mining model including at least one of a model table, an understand table, a predict table, a binary table, and an index at the relational database with data mining capabilities.

39. The computer program product of claim 38, wherein the instructions are operable to:

receive an insert statement; and

in response to receiving the insert statement;

populate the data mining model with data; and

provide information about patterns to at least one of the understand table, predict table, and binary table.

40. The computer program product of claim 38, wherein the instructions are operable to:

receive a query of the understand table; and

identify relevant patterns that were uncovered during data mining.

41. The computer program product of claim 38, wherein the instructions are operable to:

receive a query of a predict table; and

return a likely output for a given set of input values.

42. The computer program product of claim 38, wherein the computer program product is implemented on a machine readable medium at a server and wherein the instructions are operable to at least one of:

download the data mining model to a client using the binary table; and

upload a data mining model defined at the client using the binary table.