Unified relational database model for data mining

• US 20030195889A1
• Filed: 04/04/2002
• Published: 10/16/2003
• Est. Priority Date: 04/04/2002
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Yao, Albert Zhongxing, Vishnubhotla, Prasad Rajendra
• Granted Patent: US 6,970,882 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2465 Query processing support for facilitating data mining operations in structured databases
• CPC Class Codes: G06F 16/30 of unstructured textual data document management systems G06F16/93
• CPC Class Codes: Y10S 707/99937 Sorting
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

Managing data mining activities in a data mining environment, including selecting a model scoring results table, wherein the selecting is carried out in dependence upon metadata included in a model scoring results control table, the model scoring results control table being related to a data set control table including data set metadata; reading a scoring output data record from the selected model scoring results table; storing the scoring output data record in a managed representation table for scoring results; selecting a model training results file, wherein the selecting is carried out in dependence upon metadata included in a mining model control table; reading training results data from the selected model training results file; and storing the training results data in a model training results table, the model training results table comprising a relational representation of the training results data from the selected model training results file.

Claims

1. A method for managing data mining activities in a data mining environment, the data mining environment having data sets, a data mining tool, a data mining model, model training results stored in model training results files, and scoring output data stored in records in model scoring results tables, the method comprising:

selecting a model scoring results table, wherein the selecting is carried out in dependence upon metadata included in a model scoring results control table, the model scoring results control table being related to a data set control table including data set metadata;

reading a scoring output data record from the selected model scoring results table;

storing the scoring output data record in a managed representation table for scoring results;

selecting a model training results file, wherein the selecting is carried out in dependence upon metadata included in a mining model control table, the mining model control table being related to the data set control table and the model scoring results control table;

reading training results data from the selected model training results file; and

storing the training results data in at least one model training results table, the at least one model training results table comprising a relational representation of the training results data from the selected model training results file, one of the at least one model training results table referencing the mining model control table, the managed representation table for scoring results being related to the referencing model training results table, the data set control table, the mining model control table, and the model scoring results control table.

2. The method of claim 1 wherein the data set control table comprises:

an identification number for each data set;

a name for each data set;

a description for each data set;

a type for each data set;

a usage for each data set; and

a location for each data set.

3. The method of claim 1 wherein the mining model control table comprises:

an identification number for the data mining model;

a model name for the data mining model;

a description for the data mining model;

a model type for the data mining model;

an algorithm used for training the data mining model;

an identification number for a model training input data set used by the data mining tool to train the data mining model;

a date the data mining model was last trained;

an identification number for a model training results file for the data mining model;

a file name for the model training results file for the data mining model; and

a file location for the model training results file for the data mining model.

4. The method of claim 1 wherein the model scoring results control table comprises:

an identification number for each model scoring results table;

an identification number for the data mining model used for the model scoring results table;

an identification number for a scoring input data set for the model scoring results table;

a name for the model scoring results table;

a name for a scoring setting; and

a scoring status indicating whether the model scoring results table is actively used.

5. The method of claim 1 wherein the managed representation table for scoring results comprises:

an identification number for each record in each selected model scoring results table;

an identification number for each data mining model used for scoring;

an identification number for each scoring input data set;

an identification number for each model scoring results table; and

model scoring results data from each selected model scoring results table.

6. The method of claim 5, wherein the data mining model is a clustering model and the data mining tool scores scoring input data sets using the clustering model to produce scoring output data records, to establish clusters, to select from the clusters a best fitting cluster and a second best fitting cluster, to score the fitting quality of each record to the best fitting cluster, to score the fitting quality of each record to the second best fitting cluster, and to establish a confidence value of the cluster assignment of each record, the managed representation table for scoring results further comprising for each record:

a numerical index for the best fitting cluster;

a score of the fitting quality of the record to the best fitting cluster;

a numerical index for the second best fitting cluster;

a score of the fitting quality of the record to the second best filling cluster; and

a confidence value of the cluster assignment of the record.

7. The method of claim 5, wherein the data mining model is a clustering model and the data mining tool scores scoring input data sets using the clustering model to produce scoring output data records, and wherein the managed representation table further comprises a model identification column in the managed representation table for scoring results in which is stored content from the clustering model identification column in the at least one model training results table, the managed representation table for scoring results relating to the at least one model training result table through the managed representation table model identification column.

8. The method of claim 5, wherein:

each model scoring results table further comprises a record identification number column in which is stored an identification number for each record in the model scoring results table; and

the managed representation table for scoring results further comprises a record identification column in which the identification number for each record from each selected model scoring results table is stored, the identification numbers being those identification numbers stored in the model scoring results table record identification column.

9. The method of claim 1 wherein the selected model training results file comprises a hierarchical representation of the model training results.

10. The method of claim 1 wherein:

the data mining model is a clustering model;

the selected model training results file comprises a hierarchical representation of model training results; and

the at least one model training results table comprises a clustering table, a cluster table, a cluster attribute table and a cluster attribute bin table, and

reading the training results data from the selected model training results file in a model training results table further comprises;

reading, from a hierarchical clustering node embodied in the hierarchical representation of model training results, clustering data describing a clustering, and storing the clustering data in a relational clustering record in the clustering table, wherein

the relational clustering record includes a clustering identification field in which is stored content from a model identification field in the mining model control table, the clustering table relating to the mining model control table through the clustering identification field; and

the managed representation table comprises a model identification field in which is stored content from the clustering identification field in the clustering table and from a model identification field in the mining model control table, the managed representation table for scoring results relating to the clustering table and to the mining model control table through the managed representation table model identification field;

reading, from a hierarchical cluster node embodied in the hierarchical representation of model training results, cluster data describing a cluster, and storing the cluster data in a relational cluster record in the cluster table, wherein

the hierarchical cluster node is embodied in a position in the hierarchy below the hierarchical clustering node;

the relational cluster record is related to the relational clusteringrecord through a first foreign key comprising the clustering identification field; and

the relational cluster record includes a cluster identification field;

reading, from a hierarchical cluster attribute node embodied in the hierarchical representation of model training results, cluster attribute data describing a cluster attribute, and storing the cluster attribute data in a relational cluster attribute record in the cluster attribute table, wherein

the hierarchical cluster attribute node is embodied in a position in the hierarchy below the hierarchical cluster node;

the relational cluster attribute record is related to the relational cluster record through a second foreign key comprising the cluster identification field;

the relational cluster attribute record is related to the relational clustering record through a third foreign key comprising the clustering identification field; and

the relational cluster attribute record includes a cluster attribute identification field;

reading, from a hierarchical cluster attribute bin node embodied in the hierarchical representation of model training results, cluster attribute bin data describing a cluster attribute bin, and storing the cluster attribute bin data in a relational cluster attribute bin record in the cluster attribute bin table, wherein

the hierarchical cluster attribute bin node is embodied in a position in the hierarchy below the hierarchical cluster attribute node;

the relational cluster attribute bin record is related to the relational cluster attribute record through a fourth foreign key comprising the cluster attribute identification field;

the relational cluster attribute bin record is related to the relational cluster record through a fifth foreign key comprising the cluster identification field; and

the relational cluster attribute bin record is related to the relational clustering record through a sixth foreign key comprising the clustering identification field.

11. The method of claim 10 wherein the clustering data as recorded in the relational clustering record comprises:

a unique identifier for the relational clustering record;

a text name for the relational clustering record;

a text description of the purpose of a clustering represented by the relational clustering record;

a clustering type;

a number of clusters given by the clustering;

a number of attributes considered in the clustering;

an algorithm field identifying the clustering algorithm used in the clustering; and

an items numeric field that stores the number of data items input to the clustering from the scoring input data set.

12. The method of claim 10 wherein the cluster data as recorded in the relational cluster record comprises:

a unique identifier for the cluster;

a unique identifier of a relational clustering record to which the relational cluster record related;

an ordinal number of the relational cluster record;

a text name for the cluster represented by the relational cluster record;

a text description of the purpose of the cluster represented by the relational cluster record; and

a numeric field identifying the number of data items from the scoring input data set that are represented in records related to the relational cluster record.

13. The method of claim 10 wherein the cluster attribute data as recorded in the relational cluster attribute record comprises:

a unique identifier of the relational cluster attribute record;

a unique identifier of a relational clustering record to which the relational cluster attribute record is related;

a unique identifier of a relational cluster record to which the relational cluster attribute record is related;

an attribute type;

a text name of a relational cluster attribute represented by the relational cluster attribute record;

a text description of the relational cluster attribute;

a use type field;

a categories numeric field indicating a number of categories associated with the cluster attribute when the attribute type has the value “

categorical;

”

a lowest value numeric field indicating a lowest value allowed when the attribute type has the value “

continuous;

”

a highest value numeric field indicating a highest value allowed when the attribute type has the value “

continuous;

”

an items numeric field identifying the number of data items from the scoring input data set that are represented in records related to the relational cluster attribute record.

14. The method of claim 10 wherein the cluster attribute bin data as recorded in the relational cluster attribute bin record comprises:

a unique identifier of the attribute bin;

a unique identifier of a relational clustering record to which the relational attribute bin record is related;

a unique identifier of a relational cluster record to which the relational attribute bin record is related;

a unique identifier of a relational attribute record to which the relational attribute bin record is related;

a text name for the attribute bin represented by the relational attribute bin record;

a category name;

a lowest value numeric field in which is stored the lowest value in the bin;

a highest value numeric record in which is stored the highest value in the bin;

an items numeric field in which is stored the number of items in the bin;

a values sum numeric field in which is stored the sum of the values in the bin; and

a squares numeric field in which is stored the sum of the squares of values in the bin.

15. The method of claim 1, wherein:

the mining model control table is related to the data set control table through a mining model control table data set foreign key;

the model scoring results table is related to the data set control table through a model scoring results table data set foreign key and to the mining model control table through a model scoring results table mining model foreign key;

the managed representation table for scoring results is related to the data set control table through a managed representation table data set foreign key, to the mining model control table through a managed representation table mining model foreign key, and to the model scoring results control table through a managed representation table model scoring results foreign key; and

the referencing model training results table references the mining model control table through a model training results table foreign key, and the managed representation table for scoring results is related to the referencing model training results table through a managed representation table training results foreign key.

16. The method of claim 15, wherein the mining model control table foreign key comprises a model training input data set identification column in the mining model control table in which is stored content from a data set identification column of the data set control table, the mining model control table relating to the data set control table through the foreign key.

17. The method of claim 15, wherein the mining model control table foreign key comprises a model training results file identification column in the mining model control table in which is stored content from a data set identification column of the data set control table, the mining model control table relating to the data set control table through the foreign key.

18. The method of claim 15, wherein the model scoring results control table data set foreign key comprises a model scoring results table identification column in the model scoring results control table in which is stored content from a data set identification column of the data set control table, the model scoring results control table relating to the data set control table through the foreign key.

19. The method of claim 15, wherein the model scoring results control table data set foreign key comprises a scoring input data set identification column in the model scoring results control table in which is stored content from a data set identification column of the data set control table, the model scoring results control table relating to the data set control table through the foreign key.

20. The method of claim 15, wherein the model scoring results control table mining model foreign key comprises a model identification column in the model scoring results control table in which is stored content from a model identification column of the mining model control table, the model scoring results control table relating to the mining model control table through the foreign key.

21. The method of claim 15, wherein the managed representation table data set foreign key comprises a model scoring input data set identification column in the managed representation table for scoring results in which is stored content from a data set identification column of the data set control table, the managed representation table for scoring results relating to the data set control table through the managed representation table data set foreign key.

22. The method of claim 15, wherein the managed representation table mining model foreign key comprises a model identification column in the managed representation table for scoring results in which is stored content from a model identification column of the mining model control table, the managed representation table for scoring results relating to the mining model control table through the managed representation table mining model foreign key.

23. The method of claim 15, wherein the managed representation table model scoring results foreign key comprises:

a scoring input data set identification column in the managed representation table for scoring results in which is stored content from a scoring input data set identification column of the model scoring results control table,

a model identification column in the managed representation table for scoring results in which is stored content from a model identification column of the model scoring results control table, and

a model scoring results table identification column in the managed representation table for scoring results in which is stored content from a model scoring results table identification column of the model scoring results control table,

the managed representation table for scoring results relating to the model scoring results control table through the managed representation table model scoring results foreign key.

24. The method of claim 15, wherein the model training results table foreign key comprises a model identification column in one of the at least one model training results table in which is stored content from a model identification column of the mining model control table, the model training results table relating to the mining model control table through the model training results table foreign key.

25. The method of claim 15, wherein the managed representation table training results foreign key comprises a model identification column in the managed representation table for scoring results in which is stored content from a model identification column in one of the at least one model training results tables.

26. The method of claim 1, wherein the model training results file is in Predictive Model Markup Language format.

27. A system for managing data mining activities in a data mining environment, the data mining environment having data sets, a data mining tool, a data mining model, model training results stored in model training results files, and scoring output data stored in records in model scoring results tables, the system comprising:

means for selecting a model scoring results table, wherein the means for selecting includes metadata in a model scoring results control table, the model scoring results control table being related to a data set control table including data set metadata;

means for reading a scoring output data record from the selected model scoring results table;

means for storing the scoring output data record in a managed representation table for scoring results;

means for selecting a model training results file, wherein the means for selecting includes metadata in a mining model control table, the mining model control table being related to the data set control table and the model scoring results control table;

means for reading training results data from the selected model training results file; and

means for storing the training results data in at least one model training results table, the at least one model training results table comprising a relational representation of the training results data from the selected model training results file, one of the at least one model training results table referencing the mining model control table, the managed representation table for scoring results being related to the referencing model training results table, the data set control table, the mining model control table, and the model scoring results control table.

28. The system of claim 27 wherein the data set control table comprises:

an identification number for each data set;

a name for each data set;

a description for each data set;

a type for each data set;

a usage for each data set; and

a location for each data set.

29. The system of claim 27 wherein the mining model control table comprises:

an identification number for the data mining model;

a model name for the data mining model;

a description for the data mining model;

a model type for the data mining model;

an algorithm used for training the data mining model;

an identification number for a model training input data set used by the data mining tool to train the data mining model;

a date the data mining model was last trained;

an identification number for a model training results file for the data mining model;

a file name for the model training results file for the data mining model; and

a file location for the model training results file for the data mining model.

30. The system of claim 27 wherein the model scoring results control table comprises:

an identification number for each model scoring results table;

an identification number for the data mining model used for the model scoring results table;

an identification number for a scoring input data set for the model scoring results table;

a name for the model scoring results table;

a name for a scoring setting; and

a scoring status indicating whether the model scoring results table is actively used.

31. The system of claim 27 wherein the managed representation table for scoring results comprises:

an identification number for each record in each selected model scoring results table;

an identification number for each data mining model used for scoring;

an identification number for each scoring input data set;

an identification number for each model scoring results table; and

model scoring results data from each selected model scoring results table.

32. The system of claim 31, wherein the data mining model is a clustering model and the data mining tool scores scoring input data sets using the clustering model to produce scoring output data records, to establish clusters, to select from the clusters a best fitting cluster and a second best fitting cluster, to score the fitting quality of each record to the best fitting cluster, to score the fitting quality of each record to the second best fitting cluster, and to establish a confidence value of the cluster assignment of each record, the managed representation table for scoring results further comprising for each record:

a numerical index for the best fitting cluster;

a score of the fitting quality of the record to the best fitting cluster;

a numerical index for the second best fitting cluster;

a score of the fitting quality of the record to the second best filling cluster; and

a confidence value of the cluster assignment of the record.

33. The system of claim 31, wherein the data mining model is a clustering model and the data mining tool scores scoring input data sets using the clustering model to produce scoring output data records, and wherein the managed representation table further comprises a model identification column in the managed representation table for scoring results in which is stored content from the clustering model identification column in the at least one model training results table, the managed representation table for scoring results relating to the at least one model training result table through the managed representation table model identification column.

34. The system of claim 31, wherein:

each model scoring results table further comprises a record identification number column in which is stored an identification number for each record in the model scoring results table; and

the managed representation table for scoring results further comprises a record identification column in which the identification number for each record from each selected model scoring results table is stored, the identification numbers being those identification numbers stored in the model scoring results table record identification column.

35. The system of claim 27 wherein the selected model training results file comprises a hierarchical representation of the model training results.

36. The system of claim 27 wherein:

the data mining model is a clustering model;

the selected model training results file comprises a hierarchical representation of model training results; and

the at least one model training results table comprises a clustering table, a cluster table, a cluster attribute table and a cluster attribute bin table, and

means for reading the training results data from the selected model training results file in a model training results table further comprises;

means for reading, from a hierarchical clustering node embodied in the hierarchical representation of model training results, clustering data describing a clustering, and means for storing the clustering data in a relational clustering record in the clustering table, wherein

the relational clustering record includes a clustering identification field in which is stored content from a model identification field in the mining model control table, the clustering table relating to the mining model control table through the clustering identification field; and

the managed representation table comprises a model identification field in which is stored content from the clustering identification field in the clustering table and from a model identification field in the mining model control table, the managed representation table for scoring results relating to the clustering table and to the mining model control table through the managed representation table model identification field;

means for reading, from a hierarchical cluster node embodied in the hierarchical representation of model training results, cluster data describing a cluster, and means for storing the cluster data in a relational cluster record in the cluster table, wherein

the hierarchical cluster node is embodied in a position in the hierarchy below the hierarchical clustering node;

the relational cluster record is related to the relational clustering record through a first foreign key comprising the clustering identification field; and

the relational cluster record includes a cluster identification field;

means for reading, from a hierarchical cluster attribute node embodied in the hierarchical representation of model training results, cluster attribute data describing a cluster attribute, and means for storing the cluster attribute data in a relational cluster attribute record in the cluster attribute table, wherein

the hierarchical cluster attribute node is embodied in a position in the hierarchy below the hierarchical cluster node;

the relational cluster attribute record is related to the relational cluster record through a second foreign key comprising the cluster identification field;

the relational cluster attribute record is related to the relational clustering record through a third foreign key comprising the clustering identification field; and

the relational cluster attribute record includes a cluster attribute identification field;

means for reading, from a hierarchical cluster attribute bin node embodied in the hierarchical representation of model training results, cluster attribute bin data describing a cluster attribute bin, and means for storing the cluster attribute bin data in a relational cluster attribute bin record in the cluster attribute bin table, wherein

the hierarchical cluster attribute bin node is embodied in a position in the hierarchy below the hierarchical cluster attribute node;

the relational cluster attribute bin record is related to the relational cluster attribute record through a fourth foreign key comprising the cluster attribute identification field;

the relational cluster attribute bin record is related to the relational cluster record through a fifth foreign key comprising the cluster identification field; and

the relational cluster attribute bin record is related to the relational clustering record through a sixth foreign key comprising the clustering identification field.

37. The system of claim 36 wherein the clustering data as recorded in the relational clustering record comprises:

a unique identifier for the relational clustering record;

a text name for the relational clustering record;

a text description of the purpose of a clustering represented by the relational clustering record;

a clustering type;

a number of clusters given by the clustering;

a number of attributes considered in the clustering;

an algorithm field identifying the clustering algorithm used in the clustering; and

an items numeric field that stores the number of data items input to the clustering from the scoring input data set.

38. The system of claim 36 wherein the cluster data as recorded in the relational cluster record comprises:

a unique identifier for the cluster;

a unique identifier of a relational clustering record to which the relational cluster record related;

an ordinal number of the relational cluster record;

a text name for the cluster represented by the relational cluster record;

a text description of the purpose of the cluster represented by the relational cluster record; and

a numeric field identifying the number of data items from the scoring input data set that are represented in records related to the relational cluster record.

39. The system of claim 36 wherein the cluster attribute data as recorded in the relational cluster attribute record comprises:

a unique identifier of the relational cluster attribute record;

a unique identifier of a relational clustering record to which the relational cluster attribute record is related;

a unique identifier of a relational cluster record to which the relational cluster attribute record is related;

an attribute type;

a text name of a relational cluster attribute represented by the relational cluster attribute record;

a text description of the relational cluster attribute;

a use type field;

a categories numeric field indicating a number of categories associated with the cluster attribute when the attribute type has the value “

categorical;

”

a lowest value numeric field indicating a lowest value allowed when the attribute type has the value “

continuous;

”

a highest value numeric field indicating a highest value allowed when the attribute type has the value “

continuous;

”

an items numeric field identifying the number of data items from the scoring input data set that are represented in records related to the relational cluster attribute record.

40. The system of claim 36 wherein the cluster attribute bin data as recorded in the relational cluster attribute bin record comprises:

a unique identifier of the attribute bin;

a unique identifier of a relational clustering record to which the relational attribute bin record is related;

a unique identifier of a relational cluster record to which the relational attribute bin record is related;

a unique identifier of a relational attribute record to which the relational attribute bin record is related;

a text name for the attribute bin represented by the relational attribute bin record;

a category name;

a lowest value numeric field in which is stored the lowest value in the bin;

a highest value numeric record in which is stored the highest value in the bin;

an items numeric field in which is stored the number of items in the bin;

a values sum numeric field in which is stored the sum of the values in the bin; and

a squares numeric field in which is stored the sum of the squares of values in the bin.

41. The system of claim 27, wherein:

the mining model control table is related to the data set control table through a mining model control table data set foreign key;

the model scoring results table is related to the data set control table through a model scoring results table data set foreign key and to the mining model control table through a model scoring results table mining model foreign key;

the managed representation table for scoring results is related to the data set control table through a managed representation table data set foreign key, to the mining model control table through a managed representation table mining model foreign key, and to the model scoring results control table through a managed representation table model scoring results foreign key; and

the referencing model training results table references the mining model control table through a model training results table foreign key, and the managed representation table for scoring results is related to the referencing model training results table through a managed representation table training results foreign key.

42. The system of claim 41, wherein the mining model control table foreign key comprises a model training input data set identification column in the mining model control table in which is stored content from a data set identification column of the data set control table, the mining model control table relating to the data set control table through the foreign key.

43. The system of claim 41, wherein the mining model control table foreign key comprises a model training results file identification column in the mining model control table in which is stored content from a data set identification column of the data set control table, the mining model control table relating to the data set control table through the foreign key.

44. The system of claim 41, wherein the model scoring results control table data set foreign key comprises a model scoring results table identification column in the model scoring results control table in which is stored content from a data set identification column of the data set control table, the model scoring results control table relating to the data set control table through the foreign key.

45. The system of claim 41, wherein the model scoring results control table data set foreign key comprises a scoring input data set identification column in the model scoring results control table in which is stored content from a data set identification column of the data set control table, the model scoring results control table relating to the data set control table through the foreign key.

46. The system of claim 41, wherein the model scoring results control table mining model foreign key comprises a model identification column in the model scoring results control table in which is stored content from a model identification column of the mining model control table, the model scoring results control table relating to the mining model control table through the foreign key.

47. The system of claim 41, wherein the managed representation table data set foreign key comprises a model scoring input data set identification column in the managed representation table for scoring results in which is stored content from a data set identification column of the data set control table, the managed representation table for scoring results relating to the data set control table through the managed representation table data set foreign key.

48. The system of claim 41, wherein the managed representation table mining model foreign key comprises a model identification column in the managed representation table for scoring results in which is stored content from a model identification column of the mining model control table, the managed representation table for scoring results relating to the mining model control table through the managed representation table mining model foreign key.

49. The system of claim 41, wherein the managed representation table model scoring results foreign key comprises:

a scoring input data set identification column in the managed representation table for scoring results in which is stored content from a scoring input data set identification column of the model scoring results control table,

a model identification column in the managed representation table for scoring results in which is stored content from a model identification column of the model scoring results control table, and

a model scoring results table identification column in the managed representation table for scoring results in which is stored content from a model scoring results table identification column of the model scoring results control table,

the managed representation table for scoring results relating to the model scoring results control table through the managed representation table model scoring results foreign key.

50. The system of claim 41, wherein the model training results table foreign key comprises a model identification column in one of the at least one model training results table in which is stored content from a model identification column of the mining model control table, the model training results table relating to the mining model control table through the model training results table foreign key.

51. The system of claim 41, wherein the managed representation table training results foreign key comprises a model identification column in the managed representation table for scoring results in which is stored content from a model identification column in one of the at least one model training results tables.

52. The system of claim 27, wherein the model training results file is in Predictive Model Markup Language format.

53. A computer program product for managing data mining activities in a data mining environment, the data mining environment having data sets, a data mining tool, a data mining model, model training results stored in model training results files, and scoring output data stored in records in model scoring results tables, the computer program product comprising:

a recording medium;

means, recorded on the recording medium, for selecting a model scoring results table, wherein the means for selecting includes metadata in a model scoring results control table, the model scoring results control table being related to a data set control table including data set metadata;

means, recorded on the recording medium, for reading a scoring output data record from the selected model scoring results table;

means, recorded on the recording medium, for storing the scoring output data record in a managed representation table for scoring results;

means, recorded on the recording medium, for selecting a model training results file, wherein the means for selecting includes metadata in a mining model control table, the mining model control table being related to the data set control table and the model scoring results control table;

means, recorded on the recording medium, for reading training results data from the selected model training results file; and

means, recorded on the recording medium, for storing the training results data in at least one model training results table, the at least one model training results table comprising a relational representation of the training results data from the selected model training results file, one of the at least one model training results table referencing the mining model control table, the managed representation table for scoring results being related to the referencing model training results table, the data set control table, the mining model control table, and the model scoring results control table.

54. The computer program product of claim 53 wherein the data set control table comprises:

an identification number for each data set;

a name for each data set;

a description for each data set;

a type for each data set;

a usage for each data set; and

a location for each data set.

55. The computer program product of claim 53 wherein the mining model control table comprises:

an identification number for the data mining model;

a model name for the data mining model;

a description for the data mining model;

a model type for the data mining model;

an algorithm used for training the data mining model;

an identification number for a model training input data set used by the data mining tool to train the data mining model;

a date the data mining model was last trained;

an identification number for a model training results file for the data mining model;

a file name for the model training results file for the data mining model; and

a file location for the model training results file for the data mining model.

56. The computer program product of claim 53 wherein the model scoring results control table comprises:

an identification number for each model scoring results table;

an identification number for the data mining model used for the model scoring results table;

an identification number for a scoring input data set for the model scoring results table;

a name for the model scoring results table;

a name for a scoring setting; and

a scoring status indicating whether the model scoring results table is actively used.

57. The computer program product of claim 53 wherein the managed representation table for scoring results comprises:

an identification number for each record in each selected model scoring results table;

an identification number for each data mining model used for scoring;

an identification number for each scoring input data set;

an identification number for each model scoring results table; and

model scoring results data from each selected model scoring results table.

58. The computer program product of claim 57, wherein the data mining model is a clustering model and the data mining tool scores scoring input data sets using the clustering model to produce scoring output data records, to establish clusters, to select from the clusters a best fitting cluster and a second best fitting cluster, to score the fitting quality of each record to the best fitting cluster, to score the fitting quality of each record to the second best fitting cluster, and to establish a confidence value of the cluster assignment of each record, the managed representation table for scoring results further comprising for each record:

a numerical index for the best fitting cluster;

a score of the fitting quality of the record to the best fitting cluster;

a numerical index for the second best fitting cluster;

a score of the fitting quality of the record to the second best filling cluster; and

a confidence value of the cluster assignment of the record.

59. The computer program product of claim 57, wherein the data mining model is a clustering model and the data mining tool scores scoring input data sets using the clustering model to produce scoring output data records, and wherein the managed representation table further comprises a model identification column in the managed representation table for scoring results in which is stored content from the clustering model identification column in the at least one model training results table, the managed representation table for scoring results relating to the at least one model training result table through the managed representation table model identification column.

60. The computer program product of claim 57, wherein:

each model scoring results table further comprises a record identification number column in which is stored an identification number for each record in the model scoring results table; and

the managed representation table for scoring results further comprises a record identification column in which the identification number for each record from each selected model scoring results table is stored, the identification numbers being those identification numbers stored in the model scoring results table record identification column.

61. The computer program product of claim 53 wherein the selected model training results file comprises a hierarchical representation of the model training results.

62. The computer program product of claim 53 wherein:

the data mining model is a clustering model;

the selected model training results file comprises a hierarchical representation of model training results; and

the at least one model training results table comprises a clustering table, a cluster table, a cluster attribute table and a cluster attribute bin table, and

means, recorded on the recording medium, for reading the training results data from the selected model training results file in a model training results table further comprises;

means, recorded on the recording medium, for reading, from a hierarchical clustering node embodied in the hierarchical representation of model training results, clustering data describing a clustering, and means, recorded on the recording medium, for storing the clustering data in a relational clustering record in the clustering table, wherein

the relational clustering record includes a clustering identification field in which is stored content from a model identification field in the mining model control table, the clustering table relating to the mining model control table through the clustering identification field; and

the managed representation table comprises a model identification field in which is stored content from the clustering identification field in the clustering table and from a model identification field in the mining model control table, the managed representation table for scoring results relating to the clustering table and to the mining model control table through the managed representation table model identification field;

means, recorded on the recording medium, for reading, from a hierarchical cluster node embodied in the hierarchical representation of model training results, cluster data describing a cluster, and means, recorded on the recording medium, for storing the cluster data in a relational cluster record in the cluster table, wherein

the hierarchical cluster node is embodied in a position in the hierarchy below the hierarchical clustering node;

the relational cluster record is related to the relational clustering record through a first foreign key comprising the clustering identification field; and

the relational cluster record includes a cluster identification field;

means, recorded on the recording medium, for reading, from a hierarchical cluster attribute node embodied in the hierarchical representation of model training results, cluster attribute data describing a cluster attribute, and means, recorded on the recording medium, for storing the cluster attribute data in a relational cluster attribute record in the cluster attribute table, wherein

the hierarchical cluster attribute node is embodied in a position in the hierarchy below the hierarchical cluster node;

the relational cluster attribute record is related to the relational cluster record through a second foreign key comprising the cluster identification field;

the relational cluster attribute record is related to the relational clustering record through a third foreign key comprising the clustering identification field; and

the relational cluster attribute record includes a cluster attribute identification field;

means, recorded on the recording medium, for reading, from a hierarchical cluster attribute bin node embodied in the hierarchical representation of model training results, cluster attribute bin data describing a cluster attribute bin, and means, recorded on the recording medium, for storing the cluster attribute bin data in a relational cluster attribute bin record in the cluster attribute bin table, wherein

the hierarchical cluster attribute bin node is embodied in a position in the hierarchy below the hierarchical cluster attribute node;

the relational cluster attribute bin record is related to the relational cluster attribute record through a fourth foreign key comprising the cluster attribute identification field;

the relational cluster attribute bin record is related to the relational cluster record through a fifth foreign key comprising the cluster identification field; and

the relational cluster attribute bin record is related to the relational clustering record through a sixth foreign key comprising the clustering identification field.

63. The computer program product of claim 62 wherein the clustering data as recorded in the relational clustering record comprises:

a unique identifier for the relational clustering record;

a text name for the relational clustering record;

a text description of the purpose of a clustering represented by the relational clustering record;

a clustering type;

a number of clusters given by the clustering;

a number of attributes considered in the clustering;

an algorithm field identifying the clustering algorithm used in the clustering; and

an items numeric field that stores the number of data items input to the clustering from the scoring input data set.

64. The computer program product of claim 62 wherein the cluster data as recorded in the relational cluster record comprises:

a unique identifier for the cluster;

a unique identifier of a relational clustering record to which the relational cluster record related;

an ordinal number of the relational cluster record;

a text name for the cluster represented by the relational cluster record;

a text description of the purpose of the cluster represented by the relational cluster record; and

a numeric field identifying the number of data items from the scoring input data set that are represented in records related to the relational cluster record.

65. The computer program product of claim 62 wherein the cluster attribute data as recorded in the relational cluster attribute record comprises:

a unique identifier of the relational cluster attribute record;

a unique identifier of a relational clustering record to which the relational cluster attribute record is related;

a unique identifier of a relational cluster record to which the relational cluster attribute record is related;

an attribute type;

a text name of a relational cluster attribute represented by the relational cluster attribute record;

a text description of the relational cluster attribute;

a use type field;

a categories numeric field indicating a number of categories associated with the cluster attribute when the attribute type has the value “

categorical;

”

a lowest value numeric field indicating a lowest value allowed when the attribute type has the value “

continuous;

”

a highest value numeric field indicating a highest value allowed when the attribute type has the value “

continuous;

”

an items numeric field identifying the number of data items from the scoring input data set that are represented in records related to the relational cluster attribute record.

66. The computer program product of claim 62 wherein the cluster attribute bin data as recorded in the relational cluster attribute bin record comprises:

a unique identifier of the attribute bin;

a unique identifier of a relational clustering record to which the relational attribute bin record is related;

a unique identifier of a relational cluster record to which the relational attribute bin record is related;

a unique identifier of a relational attribute record to which the relational attribute bin record is related;

a text name for the attribute bin represented by the relational attribute bin record;

a category name;

a lowest value numeric field in which is stored the lowest value in the bin;

a highest value numeric record in which is stored the highest value in the bin;

an items numeric field in which is stored the number of items in the bin;

a values sum numeric field in which is stored the sum of the values in the bin; and

a squares numeric field in which is stored the sum of the squares of values in the bin.

67. The computer program product of claim 53, wherein:

the mining model control table is related to the data set control table through a mining model control table data set foreign key;

the model scoring results table is related to the data set control table through a model scoring results table data set foreign key and to the mining model control table through a model scoring results table mining model foreign key;

the managed representation table for scoring results is related to the data set control table through a managed representation table data set foreign key, to the mining model control table through a managed representation table mining model foreign key, and to the model scoring results control table through a managed representation table model scoring results foreign key; and

the referencing model training results table references the mining model control table through a model training results table foreign key, and the managed representation table for scoring results is related to the referencing model training results table through a managed representation table training results foreign key.

68. The computer program product of claim 67, wherein the mining model control table foreign key comprises a model training input data set identification column in the mining model control table in which is stored content from a data set identification column of the data set control table, the mining model control table relating to the data set control table through the foreign key.

69. The computer program product of claim 67, wherein the mining model control table foreign key comprises a model training results file identification column in the mining model control table in which is stored content from a data set identification column of the data set control table, the mining model control table relating to the data set control table through the foreign key.

70. The computer program product of claim 67, wherein the model scoring results control table data set foreign key comprises a model scoring results table identification column in the model scoring results control table in which is stored content from a data set identification column of the data set control table, the model scoring results control table relating to the data set control table through the foreign key.

71. The computer program product of claim 67, wherein the model scoring results control table data set foreign key comprises a scoring input data set identification column in the model scoring results control table in which is stored content from a data set identification column of the data set control table, the model scoring results control table relating to the data set control table through the foreign key.

72. The computer program product of claim 67, wherein the model scoring results control table mining model foreign key comprises a model identification column in the model scoring results control table in which is stored content from a model identification column of the mining model control table, the model scoring results control table relating to the mining model control table through the foreign key.

73. The computer program product of claim 67, wherein the managed representation table data set foreign key comprises a model scoring input data set identification column in the managed representation table for scoring results in which is stored content from a data set identification column of the data set control table, the managed representation table for scoring results relating to the data set control table through the managed representation table data set foreign key.

74. The computer program product of claim 67, wherein the managed representation table mining model foreign key comprises a model identification column in the managed representation table for scoring results in which is stored content from a model identification column of the mining model control table, the managed representation table for scoring results relating to the mining model control table through the managed representation table mining model foreign key.

75. The computer program product of claim 67, wherein the managed representation table model scoring results foreign key comprises:

a scoring input data set identification column in the managed representation table for scoring results in which is stored content from a scoring input data set identification column of the model scoring results control table,

a model identification column in the managed representation table for scoring results in which is stored content from a model identification column of the model scoring results control table, and

a model scoring results table identification column in the managed representation table for scoring results in which is stored content from a model scoring results table identification column of the model scoring results control table,

the managed representation table for scoring results relating to the model scoring results control table through the managed representation table model scoring results foreign key.

76. The computer program product of claim 67, wherein the model training results table foreign key comprises a model identification column in one of the at least one model training results table in which is stored content from a model identification column of the mining model control table, the model training results table relating to the mining model control table through the model training results table foreign key.

77. The computer program product of claim 67, wherein the managed representation table training results foreign key comprises a model identification column in the managed representation table for scoring results in which is stored content from a model identification column in one of the at least one model training results tables.

78. The computer program product of claim 53, wherein the model training results file is in Predictive Model Markup Language format.