System for data mining using neuroagents

• US 5,970,482 A
• Filed: 02/12/1996
• Issued: 10/19/1999
• Est. Priority Date: 02/12/1996
• Status: Expired due to Term

• Current Assignee: Datamind Corporation
• Original Assignee: Datamind Corporation
• Inventors: Piffero, Veronique, Rajkovic, Eric Bertrand, Pham, Khai Minh
• Primary Examiner(s): Downs, Robert W.
• Application Number:
  US08/600,229
• Time in Patent Office: 1,345 Days
• Field of Search: 395/10, 395/22, 395/77, 395/50, 706/12, 706/45, 706/16
• US Class Current: 706/16
• CPC Class Codes: G06N 3/042 Knowledge-based neural networks; Logical representations of neural networks

Abstract

A neuroagent approach is used in an automated and unified data mining system to provide an explicitly predictive knowledge model. The neuroagent is a neural multi-agent approach based on macro-connectionism and comprises a double integration at the association and symbolic level as well as the knowledge model level. This data mining system permits discovery, evaluation and prediction of the correlative factors of data, i.e., the conjunctions, as corresponding to neuroexpressions (a semantic connection of neuroagents) connected to an output neuroagent which corresponds to the data output, the connection weights yielding the relative significance of these factors to the given output. The system takes data sets called Domains, establishes candidate dimensions or Parameters, categorizes Parameters into discrete bins, and trains a neuroagent network composed of neuroagents allocated for each bin and each output based on a discovery data set, called a Discovery Domain, and by building up the various minimal and contextual neuroexpressions, and setting the appropriate connection weights, the results may therefore be compared with an optional evaluation data set, called an Evaluation Domain to establish the accuracy of the knowledge model, and thereafter applied with some degree of confidence to a prediction set or Prediction Domain. The ranking in importance of the composite Parameters may be calculated as well as the discrimination between the various outputs, which permits the relevant factors of interest to a decision maker to come into focus.

Claims

1. A data mining system comprising:

a study manager with which a discovery domain, an evaluation domain and a prediction domain can be selected from at least one data sources, each data source including one or more data records, each data record having one or more parameters;

a knowledge model engine, coupled to the study manager, which;

(i) when presented with the discovery domain, constructs an explicitly predictive knowledge model in the form of a neuroagent network therefrom and returns a discovery results set, and

(ii) when presented with either of the evaluation or prediction domains, applies the neuroagent network thereto and returns an evaluation or prediction results set, respectively;

a discovery manager, coupled to the knowledge model engine, which takes the discovery results set from the knowledge model engine and calculates the relative significance of the parameters under the knowledge model;

an evaluation manager, coupled to the knowledge model engine, which takes the evaluation results set from the knowledge model engine and calculates the accuracy of the knowledge model; and

a prediction manager, coupled to the knowledge model engine, which takes the prediction results set from the knowledge model engine and calculates the predictions of the knowledge model.

2. The data mining system of claim 1, the neuroagent network of the knowledge model engine including:

a plurality of input neuroagents;

a plurality of output neuroagents, each output neuroagent having a minimal excitation zone and a contextual excitation zone, the contextual excitation zone having an excitation level and an excitation threshold associated therewith; and

one or more connections formed between the input neuroagents and the output neuroagents, each connection of a input neuroagent to an output neuroagent formed either with the minimal or the contextual excitation zone thereof, each connection formed with the contextual excitation zone having a connection weight associated therewith, each connection weight having a stimulation function associated therewith,

wherein the excitation level of an output neuroagent is a summation of the stimulation functions associated with the connections formed with the contextual excitation zone of that output neuroagent.

3. The data mining system of claim 2 wherein further each parameter is designated as either an input parameter or an output parameter, each parameter having a value, each value of an input parameter associated with at most one input neuroagent, each value of an output parameter parameter associated with at most one output neuroagent, a match between a parameter value and its associated input or output neuroagent termed a validation of such neuroagent.

4. The data mining system of claim 3 wherein application of the neuroagent network to either of the evaluation or prediction domains includes, for each data record of such domain:

a validation of those input neuroagents associated to the parameter values of the data record, and for each output neuroagent, calculation of its associated excitation level.

5. The data mining system of claim 4 wherein further:

the values of the output parameter of the evaluation domain are specified before being presented to the knowledge model; and

calculation of the accuracy of the knowledge model, for a given data record having a first value specified for the output parameter, includes identification of a first output neuroagent having the highest excitation level, this level termed a winner level, and comparison with an output neuroagent which is associated with the first value.

6. The data mining system of claim 5 wherein calculation of the accuracy of the knowledge model, for a given data record, further includes verification that the winner level exceeds the excitation threshold of the first output neuroagent.

7. The data mining system of claim 5 wherein calculation of the accuracy of the knowledge model, for a given data record, further includes:

identification of an output neuroagent having the next-highest excitation level as compared with winner level, this level termed a challenger level, and comparison of the winner level with challenger level.

8. The data mining system of claim 7 wherein calculation of the accuracy of the knowledge model, for a given data record, further includes verification that the relative difference between the winner level and the challenger level is less than a predetermined value, termed a rating error.

9. The data mining system of claim 4 wherein calculation of the predictions of the knowledge model, for a given data record, includes identification of an output neuroagent having the highest excitation level.

10. The data mining system of claim 9 wherein calculation of the prediction of the knowledge model, for a given data record, further includes formation of the discrimination between excitation levels of the output neuroagents.

11. The data mining system of claim 3 wherein the neuroagent network further includes:

one or more neuroexpressions connected to at least one output neuroagent, each neuroexpression representing one or more input neuroagents that are, for each data record of the discovery domain, either all validated or all not validated when the output neuroagent with which these neuroagents are connected is validated.

12. The data mining system of claim 11 wherein further:

the one or more neuroexpressions include at least a minimal neuroexpression or at least a contextual neuroexpression,

a minimal neuroexpression meaning that, for each data record of the discovery domain, the one or more neuroagents of the minimal neuroexpression are always validated when the output neuroagent with which these neuroagents are connected is validated, and

a contextual neuroexpression meaning that there is at least one data record of the discovery domain where the one or more neuroagents of the contextual neuroexpression are all not validated when the output neuroagent with which these neuroagents are connected is validated.

13. The data mining system of claim 2 wherein the calculation of the relative significance of the parameters includes the determination of the connection weight of the output neuroagents.

14. The data mining system of claim 13 wherein the calculation of the relative significance of the parameters further includes determination of the discrimination between the connection weights of the output neuroagents.

15. The data mining system of claim 2 wherein the calculation of the relative significance of the parameters further includes identification of a first set of input neuroagents for which a connection exists only to a single output neuroagent.

16. The data mining system of claim 2 wherein the calculation of the relative significance of the parameters further includes identification, for a given output neuroagent, a second set of input neuroagents for which no connection exists therewith.

17. The data mining system of claim 1 further comprising:

a graphical user interface including

a set of dialog screens through which the user can perform the selection of the discovery, evaluation and prediction domains, and

a spreadsheet and graphics generator by which the results sets from the neuroagent engine and the calculated values therefrom are displayed.

18. The data mining system of claim 17 wherein the spreadsheet and graphics generator is embodied, at least in part in an external spreadsheet application package.

19. The data mining system of claim 1 further comprising:

a flat file system; and

an open database connectivity to one or more databases,

wherein the one or more data sources are selected from either the flat file system or the database.

20. The data mining system of claim 1 further comprising:

an open database connectivity to one or more databases, wherein at least one database is adapted to perform data warehousing.

21. A data mining system comprising:

a study manager with which a discovery domain can be selected from one or more data sources, each data source including at least one data records, each data record having one or more parameters;

a knowledge model engine, coupled to the study manager, which when presented with the discovery domain, constructs an explicitly predictive knowledge model in the form of a neuroagent network therefrom and returns a discovery results set; and

a discovery manager, coupled to the knowledge model engine, which takes the discovery results set from the knowledge model engine and calculates the relative significance of the parameters under the knowledge model.

22. A data mining system comprising:

a study manager with which a prediction domain can be selected from one or more data sources, each data source including at least one data records, each data record having one or more parameters;

a knowledge model engine, coupled to the study manager, which when presented with the prediction domain, applies an explicitly predictive knowledge model in the form of a neuroagent network thereto and returns a prediction results set; and

a prediction manager, coupled to the knowledge model engine, which takes the prediction results set from the knowledge model engine and calculates the predictions of the knowledge model.

33. The method of claim 22 wherein the importance calculation is subject to an occurrence threshold.

23. In a data mining system wherein a discovery domain includes a plurality of parameters having one or more input parameters and one output parameter, the discovery domain selected from one or more data sources, each data source providing one or more data records, a method of creating meta data from the discovery domain comprising:

(i) for each parameter, input or output, determining the parameter'"'"'s maximal value, minimal value, number of distinct values and number of null values;

(ii) if, for a parameter, the number of distinct values exceeds a first predetermined number, setting the number of distinct values to the first predetermined number;

(iii) for each parameter, creating a statistical meta data table, with a row allocated for each distinct value of the parameter, and the count of data records from the data sources having that distinct value, up to the first predetermined number; and

(iv) saving the statistical meta data tables for each of the parameters.

24. The method of claim 23, further comprising:

(v) if a parameter is continuous or specially designated, creating a population meta data table with a second predetermined number of rows corresponding to data bins into which the parameter will be categorized;

(vi) allocating the ranges of the second predetermined of data bins;

(vii) setting each row of the population data table to the range of the corresponding data bin, and the count of the data records from the data sources of the parameter falling within that range; and

(ix) saving the population data table.

25. The method of claim, 24, further comprising:

(x) repeating steps (v)-(ix) in the event that the parameter is re-categorized.

26. The method of claim 24 wherein the allocating step is performed on a equidistant basis.

27. The method of claim 24 wherein the allocating step is performed on a equidistributed basis.

28. The method of claim 24 wherein the allocating step is performed on a fuzzy distributed basis.

29. A method of data mining using a discovery domain comprised of a plurality of data records, each data record comprised of a plurality of parameters, one of the plurality of parameters termed an output parameter, all other of the plurality of parameters termed input parameters, each parameter categorized into a plurality of data bins, a neuroagent initialized for each represented data bin of the plurality of data bins for each parameter, each neuroagent of an input parameter termed an input neuroagent, each neuroagent of an output parameter termed an output neuroagent, each neuroagent including a communication and activation envelope, the method comprising the steps of:

(i) creating a population meta data table for each parameter in accordance with which one or more neuroagents which are initialized;

(ii) for a first data record of the plurality of data records whose output parameter corresponds to a first output neuroagent,

(a) identifying in the first data record a first set of neuroagents corresponding to data bins represented in the first data record,

(b) forming a first neuroexpression from the first set of neuroagents, and

(c) connecting the first neuroexpression to the communication and activation envelope of the first output neuroagent;

(iii) for any subsequent second data record of the plurality of data records whose output parameter corresponds to the first output neuroagent, a current set of neuroexpressions connected to the communication and activation envelope of the first output neuroagent, each of the current set of neuroexpressions including a current set of neuroagents,

(a) identifying a second set of neuroagents corresponding to data bins represented in the second data record,

(b) forming a second neuroexpression from the second set of neuroagents,

(c) forming a new neuroexpression from a difference of the second neuroexpression and a union of the current set of neuroexpressions;

(d) from each current neuroexpression of the current set of neuroexpressions, forming a common neuroexpression as an intersection of the current neuroexpression with the second neuroexpression, and a non-common neuroexpression as a difference of the current neuroexpression with the second neuroexpression, and

(c) from the communication and activation envelope of the first output neuroagent, disconnecting the current set of neuroexpressions and connecting the new neuroexpression and for each of the current set of neuroexpressions, the common neuroexpressions and the non-common neuroexpressions, all these neuroexpressions so connected designated as a new current set of neuroexpressions;

(iv) repeating step (iii) above for all the subsequent second data records; and

(v) for the network of neuroagents formed by steps (i)-(iv) above, calculating the importance of at least one input neuroagent towards at least one output neuroagent.

30. The method of claim 29, wherein at least one of the identifications performed in steps (ii)(a) or (iii)(a) is performed through a lookup of the population meta data table.

31. The method of claim 29, wherein further:

the communication and activation envelope of each output neuroagent having a minimal excitation zone and a contextual excitation zone,

for the first data record, the first neuroexpression is connected to the minimal excitation zone of the first output neuroagent,

for all the subsequent data records, at most one of the new current neuroexpressions is connected to the minimal excitation zone of the first output neuroagent, formed by the intersection of the current neuroexpression with at most one of the current neuroexpressions that was disconnected from the minimal excitation zone of the first output neuroagent in step (iii).

32. The method of claim 29, wherein importance, for a given output neuroagent and a certain neuroexpression connected to the communication and activation envelope of the output neuroagent, is calculated as the probability that the given output neuroagent is validated when the certain neuroexpression is true.