System and method for selecting a data mining modeling algorithm for data mining applications

• US 20070011188A1
• Filed: 07/05/2005
• Published: 01/11/2007
• Est. Priority Date: 07/05/2005
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Chitgupakar, Milind, Ramsey, Mark, Selby, David
• Granted Patent: US 7,512,626 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2465 Query processing support for facilitating data mining operations in structured databases
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99937 Sorting
• CPC Class Codes: Y10S 707/99938 Concurrency, e.g. lock management in shared database
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface

Abstract

A computing system and method for selecting a data mining modeling algorithm. The computing system comprises a computer readable medium and computing devices electrically coupled through an interface apparatus. A plurality of different data mining modeling algorithms and test data are stored on the computer readable medium. Each of the computing devices comprises a data subset from a plurality of data subsets. A technique is selected for generating a data mining model applied to each of the data subsets. Each of the different data mining modeling algorithms is run simultaneously to generate an associated data mining model on each of the computing devices. Each of the data mining models is compared to the test data to determine a best data model. A best data mining modeling algorithm from the different data mining modeling algorithms is selected in accordance with the best data mining model.

Claims

1. A data mining method, comprising:

providing a computing system comprising a computer readable medium and computing devices electrically coupled through an interface apparatus, wherein a plurality of different data mining modeling algorithms and test data are stored on said computer readable medium, wherein each of said computing devices comprises at least one central processing unit (CPU) and an associated memory device, wherein data has been divided by the computing system into a plurality of data subsets, and wherein each of said associated memory devices comprises a data subset from said plurality of data subsets;

selecting a technique for generating a data mining model applied to each of said data subsets;

running simultaneously, each of said different data mining modeling algorithms using said selected technique to generate an associated data mining model on each of said computing devices;

comparing each of said data mining models on each of said computing devices to said test data to determine a best data model of said data mining models; and

determining, a best data mining modeling algorithm from said different data mining modeling algorithms in accordance with said selected technique, wherein said best data mining modeling algorithm is the data mining modeling algorithm that is associated with said best data mining model.

2. The data mining method of claim 1, wherein said test data comprises existing data related to a marketing offer accepted by a first plurality of candidates, and wherein each of said data mining models comprises an acceptance probability that said marketing offer will be accepted by a second plurality of candidates.

3. The data mining method of claim 2, wherein said best data mining model comprises a higher acceptance probability than said acceptance probabilities for any other of said data mining models.

4. The data mining method of claim 1, wherein said selected technique comprises a lift chart technique, and wherein said method further comprises:

determining by said lift chart technique an effectiveness of each of said generated data mining models.

5. The data mining method of claim 1, wherein said selected technique comprises a root mean squared technique, and wherein said method further comprises:

determining by said root mean squared technique, an error for each of said generated data mining models.

6. The data mining method of claim 1, wherein each of said data mining modeling algorithms are selected from the group consisting of a decision tree algorithm, a clustering algorithm, a radial basis function algorithm, a linear regression algorithm, an associations algorithm, and a neural network algorithm.

7. The data mining method of claim 1, wherein said interface apparatus is a high speed switching apparatus.

8. The data mining method of claim 1, wherein said computing system comprises a system selected from the group consisting of a massively parallel processing system, a symmetric multiprocessing system, and a combination of a massively parallel processing system and a symmetric multiprocessing system.

9. The data mining method of claim 1, wherein said computing system further comprises a relational database software system.

10. A computing system comprising a processor coupled to a computer readable medium and computing devices electrically coupled through an interface apparatus, wherein said computer readable medium comprises a plurality of different data mining modeling algorithms, test data, and instructions that when executed by the processor implement a data mining method, wherein each of said computing devices comprises at least one central processing unit (CPU) and an associated memory device, wherein data has been divided by the computing system into a plurality of data subsets, and wherein each of said associated memory devices comprises a data subset from said plurality of data subsets, said method comprising the computer implemented steps of:

selecting a technique for generating a data mining model applied to each of said data subsets;

running simultaneously, each of said different data mining modeling algorithms using said selected technique to generate an associated data mining model on each of said computing devices;

comparing each of said data mining models on each of said computing devices to said test data to determine a best data model of said data mining models; and

determining, a best data mining modeling algorithm from said different data mining modeling algorithms in accordance with said selected technique, wherein said best data mining modeling algorithm is the data mining modeling algorithm that is associated with said best data mining model.

11. The computing system of claim 10, wherein said test data comprises existing data related to a marketing offer accepted by a first plurality of candidates, and wherein each of said data mining models comprises an acceptance probability that said marketing offer will be accepted by a second plurality of candidates.

12. The computing system of claim 11, wherein said best data mining model comprises a higher acceptance probability than said acceptance probabilities for any other of said data mining models.

13. The computing system of claim 10, wherein said selected technique comprises a lift chart technique, and wherein said method further comprises:

determining by said lift chart technique an effectiveness of each of said generated data mining models.

14. The computing system of claim 10, wherein said selected technique comprises and a root mean squared technique, and wherein said method further comprises:

determining by said root mean squared technique, an error for each of said generated data mining models.

15. The computing system of claim 10, wherein each of said data mining modeling algorithms are selected from the group consisting of a decision tree algorithm, a clustering algorithm, a radial basis function algorithm, a linear regression algorithm, an associations algorithm, and a neural network algorithm.

16. The computing system of claim 10, wherein said interface apparatus is a high speed switching apparatus.

17. The computing system of claim 10, wherein said computing devices electrically coupled through said interface apparatus is a computing system selected from the group consisting of a massively parallel processing system, a symmetric multiprocessing system, and a combination of a massively parallel processing system and a symmetric multiprocessing system.

18. The computing system of claim 10, wherein said computing devices electrically coupled through said interface apparatus comprise a relational database software system.

19. A process for integrating computing infrastructure, comprising integrating computer-readable code into a computing system, wherein the code in combination with the computing system comprises a computer readable medium and computing devices electrically coupled through an interface apparatus, wherein a plurality of different data mining modeling algorithms and test data are stored on said computer readable medium, wherein each of said computing devices comprises at least one central processing unit (CPU) and an associated memory device, wherein data has been divided by the computing system into a plurality of data subsets, and wherein each of said associated memory devices comprises a data subset from said plurality of data subsets, and wherein the code in combination with the computing system is adapted to implement a method for performing the steps of:

selecting a technique for generating a data mining model applied to each of said data subsets;

running simultaneously, each of said different data mining modeling algorithms using said selected technique to generate an associated data mining model on each of said computing devices;

comparing each of said data mining models on each of said computing devices to said test data to determine a best data model of said data mining models; and

determining, a best data mining modeling algorithm from said different data mining modeling algorithms in accordance with said selected technique, wherein said best data mining modeling algorithm is the data mining modeling algorithm that is associated with said best data mining model.

20. The process of claim of claim 19, wherein said test data comprises existing data related to a marketing offer accepted by a first plurality of candidates, and wherein each of said data mining models comprises an acceptance probability that said marketing offer will be accepted by a second plurality of candidates.

21. The process of claim 20, wherein said best data mining model comprises a higher acceptance probability than said acceptance probabilities for any other of said data mining models.

22. The process of claim 19, wherein said selected technique comprises a lift chart technique, and wherein said method further comprises:

determining by said lift chart technique an effectiveness of each of said generated data mining models.

23. The process of claim 19, wherein said selected technique comprises and a root mean squared technique, and wherein said method further comprises:

determining by said root mean squared technique, an error for each of said generated data mining models.

24. The process of claim 19, wherein each of said data mining modeling algorithms are selected from the group consisting of a decision tree algorithm, a clustering algorithm, a radial basis function algorithm, a linear regression algorithm, an associations algorithm, and a neural network algorithm.

25. The process of claim 19, wherein said interface apparatus is a high speed switching apparatus.

26. The process of claim 19, wherein said computing system comprises a system selected from the group consisting of a massively parallel processing system, a symmetric multiprocessing system, and a combination of a massively parallel processing system and a symmetric multiprocessing system.

27. The process of claim 19, wherein said computing system further comprises a relational database software system.

28. A computer program product, comprising a computer usable medium having a computer readable program code embodied therein, said computer readable program code comprising an algorithm adapted to implement a data mining method within a computing system, said computing system comprising a computer readable medium and computing devices electrically coupled through an interface apparatus, wherein a plurality of different data mining modeling algorithms and test data are stored on said computer readable medium, wherein each of said computing devices comprises at least one central processing unit (CPU) and an associated memory device, wherein data has been divided by the computing system into a plurality of data subsets, and wherein each of said associated memory devices comprises a data subset from said plurality of data subsets, said method comprising the steps of:

selecting a technique for generating a data mining model applied to each of said data subsets;

running simultaneously, each of said different data mining modeling algorithms using said selected technique to generate an associated data mining model on each of said computing devices;

comparing each of said data mining models on each of said computing devices to said test data to determine a best data model of said data mining models; and

determining, a best data mining modeling algorithm from said different data mining modeling algorithms in accordance with said selected technique, wherein said best data mining modeling algorithm is the data mining modeling algorithm that is associated with said best data mining model.

29. The computer program product of claim 28, wherein said test data comprises existing data related to a marketing offer accepted by a first plurality of candidates, and wherein each of said data mining models comprises an acceptance probability that said marketing offer will be accepted by a second plurality of candidates.

30. The computer program product of claim 29, wherein said best data mining model comprises a higher acceptance probability than said acceptance probabilities for any other of said data mining models.

31. The computer program product of claim 28, wherein said selected technique comprises a lift chart technique, and wherein said method further comprises:

determining by said lift chart technique an effectiveness of each of said generated data mining models.

32. The computer program product of claim 28, wherein said selected technique comprises and a root mean squared technique, and wherein said method further comprises:

determining by said root mean squared technique, an error for each of said generated data mining models.

33. The computer program product of claim 28, wherein each of said data mining modeling algorithms are selected from the group consisting of a decision tree algorithm, a clustering algorithm, a radial basis function algorithm, a linear regression algorithm, an associations algorithm, and a neural network algorithm.

34. The computer program product of claim 28, wherein said interface apparatus is a high speed switching apparatus.

35. The computer program product of claim 28, wherein said computing system comprises a system selected from the group consisting of a massively parallel processing system, a symmetric multiprocessing system, and a combination of a massively parallel processing system and a symmetric multiprocessing system.

36. The computer program product of claim 28, wherein said computing system further comprises a relational database software system.