Asynchronous data mining system for database management system

• US 7,328,192 B1
• Filed: 08/29/2002
• Issued: 02/05/2008
• Est. Priority Date: 05/10/2002
• Status: Expired due to Term

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Venkayala, Sunil, Stengard, Peter
• Primary Examiner(s): Vincent; David
• Assistant Examiner(s): Tran; Mai T
• Application Number:
  US10/230,267
• Time in Patent Office: 1,986 Days
• Field of Search: 707 1-206, 706/12
• US Class Current: 706/12
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99936 Pattern matching access

Abstract

A data mining system for a database management system and method and computer program product therefore, that provides improved functionality over synchronous data mining systems, and which provides features such as interruptible tasks and status output. A data mining system for a database management system comprises a plurality of data mining task objects operable to perform data mining functions, a data mining task queue table operable to maintain at least one queue to manage execution of the data mining tasks, and a data mining system task monitor operable to monitor execution of currently executing data mining tasks, examine the data mining task queue table, select at least one task for execution, dequeue data mining tasks from the data mining task queue table, and initiate execution of the dequeued tasks.

Claims

1. A computer-implemented data mining system configured to identify patterns and relationships among a collection of stored data for a computerized database management system comprising a processor operable to execute computer program instructions and a memory operable to store computer program instructions executable by the processor, comprising:

a plurality of data mining task objects operable to perform data mining functions upon data mining models, wherein each data mining model accepts as input at least a portion of the collection of stored data representing tangible activities including at least one of;

data collected by companies and other organizations, accounting activities, billing activities, profiling of customer activities and relations, manufacturing operations, web-site activity, and marketing activities, and identifies and outputs patterns and relationships among the collection of stored data, the output patterns and relationships representing actual patterns and relationships among the tangible activities represented by the collection of stored data, wherein the plurality of data mining task objects are selected from a group of data mining task objects comprising;

a data mining build task object operable to perform a data mining function for building a data mining model for identifying the patterns and relationships among the collection of stored data, a data mining apply task object operable to perform a data mining function for applying a built data mining model to the collection of stored data to score the collection of stored data, and compute associated probabilities for the collection of stored data;

a data mining test task object operable to perform a data mining function for testing a built mining model with a dataset to find an accuracy of the data mining model and compute a confusion matrix;

a data mining lift task object operable to perform a data mining function for computing a lift of a data mining model, based on a positive target value and a number of quantiles, a data mining cross-validation task object operable to perform a data mining function for performing a cross-validation on a data mining model to determine an accuracy of the data mining model, a data mining import task object operable to perform a data mining function for importing a data mining model in an external format to an internal format of the data mining system, and a data mining export task object operable to perform a data mining function for exporting a data mining model in an internal format of the data mining system to an external format;

a data mining task queue table operable to maintain at least one queue to manage execution of the data mining task objects; and

a data mining system task monitor operable to monitor execution of currently executing data mining task objects, examine the data mining task queue table, select at least one data mining task object for execution, dequeue data mining task objects from the data mining task queue table, and initiate execution of the dequeued task objects.

2. The computer-implemented data mining system of claim 1, wherein a data mining task object to be enqueued in the data mining task queue table is received from a client application.

3. The computer-implemented data mining system of claim 2, wherein control is returned to the client application when enqueuing of the data mining task object is complete.

4. The computer-implemented data mining system of claim 3, wherein data mining system task monitor is further operable to perform a data mining functions for determining a number of concurrent data mining task objects executing in the data mining system, initiating execution of at least one data mining task objects, if the number of concurrent data mining task objects running in the data mining system is less than a maximum number of concurrent data mining data mining task objects that the data mining system can handle, and continue to monitor the data mining task objects that are being executed.

5. The computer-implemented data mining system of claim 4, wherein each of the data mining task objects has a state indicating a behavior of the data mining task object.

6. The computer-implemented data mining system of claim 5, wherein the state of a data mining task object comprises:

a ready state indicating that the data mining task object is ready to execute;

a queued state indicating that the data mining task object is in the data mining task queue table and waiting to be executed;

an initiated state indicating that the data mining task object is actually submitted for execution;

an executing state indicating that the data mining task object is in execution;

a success state indicating that a data mining task object execution succeeded;

an error state indicating that the data mining task object execution failed;

a terminating state indicating that the data mining task object is in a process of terminating; and

a terminated state indicating that the data mining task object is terminated.

7. A computer-implemented method of performing data mining to identify patterns and relationships among a collection of stored data in a computerized database management system, comprising the steps of:

receiving a data mining task object operable to perform a data mining function upon data mining models, wherein each data mining model accepts as input at least a portion of the collection of stored data representing tangible activities including at least one of;

data collected by companies and other organizations, accounting activities, billing activities, profiling of customer activities and relations, manufacturing operations, web-site activity, and marketing activities, and identifies and outputs patterns and relationships among the collection of stored data, the output patterns and relationships representing actual patterns and relationships among the tangible activities represented by the collection of stored data, the data mining task object to be enqueued in a data mining task queue table, wherein the data mining task object is selected from a group of data mining task objects comprising;

a data mining build task object operable to perform a data mining function for building a data mining model for identifying the patterns and relationships among the collection of stored data, a data mining apply task object operable to perform a data mining function for applying a built data mining model to the collection of stored data to score the collection of stored data, and compute associated probabilities for the collection of stored data;

a data mining test task object operable to perform a data mining function for testing a built mining model with a dataset to find an accuracy of the data mining model and compute a confusion matrix;

a data mining lift task object operable to perform a data mining function for computing a lift of a data mining model, based on a positive target value and a number of quantiles, a data mining cross-validation task object operable to perform a data mining function for performing a cross-validation on a data mining model to determine an accuracy of the data mining model, a data mining import task object operable to perform a data mining function for importing a data mining model in an external format to an internal format of the data mining system, and a data mining export task object operable to perform a data mining function for exporting a data mining model in an internal format of the data mining system to an external format;

enqueuing the received data mining task object in the data mining task queue table;

monitoring execution of currently executing data mining task objects examining the data mining task queue table;

selecting at least one data mining task object for execution;

dequeuing the selected data mining task object from the data mining task queue table; and

initiating execution of the dequeued data mining task object.

8. The computer-implemented method of claim 7, wherein the data mining task object to be enqueued is received from a client application.

9. The computer-implemented method of claim 8, further comprising the step of:

returning control to the client application when enqueuing of the data mining task object is complete.

10. The computer-implemented method of claim 9, wherein the step of monitoring execution of currently executing data mining tasks comprises the steps of:

determining a number of concurrent data mining task objects executing in the data mining system;

initiating execution of at least one data mining task objects, if the number of concurrent data mining task objects running in the data mining system is less than a maximum number of concurrent data mining task objects that the data mining system can handle; and

continuing to monitor the data mining task objects that are being executed, if the number of concurrent data mining task objects running in the data mining system is greater than or equal to the maximum number of concurrent data mining task objects that the data mining system can handle.

11. The computer-implemented method of claim 10, wherein each of the data mining task objects has a state indicating a behavior of the data mining task object.

12. The computer-implemented method of claim 11, wherein the state of a data mining task object comprises:

a ready state indicating that the data mining task object is ready to execute;

a queued state indicating that the data mining task object is in the data mining task queue table and waiting to be executed;

an initiated state indicating that the data mining task object is actually submitted for execution;

an executing state indicating that the data mining task object is in execution;

a success state indicating that a data mining task object execution succeeded;

an error state indicating that the data mining task object execution failed;

a terminating state indicating that the data mining task object is in a process of terminating; and

a terminated state indicating that the data mining task object is terminated.

13. A system for performing data mining to identify patterns and relationships among a collection of stored data in a database management system, comprising:

a processor operable to execute computer program instructions; and

a memory operable to store computer program instructions executable by the processor, for performing the steps of;

receiving a data mining task object operable to perform a data mining function upon data mining models, wherein each data mining model accepts as input at least a portion of the collection of stored data representing tangible activities including at least one of;

data collected by companies and other organizations, accounting activities, billing activities, profiling of customer activities and relations, manufacturing operations, web-site activity, and marketing activities, and identifies and outputs for identifying the patterns and relationships among the collection of stored data, the output relationships representing actual patterns and relationships among the tangible activities represented by the collection of stored data, the data mining task object to be enqueued in a data mining task queue table, wherein the data mining task object is selected from a group of data mining task objects comprising;

a data mining build task object operable to perform a data mining function for building a data mining model for identifying the patterns and relationships among the collection of stored data, a data mining apply task object operable to perform a data mining function for applying a built data mining model to the collection of stored data to score the collection of stored data, and compute associated probabilities for the collection of stored data;

a data mining test task object operable to perform a data mining function for testing a built mining model with a dataset to find an accuracy of the data mining model and compute a confusion matrix;

a data mining lift task object operable to perform a data mining function for computing a lift of a data mining model, based on a positive target value and a number of quantiles, a data mining cross-validation task object operable to perform a data mining function for performing a cross-validation on a data mining model to determine an accuracy of the data mining model, a data mining import task object operable to perform a data mining function for importing a data mining model in an external format to an internal format of the data mining system, and a data mining export task object operable to perform a data mining function for exporting a data mining model in an internal format of the data mining system to an external format;

enqueuing the received data mining task object in the data mining task queue table;

monitoring execution of currently executing data mining task objects examining the data mining task queue table;

selecting at least one data mining task object for execution;

dequeuing the selected data mining task object from the data mining task queue table; and

initiating execution of the dequeued data mining task object.

14. The system of claim 13, wherein the data mining task object to be enqueued is received from a client application.

15. The system of claim 14, further comprising the step of:

returning control to the client application when enqueuing of the data mining task object is complete.

16. The system of claim 15, wherein the step of monitoring execution of currently executing data mining tasks comprises the steps of:

determining a number of concurrent data mining task objects executing in the data mining system;

initiating execution of at least one data mining task objects, if the number of concurrent data mining task objects running in the data mining system is less than a maximum number of concurrent data mining task objects that the data mining system can handle; and

continuing to monitor the data mining task objects that are being executed.

17. The system of claim 16, wherein each of the data mining task objects has a state indicating a behavior of the data mining task object.

18. The system of claim 17, wherein the state of a data mining task object comprises:

a ready state indicating that the data mining task object is ready to execute;

a queued state indicating that the data mining task object is in the data mining task queue table and waiting to be executed;

an initiated state indicating that the data mining task object is actually submitted for execution;

an executing state indicating that the data mining task object is in execution;

a success state indicating that a data mining task object execution succeeded;

an error state indicating that the data mining task object execution failed;

a terminating state indicating that the data mining task object is in a process of terminating; and

a terminated state indicating that the data mining task object is terminated.

19. A computer program product for performing data mining to identify patterns and relationships among a collection of stored data in a database management system, comprising:

a computer readable storage medium;

computer program instructions, recorded on the computer readable storage medium, executable by a processor, for performing the steps of;

receiving a data mining task object operable to perform a data mining function upon data mining models, wherein each data mining model accepts as input at least a portion of the collection of stored data representing tangible activities including at least one of;

data collected by companies and other organizations, accounting activities, billing activities, profiling of customer activities and relations, manufacturing operations, web-site activity, and marketing activities, and identifies and outputs patterns and relationships among the collection of stored data, the output patterns and relationships representing actual patterns and relationships among the tangible activities represented by the collection of stored data, the data mining task object to be enqueued in a data mining task queue table, wherein the data mining task object is selected from a group of data mining task objects comprising;

a data mining build task object operable to perform a data mining function for building a data mining model for identifying the patterns and relationships among the collection of stored data, a data mining apply task object operable to perform a data mining function for applying a built data mining model to the collection of stored data to score the collection of stored data, and compute associated probabilities for the collection of stored data;

a data mining test task object operable to perform a data mining function for testing a built mining model with a dataset to find an accuracy of the data mining model and compute a confusion matrix;

a data mining lift task object operable to perform a data mining function for computing a lift of a data mining model, based on a positive target value and a number of quantiles, a data mining cross-validation task object operable to perform a data mining function for performing a cross-validation on a data mining model to determine an accuracy of the data mining model, a data mining import task object operable to perform a data mining function for importing a data mining model in an external format to an internal format of the data mining system, and a data mining export task object operable to perform a data mining function for exporting a data mining model in an internal format of the data mining system to an external format;

enqueuing the received data mining task object in the data mining task queue table;

monitoring execution of currently executing data mining task objects;

examining the data mining task queue table;

selecting at least one data mining task object for execution;

dequeuing the selected data mining task object from the data mining task queue table; and

initiating execution of the dequeued data mining task object.

20. The computer program product of claim 19, wherein the data mining task object to be enqueued is received from a client application.

21. The computer program product of claim 20, further comprising the step of:

returning control to the client application when enqueuing of the data mining task object is complete.

22. The computer program product of claim 21, wherein the step of monitoring execution of currently executing data mining tasks comprises the steps of:

determining a number of concurrent data mining task objects executing in the data mining system;

initiating execution of at least one data mining task objects, if the number of concurrent data mining task objects running in the data mining system is less than a maximum number of concurrent data mining task objects that the data mining system can handle; and

continuing to monitor the data mining task objects that are being executed.

23. The computer program product of claim 22, wherein each of the data mining task objects has a state indicating a behavior of the data mining task object.

24. The computer program product of claim 23, wherein the state of a data mining task comprises:

a ready state indicating that the data mining task object is ready to execute;

a queued state indicating that the data mining task object is in the data mining task queue table and waiting to be executed;

an initiated state indicating that the data mining task object is actually submitted for execution;

an executing state indicating that the data mining task object is in execution;

a success state indicating that a data mining task object execution succeeded;

an error state indicating that the data mining task object execution failed;

a terminating state indicating that the data mining task object is in a process of terminating; and

a terminated state indicating that the data mining task object is terminated.