Data mining agents for efficient hardware utilization

• US 6,912,533 B1
• Filed: 07/31/2001
• Issued: 06/28/2005
• Est. Priority Date: 07/31/2001
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Hornick, Mark
• Primary Examiner(s): Alam, Shahid
• Assistant Examiner(s): FLEURANTIN, JEAN B
• Application Number:
  US09/917,763
• Time in Patent Office: 1,428 Days
• Field of Search: 707 1- 10, 707100-1041, 707200-206, 709201-202
• US Class Current: 707/700
• CPC Class Codes: G06F 2209/5019 Workload prediction
• CPC Class Codes: G06F 2209/503 Resource availability
• CPC Class Codes: G06F 2216/03 Data mining
• CPC Class Codes: G06F 9/505 considering the load
• CPC Class Codes: G06F 9/5055 considering software capabilities, i.e. software resources associated or available to the machine
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

A method, system, and computer program product for allocating data mining processing tasks that does not use complex internal schemes, yet results in better performance than is possible with general-purpose operating system based schemes. The present invention uses a data mining agent that operates autonomously, proactively, reactively, deliberatively, and cooperatively to allocate and reallocate data mining processing tasks among computer systems, and/or among processors. The data mining agent reacts to its own environment, determines if a data mining activity can be completed as expected, solicits bids from other data mining agents, determines if anther data mining system could complete the data mining activity and migrates that data mining activity to the selected data mining system.

Claims

1. In a data mining agent executing in a computer system, a method of data mining comprising the steps of:

examining a request queue comprising at least one request for data mining processing;

determining if the at least one request for data mining processing can be processed;

accepting the at least one request for data mining processing if it is determined that the at least one request for data mining processing can be processed; and

processing the accepted request for data mining processing in the computer system;

wherein the determining step comprises the steps of;

determining if an algorithm required to process the at least one request for data mining processing is supported by the computer system;

if the algorithm required to process the at least one request for data mining processing is supported determining whether the computer system is available for additional processing;

if the computer system is not available for additional processing, determining whether the computer system will become available for additional processing before other computer systems that might process the at least one request;

if the computer system is available for additional processing, or if the computer system will become available for additional processing before other computer systems that might process the at least one request, determining whether the computer system will be able to complete requested processing in an allotted time; and

if the computer system will be able to complete the requested processing in the allotted time, determining that the computer system can process the at least one request for data mining processing.

2. The method of claim 1, wherein the at least one request for data mining processing comprises data defining at least one algorithm that must be performed in order to perform the requested data mining processing.

3. The method of claim 2, wherein there is data defining algorithms that are supported by the computer system.

4. The method of claim 3, wherein the step of determining if an algorithm required to process the at least one request for data mining processing is supported comprises comparing the data defining at least one algorithm that must be performed in order to perform the requested data mining processing with data defining algorithms that are supported by the computer system.

5. The method of claim 4, wherein the data defining at least one algorithm that must be performed in order to perform the requested data mining processing and the data defining algorithms that are supported by the computer system are in extensible markup language format.

6. The method of claim 4, wherein the step of determining whether the computer system is available for additional processing comprises the step of:

determining whether available idle time of the computer system is greater than a predefined or a dynamically calculated threshold.

7. The method of claim 4, wherein the computer system comprises a plurality of processors and the step of determining whether the computer system is available for additional processing comprises the step of:

determining whether any of the plurality of processors is available for additional processing.

8. The method of claim 7, wherein the step of determining whether any of the plurality of processors is available for additional processing comprises the step of

determining, for each of the plurality of processors, whether available idle time of the processor is greater than a predefined or a dynamically calculated threshold.

9. The method of claim 4, wherein the step of determining whether the computer system is available for additional processing comprises the step of:

determining availability of the computer system for additional processing relative to at least one other computer system.

10. The method of claim 4, wherein the step of determining whether the computer system will become available for additional processing before other computer systems that might process the at least one request comprises the steps of:

estimating a time to availability of the computer system;

exchanging an estimate of a time to availability of the at least one other computer system; and

comparing the time to availability of the computer system with the time to availability of the at least one other computer system.

11. The method of claim 10, wherein the step of determining whether the computer system will be able to complete requested processing in an allotted time comprises the steps of:

estimating a time to completion for the requested processing on the computer system;

comparing the time to completion for the requested processing on the computer system with time allocation information included in the request for data mining processing.

12. A computer system for performing data mining, comprising:

a processor operable to execute computer program instructions;

a memory operable to store computer program instructions executable by the processor; and

a data mining agent comprising computer program instructions, for performing the steps of;

examining a request queue comprising at least one request for data mining processing,

determining if the at least one request for data mining processing can be processed,

accepting the at least one request for data mining processing if it is determined that the at least one request for data mining processing can be processed, and

processing the accepted request for data mining processing in the computer system;

wherein the determining step comprises the steps of;

determining if an algorithm required to process the at least one request for data mining processing is supported by the computer system;

if the algorithm required to process the at least one request for data mining processing is supported, determining whether the computer system is available for additional processing;

if the computer system is not available for additional processing, determining whether the computer system will become available for additional processing before other computer systems that might process the at least one request;

if the computer system is available for additional processing, or if the computer system will become available for additional processing before other computer systems that might process the at least one request, determining whether the computer system will be able to complete requested processing in an allotted time; and

if the computer system will be able to complete the requested processing in the allotted time, determining that the computer system can process the at least one request for data mining processing.

13. The system of claim 12, wherein the at least one request for data mining processing comprises data defining at least one algorithm that must be performed in order to perform the requested data mining processing.

14. The system of claim 13, wherein there is data defining algorithms that are supported by the computer system.

15. The system of claim 14, wherein the step of determining if an algorithm required to process the at least one request for data mining processing is supported comprises comparing the data defining at least one algorithm that must be performed in order to perform the requested data mining processing with data defining algorithms that are supported by the computer system.

16. The system of claim 15, wherein the data defining at least one algorithm that must be performed in order to perform the requested data mining processing and the data defining algorithms that are supported by the computer system are in extensible markup language format.

17. The system of claim 15, wherein the step of determining whether the computer system is available for additional processing comprises the step of:

determining whether available idle time of the computer system is greater than a predefined or a dynamically calculated threshold.

18. The system of claim 15, wherein the computer system comprises a plurality of processors and the step of determining whether the computer system is available for additional processing comprises the step of:

determining whether any of the plurality of processors is available for additional processing.

19. The system of claim 18, wherein the step of determining whether any of the plurality of processors is available for additional processing comprises the step of

determining, for each of the plurality of processors, whether available idle time of the processor is greater than a predefined or a dynamically calculated threshold.

20. The system of claim 15, wherein the step of determining whether the computer system is available for additional processing comprises the step of:

determining availability of the computer system for additional processing relative to at least one other computer system.

21. The system of claim 15, wherein the step of determining whether the computer system will become available for additional processing before other computer systems that might process the at least one request comprises the steps of:

estimating a time to availability of the computer system;

exchanging an estimate of a time to availability of the at least one other computer system; and

comparing the time to availability of the computer system with the time to availability of the at least one other computer system.

22. The system of claim 21, wherein the step of determining whether the computer system will be able to complete requested processing in an allotted time comprises the steps of:

estimating a time to completion for the requested processing on the computer system;

comparing the time to completion for the requested processing on the computer system with time allocation information included in the request for data mining processing.

23. A computer program product for performing data mining in a computer system data processing system, comprising:

a computer readable medium;

computer program instructions, recorded on the computer readable medium, executable by a processor, for performing the steps of;

examining a request queue comprising at least one request for data mining processing;

determining if the at least one request for data mining processing can be processed;

accepting the at least one request for data mining processing if it is determined that the at least one request for data mining processing can be processed; and

processing the accepted request for data mining processing in the computer system;

wherein the determining step comprises the steps of;

determining if an algorithm required to process the at least one request for data mining processing is supported by the computer system;

if the algorithm required to process the at least one request for data mining processing is supported, determining whether the computer system is available for additional processing;

if the computer system is not available for additional processing, determining whether the computer system will become available for additional processing before other computer systems that might process the at least one request;

if the computer system is available for additional processing, or if the computer system will become available for additional processing before other computer systems that might process the at least one request, determining whether the computer system will be able to complete requested processing in an allotted time; and

if the computer system will be able to complete the requested processing in the allotted time, determining that the computer system can process the at least one request for data mining processing.

24. The computer program product of claim 23, wherein the at least one request for data mining processing comprises data defining at least one algorithm that must be performed in order to perform the requested data mining processing.

25. The computer program product of claim 24, wherein there is data defining algorithms that are supported by the computer system.

26. The computer program product of claim 25, wherein the step of determining if an algorithm required to process the at least one request for data mining processing is supported comprises comparing the data defining at least one algorithm that must be performed in order to perform the requested data mining processing with data defining algorithms that are supported by the computer system.

27. The computer program product of claim 26, wherein the data defining at least one algorithm that must be performed in order to perform the requested data mining processing and the data defining algorithms that are supported by the computer system are in extensible markup language format.

28. The computer program product of claim 26, wherein the step of determining whether the computer system is available for additional processing comprises the step of:

determining whether available idle time of the computer system is greater than a predefined or a dynamically calculated threshold.

29. The computer program product of claim 26, wherein the computer system comprises a plurality of processors and the step of determining whether the computer system is available for additional processing comprises the step of:

determining whether any of the plurality of processors is available for additional processing.

30. The computer program product of claim 29, wherein the step of determining whether any of the plurality of processors is available for additional processing comprises the step of

determining, for each of the plurality of processors, whether available idle time of the processor is greater than a predefined or a dynamically calculated threshold.

31. The computer program product of claim 26, wherein the step of determining whether the computer system is available for additional processing comprises the step of:

determining availability of the computer system for additional processing relative to at least one other computer system.

32. The computer program product of claim 26, wherein the step of determining whether the computer system will become available for additional processing before other computer systems that might process the at least one request comprises the steps of:

estimating a time to availability of the computer system;

exchanging an estimate of a time to availability of the at least one other computer system; and

comparing the time to availability of the computer system with the time to availability of the at least one other computer system.

33. The computer program product of claim 32, wherein the step of determining whether the computer system will be able to complete requested processing in an allotted time comprises the steps of:

estimating a time to completion for the requested processing on the computer system;

comparing the time to completion for the requested processing on the computer system with time allocation information included in the request for data mining processing.

34. A data mining agent for managing data mining in a computer system, the data mining agent comprising:

means for examining a request queue comprising at least one request for data mining processing;

means for determining if the at least one request for data mining processing can be processed;

means for accepting the at least one request for data mining processing if it is determined that the at least one request for data mining processing can be processed; and

means for processing the accepted request for data mining processing in the computer system;

wherein the determining means comprises;

means for determining if an algorithm required to process the at least one request for data mining processing is supported by the computer system;

means for determining whether the computer system is available for additional processing, if the algorithm required to process the at least one request for data mining processing is supported;

means for determining whether the computer system will become available for additional processing before other computer systems that might process the at least one request, if the computer system is not available for additional processing;

means for determining whether the computer system will be able to complete requested processing in an allotted time, if the computer system is available for additional processing, or if the computer system will become available for additional processing before other computer systems that might process the at least one request; and

means for determining that the computer system can process the at least one request for data mining processing, if the computer system will be able to complete the requested processing in the allotted time.

35. The data mining agent of claim 34, wherein the at least one request for data mining processing comprises data defining at least one algorithm that must be performed in order to perform the requested data mining processing.

36. The data mining agent of claim 35, wherein there is data defining algorithms that are supported by the computer system.

37. The data mining agent of claim 36, wherein the means for determining if an algorithm required to process the at least one request for data mining processing is supported comprises means for comparing the data defining at least one algorithm that must be performed in order to perform the requested data mining processing with data defining algorithms that are supported by the computer system.

38. The data mining agent of claim 37, wherein the data defining at least one algorithm that must be performed in order to perform the requested data mining processing and the data defining algorithms that are supported by the computer system are in extensible markup language format.

39. The data mining agent of claim 37, wherein the means for determining whether the computer system is available for additional processing comprises:

means for determining whether available idle time of the computer system is greater than a predefined or a dynamically calculated threshold.

40. The data mining agent of claim 37, wherein the computer system comprises a plurality of processors and the means for determining whether the computer system is available for additional processing comprises:

means for determining whether any of the plurality of processors is available for additional processing.

41. The data mining agent of claim 40, wherein the means for determining whether any of the plurality of processors is available for additional processing comprises:

means for determining, for each of the plurality of processors, whether available idle time of the processor is greater than a predefined or a dynamically calculated threshold.

42. The data mining agent of claim 37, wherein the means for determining whether the computer system is available for additional processing comprises:

means for determining availability of the computer system for additional processing relative to at least one other computer system.

43. The data mining agent of claim 37, wherein the means for determining whether the computer system will become available for additional processing before other computer systems that might process the at least one request comprises:

means for estimating a time to availability of the computer system;

means for exchanging an estimate of a time to availability of the at least one other computer system; and

means for comparing the time to availability of the computer system with the time to availability of the at least one other computer system.

44. The data mining agent of claim 43, wherein the means for determining whether the computer system will be able to complete requested processing in an allotted time comprises:

means for estimating a time to completion for the requested processing on the computer system;

means for comparing the time to completion for the requested processing on the computer system with time allocation information included in the request for data mining processing.