Method and apparatus for improving the performance of a database management system through a central cache mechanism

• US 6,073,129 A
• Filed: 12/29/1997
• Issued: 06/06/2000
• Est. Priority Date: 12/29/1997
• Status: Expired due to Term

• Current Assignee: Bull HN Information Systems Incorporated (Atos SE)
• Original Assignee: Bull HN Information Systems Incorporated (Atos SE)
• Inventors: Egolf, David A., Levine, Donald P.
• Primary Examiner(s): Amsbury, Wayne
• Assistant Examiner(s): PARDO, THUY N
• Application Number:
  US08/999,248
• Time in Patent Office: 890 Days
• Field of Search: 711/122, 711/144, 711/121, 711/140, 711/164, 711/119, 707/101, 707/10, 707/3, 707/4, 707/2, 707/103, 707/5, 379/207
• US Class Current: 1/1
• CPC Class Codes: G06F 12/0813 with a network or matrix configuration
• CPC Class Codes: G06F 16/235 Update request formulation
• CPC Class Codes: G06F 16/24524 Access plan code generation and invalidation; Reuse of access plans
• CPC Class Codes: G06F 16/24552 Database cache management
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation

Abstract

A host system includes a multicache system configured within the host system'"'"'s memory which has a plurality of local and central cache systems used for storing information being utilized by a plurality of processes running on the system. Persistent shared memory is used to store control structure information entries required for operating central cache systems for substantially long periods of time in conjunction with the local caches established for the processes. Such entries includes a descriptor value for identifying a directory control structure and individual sets of descriptors for identifying a group of control structures defining those components required for operating the configured central cache systems. The cache directory structure is used for defining the name of each configured central cache system and for providing an index value identifying the particular set of descriptors associated therewith. The multicache system also includes a plurality of interfaces for configuring the basic characteristics of both local and central cache systems as a function of the type and performance requirements of application processes being run.

Claims

1. A multicache system configurable within a primary memory of a host system for improving the performance of a number of processes running on the host system in processing complex database query statements, the multicache system comprising:

(a) a plurality of local caches in main memory corresponding to the number of processes being run on the host system, each local cache being associated with a different one of the processes being run; and

(b) at least one configured central cache system located in a predefined area of primary memory shared by the processes, the central cache system being operatively coupled to each of the plurality of local cache systems associated therewith, said central cache system including;

a number of shared control structures used for controlling access to the central cache system;

a number of sets of central cache control structures corresponding to the number of central caches configured for storing code segments; and

,

a control mechanism operatively coupled to the number of shared control structures and to the number of sets of central cache structures, the control mechanism being operative to control the configuration and operation of each of the central caches so that generated code segments produced by any one of the processes for executing a complex database query statement are stored automatically in the local cache associated with the process and in the central cache configured to be associated therewith for use by other processes associated with the same central cache in processing logically equivalent complex database query statements.

2. The system of claim 1 wherein the host system further includes a number of interfaces which operatively couple to the central cache system for configuring and operating the number of central caches.

3. The system of claim 2 wherein the multicache system further includes a cache utility program component operatively coupled to receive utility directives entered into the system by a user and operatively coupled to the central cache system, the cache utility program component operatively coupled to a first one of the number of interfaces, the cache utility program component in response to the user initiated directives being operative to issue utility functions for invoking the control mechanism to perform a sequence of operations for carrying out the specified functions.

4. The system of claim 3 wherein the utility directives include commands for specifying clearing, creating and deleting central caches within the central cache system.

5. The system of claim 2 wherein the host system includes a configuration file for storing user entered configuration parameters defining the characteristics of the central caches to be configured, the number of interfaces including a second interface operatively coupled to the configuration file and to the central cache system, the second interface being operative when a process is put in execution to invoke the control mechanism for storing parameter information in specific ones of the shared control structures and into a specified one of the number of sets of central cache control structures derived from the configuration file.

6. The system of claim 5 wherein the configuration parameters include a central cache name parameter for specifying the central cache to be used and a use central cache parameter specifying if processes are to access code segments in central cache.

7. The system of claim 5 wherein the second interface operatively couples to the plurality of local caches and wherein the configuration parameters include a max local cache entries parameter for defining a maximum number of entries to be cached simultaneously in a local cache and a max local cache words parameter defining a maximum number of words to be used by a local cache.

8. The system of claim 2 wherein the number of interfaces includes a third interface operatively coupled to receive internal commands generated by a process during the processing of database query statements, the third interface being operatively coupled to the central cache system of invoking the control mechanism to carry out those operations required for executing the internal commands.

9. The system of claim 8 wherein the internal commands include an insert command, a delete command, a find command, a clear command and a print info command.

10. The system of claim 2 wherein the number of interfaces includes a fourth interface operatively coupled to receive command calls from processes running on the host system, the fourth interface being operatively coupled to the central cache system and operative in response to the command calls invoke the control mechanism for carrying out operations specified by the command calls.

11. The system of claim 10 wherein the command calls include a generate central report command call used for generating statistics for the central cache system and a reset central statistics command call used for resetting statistics for the central cache system.

12. The system of claim 1 wherein the central cache system includes a persistent memory segment for storing the shared control structures, the memory including a number of memory sections, each section being associated with a different configured central cache and each section including first descriptor values for identifying one of the shared control structures corresponding to a directory control store structure and a set of descriptors for identifying one of the number of sets of central cache structures defining those components required for operating the configured central cache associated therewith.

13. The system of claim 12 wherein the directory control structure includes a number of indexed entry locations, each location for storing a value defining a name identifier for a set of central cache control structures for operating a configured central cache and for defining an index value identifying the set of descriptors associated with the configured central cache.

14. The system of claim 13 wherein the set of control structures includes:

a cache data structure for defining the characteristics of the central cache, a bucket structure, a cnode structure and a code descriptor structure for accessing code segments in the central cache associated therewith.

15. The system of claim 1 wherein the complex database query statements are SQL statements used for accessing relational database facilities of the host system and wherein the generated code correspond to the statements for executing the query statements.

16. The system of claim 1 wherein the shared control structures are stored in a predefined area of memory which is inaccessible by the processes running on the host system and which endures for a substantially long period of time.

17. The system of claim 12 wherein the memory segment further includes a separate section for storing a number of pairs of descriptors sequentially allocated to each configured central cache during the configuration thereof, each allocated pair of descriptors providing information to straggling processes pertaining to the central caches associated therewith for enabling the processes to detect that a central cache has been deleted from the central cache system.

18. A method of organizing a multicache system within a primary memory of a host system for improving the performance of a number of processes running on the host system in processing complex database query statements, the multicache system including a plurality of local caches in main memory corresponding to the number of processes being run on the host system, each local cache being associated with a different one of the processes being run, the method comprising the steps of:

(a) configuring at least one configured central cache system in a predefined area of primary memory shared by the processes, the central cache system being operatively coupled to each of the plurality of local cache systems associated therewith;

(b) including in the central cache system, a number of shared control structures used for controlling access thereto;

(c) including in the central cache system, a number of sets of central cache control structures corresponding to the number of central caches configured for storing code segments; and

,

(d) including in the central cache system, a control mechanism operatively coupled to the number of shared control structures and to the number of sets of central cache structures, the control mechanism being operative to control the configuration and operation of each of the configured central caches so that generated code segments produced by any one of the processes for executing a complex database query statement are stored automatically in the local cache associated with the process and in the central cache associated therewith for use by other processes configured to be associated with the same central cache in processing logically equivalent complex database query statements.

19. The method of claim 18 further including the step of:

(e) including in the host system, a number of interfaces that operatively couple to the central cache system for configuring and operating the number of central caches.

20. The method of claim 19 wherein the method further includes the steps of:

including a cache utility program component operatively coupled to receive utility directives entered into the system by a user and operatively coupled to the control central cache system;

operatively coupling the cache utility program component to a first one of the number of interfaces; and

,

issuing utility functions by the cache utility program component in response to the user initiated directives for invoking the control mechanism to perform a sequence of operations for carrying out the specified functions.

21. The method of claim 20 wherein the utility directives include commands for specifying clearing, creating and deleting central caches within the central cache system.

22. The method of claim 19 wherein the host system includes a configuration file for storing user entered configuration parameters defining the characteristics of the central caches to be configured and the method further includes the steps of:

operatively coupling a second one of the interfaces to the configuration file and to the central cache system;

invoking the control mechanism by the second interface when a process is put in execution for storing parameter information in specific ones of the shared control structures and into a specified one of the number of sets of central cache control structures derived from the configuration file.

23. The method of claim 22 wherein the configuration parameters include a central cache name parameter for specifying the central cache to be used and a use central cache parameter specifying if processes are to access code segments in central cache.

24. The method of claim 22 wherein the method further includes the step of operatively coupling the second interface to the plurality of local caches and wherein the configuration parameters include a max local cache entries parameter for defining a maximum number of entries to be cached simultaneously in a local cache and a max local cache words parameter defining a maximum number of words to be used by a local cache.

25. The method of claim 19 wherein the method further includes the steps of:

including a third one of the interfaces which operatively couples to receive internal commands generated by a process during the processing of database query statements; and

operatively coupling the third interface to the central cache system for invoking the control mechanism to carry out those operations required for executing the internal commands.

26. The method of claim 25 wherein the internal commands include an insert command, a delete command, a find command, a clear command and a print info command.

27. The method of claim 19 wherein the method further includes the steps of:

including a fourth one of the interfaces that operatively couples to receive command calls from processes running on the host system; and

operatively coupling the fourth interface to the central cache system for invoking the control mechanism in response to the command calls for carrying out operations specified by the command calls.

28. The method of claim 27 wherein the command calls include a generate central report command call used for generating statistics for the central cache system and a reset central statistics command call used for resetting statistics for the central cache system.

29. The method of claim 18 wherein the method further includes:

utilizing a persistent memory segment in the central cache system for storing the shared control structures, the memory segment including a number of memory sections, each section being associated with a different configured central cache and each including first descriptor values for identifying one of the shared structures corresponding to a directory control store structure and a set of descriptors for identifying one of the number of sets of central cache structures defining those components required for operating the configured central cache associated therewith.

30. The method of claim 29 wherein the directory control structure includes a number of indexed entry locations, each location for storing a value defining a name identifier for a set of central cache control structures for operating a configured central cache and for defining an index value identifying the set of descriptors associated with the configured central cache.

31. The method of claim 30 wherein the set of control structures includes:

a cache data structure for defining the characteristics of the central cache, a bucket structure, a cnode structure and a code descriptor structure for accessing code segments in the central cache associated therewith.

32. The method of claim 29 wherein the method further includes the steps of:

utilizing a separate section for storing a number of pairs of descriptors sequentially allocated to each configured central cache during the configuration thereof, each allocated pair of descriptors providing information to straggling processes pertaining to the central caches associated therewith for enabling the processes to detect that a central cache has been deleted from the central cache system.

33. A computer main memory containing a number of segments for storing data and control structures for improving the performance of a number of processes on a host system in processing complex database query statements, the data and control structures comprising:

(a) a plurality of local caches in a corresponding number of main memory segments corresponding to the number of processes being run on the host system, each local cache being associated with a different one of the processes being run; and

(b) at least one configured central cache system located in a predefined area of another one of the segments of main memory shared by the processes, the central cache system being operatively coupled to each of the plurality of local cache systems associated therewith, said central cache system including;

a number of shared control structures used for controlling access to the central cache system; and

,

a number of sets of central cache control structures corresponding to the number of central caches configured for storing code segments;

the sets of central cache structures enabling configuration and operation of each of the central caches so that generated code segments produced by any one of the processes for executing a complex database query statement are stored automatically in the local cache associated with the process and in the central cache configured to be associated therewith for use by other processes associated with the same central cache in processing logically equivalent complex database query statements.

34. A multicache system configurable within a primary memory of a host system for improving the performance of a number of processes running on the host system in processing complex database query statements, the multicache system comprising:

(a) a plurality of local caches in main memory corresponding to the number of processes being run on the host system, each local cache being associated with a different one of the processes being run; and

(b) at least one configured central cache system located in a shared predefined area of primary memory, the central cache system being operatively coupled to each of the plurality of local cache systems associated therewith, said central cache system including;

a number of shared control structures used for controlling access to the central cache system;

a number of sets of central cache control structures corresponding to the number of central caches configured for storing code segments; and

,

a control mechanism operatively coupled to the number of shared control structures and to the number of sets of central cache structures, the control mechanism being operative to control the configuration and operation of each of the central caches so that generated code segments produced by any one of the processes for executing a complex database query statement are automatically stored in the local cache associated with the process and in the configured central cache associated therewith for use by other processes associated with the same central cache in processing logically equivalent complex database query statements;

(c) a number of interfaces which operatively couple to the central cache system for configuring and operating the number of central caches; and

,

(d) a cache utility program component operatively coupled to receive utility directives entered into the system by a user and operatively coupled to the central cache system, the cache utility program component being operatively coupled to a first one of the number of interfaces, the cache utility program component in response to the user initiated directives being operative to issue utility functions for invoking the control mechanism to perform a sequence of operations for carrying out the specified functions.

35. A multicache system configurable within a primary memory of a host system for improving the performance of a number of processes running on the host system in processing complex database query statements, the multicache system comprising:

(a) a plurality of local caches in main memory corresponding to the number of processes being run on the host system, each local cache being associated with a different one of the processes being run; and

(b) at least one configured central cache system located in a shared predefined area of primary memory, the central cache system being operatively coupled to each of the plurality of local cache systems associated therewith, said central cache system including;

a number of shared control structures used for controlling access to the central cache system;

a number of sets of central cache control structures corresponding to the number of central caches configured for storing code segments; and

,

a control mechanism operatively coupled to the number of shared control structures and to the number of sets of central cache structures, the control mechanism being operative to control the configuration and operation of each of the central caches so that generated code segments produced by any one of the processes for executing a complex database query statement are automatically stored in the local cache associated with the process and in the configured central cache associated therewith for use by other processes associated with the same central cache in processing logically equivalent complex database query statements;

(c) a number of interfaces which operatively couple to the cell cache system for configuring and operating the number of central caches; and

,

(d) a configuration file for storing user entered configuration parameters defining the characteristics of the central caches to be configured, the number of interfaces includes a second interface operatively coupled to the configuration file and to the central cache system, the second interface being operative when a process is put in execution to invoke the control mechanism for storing parameter information in specific ones of the shared control structures and into a specified one of the number of sets of central cache control structures derived from the configuration file.

36. A multicache system configurable within a primary memory of a host system for improving the performance of a number of processes running on the host system in processing complex database query statements, the multicache system comprising:

(a) a plurality of local caches in main memory corresponding to the number of processes being run on the host system, each local cache being associated with a different one of the processes being run; and

(b) at least one configured central cache system located in a shared predefined area of primary memory, the central cache system being operatively coupled to each of the plurality of local cache systems associated therewith, sad central cache system including;

a number of shared control structures used for controlling access to the central cache system;

a number of sets of central cache control structures corresponding to the number of central caches configured for storing code segments; and

,

a control mechanism operatively coupled to the number of shared control structures and to the number of sets of central cache structures, the control mechanism being operative to control the configuration and operation of each of the central caches so that generated code segments produced by any one of the processes for executing a complex database query statement are automatically stored in the local cache associated with the process and in the configured central cache associated therewith for use by other processes associated with the same central cache in processing logically equivalent complex database query statements; and

,

(c) a number of interfaces which operatively couple to the central cache system for configuring and operating the number of central caches;

the number of interfaces including a fourth interface operatively coupled to receive command calls from processes running on the host system, the fourth interface being operatively coupled to the central cache system and operative in response to the command calls to invoke the control mechanism for carrying out operations specified by the command calls, the command calls including a generate central report command call used for generating statistics for the central cache system and a reset central statistics command call used for resetting statistics for the central cache system.

37. A multicache system configurable within a primary memory of a host system for improving the performance of a number of processes running on the host system in processing complex database query statements, the multicache system comprising:

(a) a plurality of local caches in main memory corresponding to the number of processes being run on the host system, each local cache being associated with a different one of the processes being run; and

(b) at least one configured central cache system located in a shared predefined area of primary memory, the central cache system being operatively coupled to each of the plurality of local cache systems associated therewith, said central cache system including;

a number of shared control structures used for controlling access to the central cache system;

a number of sets of central cache control structures corresponding to the number of central caches configured for storing code segments; and

,

a control mechanism operatively coupled to the number of stared control structures and to the number of sets of central cache structures, the control mechanism being operative to control the configuration and operation of each of the central caches so that generated code segments produced by any one of the processes for executing a complex database query statement are automatically stored in the local cache associated with the process and in the configured central cache associated therewith for use by other processes associated with the same central cache in processing logically equivalent complex database query statements; and

,

a persistent memory segment for storing the shared control structures, the memory including a number of memory sections, each section being associated with a different configured central cache and each section including first descriptor values for identifying one of the shared control structures corresponding to a directory control store structure and a set of descriptors for identifying one of the number of sets of central cache structures defining those components required for operating the configured central cache associated therewith, the persistent memory segment further including a separate section for storing a number of pairs of descriptors sequentially allocated to each configured central cache during the configuration thereof, each allocated pair of descriptors providing information to straggling processes pertaining to the central caches associated therewith for enabling the processes to detect that a central cache has been deleted from the central cache system.

38. A method of organizing a multicache system within a primary memory of a host system for improving the performance of a number of processes running on the host system in processing complex database query statements, the multicache system including a plurality of local caches in main memory corresponding to the number of processes being run on the host system, each local cache being associated with a different one of the processes being run, the method comprising the steps of:

(a) configuring at least one central cache system in a shared predefined area of primary memory, the central cache system being operatively coupled to each of the plurality of local cache systems associated therewith;

(b) including in the central cache system, a number of shared control structures used for controlling access thereto;

(c) including in the central cache system, a number of sets of central cache control structures corresponding to the number of central caches configured for storing code segments;

(d) including in the central cache system, a control mechanism operatively coupled to the number of shared control structures and to the number of sets of central cache structures, the control mechanism being operative to control the configuration and operation of each of the central caches so that generated code segments produced by any one of the processes for executing a complex database query statement are automatically stored in the local cache associated with the process and in the configured central cache associated therewith for use by other processes associated with the same central cache in processing logically equivalent complex database query statements;

(e) including in the host system, a number of interfaces that operatively couple to the central cache system for configuring and operating the number of central caches;

(f) including a cache utility program component operatively coupled to receive utility directives entered into the system by a user and operatively coupled to the control central cache system;

operatively coupling the cache utility program component to a first one of the number of interfaces; and

,

(g) issuing utility functions by the cache utility program component in response to the user initiated directives for invoking the control mechanism to perform a sequence of operations for carrying out the specified functions.

39. A method of organizing a multicache system within a primary memory of a host system for improving the performance of a number of processes running on the host system in processing complex database query statements, the multicache system including a plurality of local caches in main memory corresponding to the number of processes being run on the host system, each local cache being associated with a different one of the processes being run, the method comprising the steps of:

(a) configuring at least one configured central cache system in a shared predefined area of primary memory, the central cache system being operatively coupled to each of the plurality of local cache systems associated therewith;

(b) including in the central cache system, a number of shared control structures used for controlling access thereto;

(c) including in the central cache system, a number of sets of central cache control structures corresponding to the number of central caches configured for storing code segments;

(d) including in the central cache system, a control mechanism operatively coupled to the number of shared control structures and to the number of sets of central cache structures, the control mechanism being operative to control the configuration and operation of each of the central caches so that generated code segments produced by any one of the processes for executing a complex database query statement are automatically stored in the local cache associated with the process and in the configured central cache associated therewith for use by other processes associated with the same central cache in processing logically equivalent complex database query statements

(e) including in the host system a number of interfaces that operatively couple to the central cache system for configuring and operating the number of central caches, the host system including a configuration file for storing user entered configuration parameters defining the characteristics of the central caches to be configured and a second one of the interfaces being operatively coupled to the configuration file; and

,

(f) invoking the control mechanism by the second interface when a process is put in execution for storing parameter information in specific ones of the shared control structures and into a specified one of the number of sets of central cache control structures derived from the configuration file.

40. A method of organizing a multicache system within a primary memory of a host system for improving the performance of a number of processes running on the host system in processing complex database query statements, the multicache system including a plurality of local caches in main memory corresponding to the number of processes being run on the host system, each local cache being associated with a different one of the processes being run, the method comprising the steps of:

(a) configuring at least one configured central cache system in a shared predefined area of primary memory, the central cache system being operatively coupled to each of the plurality of local cache systems associated therewith;

(b) including in the central cache system, a number of shared control structures used for controlling access thereto;

(c) including in the central cache system, a number of sets of central cache control structures corresponding to the number of central caches configured for storing code segments;

(d) including in the central cache system, a control mechanism operatively coupled to the number of shared control structures and to the number of sets of central cache structures, the control mechanism being operative to control the configuration and operation of each of the central caches so that generated code segments produced by any one of the processes for executing a complex database query statement are automatically stored in the local cache associated with the process and in the configured central cache associated therewith for use by other processes associated with the same central cache in processing logically equivalent complex database query statements;

(e) utilizing a persistent memory segment in the central cache system for storing the shared control structures, the memory segment including a number of memory sections, each section being associated with a different configured central cache and each including first descriptor values for identifying one of the shared structures corresponding to a directory control store structure and a set of descriptors for identifying one of the number of sets of central cache structures defining those components required for operating the configured central cache associated therewith; and

,

(f) utilizing a separate section for storing a number of pairs of descriptors sequentially allocated to each configured central cache during the configuration thereof, each allocated pair of descriptors providing information to straggling processes pertaining to the central caches associated therewith for enabling the processes to detect that a central cache has been deleted from the central cache system.