Structure and method for sharing large databases

• US 7,171,425 B2
• Filed: 03/27/2003
• Issued: 01/30/2007
• Est. Priority Date: 12/20/2002
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: Cognos Incorporated (International Business Machines Corporation)
• Inventors: Cazemier, Hendrik
• Primary Examiner(s): Vo; Tim
• Assistant Examiner(s): Ly; Cheyne D.
• Application Number:
  US10/401,056
• Publication Number:
  US 20040122827A1
• Time in Patent Office: 1,405 Days
• Field of Search: 707/1, 707/4, 707/101, 707/103
• US Class Current: 1/1
• CPC Class Codes: G06F 16/252 between a Database Management System and a front-end application
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/283 Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99944 Object-oriented database structure

Abstract

The invention provides an application programming mechanism and technique that makes referencing data look as if the data is dynamically allocated by the process. The invention also provides a mechanism whereby persisted data is shared across all processes on the same computer that access it. Small, process specific structures are created on demand as the program accesses the persisted data and makes use of the underlying memory mapping and linking technologies inherent in modern operation systems. The technique is especially useful in large data repositories where it simplifies the sharing of data between instances of an application, and also permits sharing of data between different applications. It allows for the data to be highly referenced internally while not requiring the extra memory used by pointer swizzling at runtime.

Claims

1. A memory management method for use in a business intelligence system having a general purpose operating system and one or more applications accessing data, the data comprising original data stored in a data repository, the method comprising the steps of:

a) translating the original data from the data repository to a newly created compacted database having a format required by the one or more applications for which the compacted database is being prepared;

b) loading the compacted database into a shared memory;

c) computing a base address of the compacted database in the shared memory;

d) preparing a translation object for each of the one or more applications;

e) creating and storing within the translation object, a pointer object to a data location in the compacted database, comprising an offset to the data location so that the base address and the offset to the data location taken together give access to values equivalent to those in the original data;

f) retrieving data from the compacted database loaded in the shared memory; and

g) accessing data within the compacted database using the translation object and the pointer object.

2. The method of claim 1 wherein the translating step further includes the step of omitting from the compacted database any data in the original data not required by any of the one or more applications.

3. The method of claim 1 wherein the applications include one or more instantiations of an application.

4. The method of claim 1 wherein the applications include a combination of instantiations of an application and at least one other application.

5. The method of claim 1 wherein the translating step further includes compressing original data to further reduce the size of the compacted database.

6. The method of claim 1 wherein the original data are non-volatile.

7. The method of claim 1 wherein the original data is selected from a group consisting of business intelligence data, historical data, and a combination thereof.

8. The method of claim 1, wherein:

the translating step further includes the steps of;

preparing a class definition file including data required by an application instance;

computing a position of each datum as an offset with respect to an origin of an object described by the class definition file, and

storing references to the data items as offsets; and

wherein the retrieving step further includes the steps of;

accessing, for each application and user, the compacted database loaded in a shared memory and,

generating on demand at run-time from the compacted database loaded in the shared memory, for each reference to a datum, an actual address for the datum based on a base address and a derived reference that is an offset of data location within the object described by the class definition file, the base address and the offset taken together giving access to equivalent values in the original data.

9. The method of claim 8 wherein the class definition file further comprises meta-data describing one or more formats required by the one or more applications.

10. The method of claim 1 wherein the retrieving step uses hardware to generate the actual address for a datum.

11. The method of claim 1 wherein the retrieving step uses software to generate the actual address for a datum.

12. The method of claim 1 further comprises the step of providing high level application programming interfaces, thereby making the integration into an application straightforward.

13. The method of claim 12 wherein the high level application programming interfaces include support for variable length strings, linked lists and associative arrays.

16. Computer executable software code stored on a computer readable medium, comprising code modules to perform the method of claim 1.

14. A system for the creation of a database accessible from one or more applications, the applications being capable of being used by one or more users, the system comprising one or more computers, and an associated data repository containing original data, each computer including software for:

a) translating original data from a data repository to a newly created compacted database having a format required by the one or more applications for which the compacted database is being prepared;

b) loading the compacted database into a shared memory;

c) computing a base address of the compacted database in shared memory;

d) preparing a translation object for each of the one or more applications;

e) creating and storing within the translation object, a pointer object to a data location in the compacted database, comprising an offset to the data location so that the base address and the offset to the data location taken together give access to values equivalent to those in the original data;

f) retrieving data from the compacted database loaded in the shared memory; and

g) accessing data within the compacted database using the translation object and the pointer object.

15. A memory management system for use in a business intelligence system having a general purpose operating system and one or more applications accessing data, the data comprising original data stored in a data repository, the system comprising:

a) means for translating original data from a data repository to a newly created compacted database having a format required by the one or more applications for which the compacted database is being prepared;

b) means for loading the compacted database into a shared memory;

c) means for computing a base address of the compacted database in the shared memory;

d) means for preparing a translation object for each of the one or more applications;

e) means for creating and storing within the translation object, when data are required, a pointer object to a data location in the compacted database, comprising an offset to the data location so that the base address and the offset to the data location taken together give access to values equivalent to those in the original data;

f) means for retrieving data from the compacted database loaded in the shared memory; and

g) means for accessing data within the compacted database using the translation object and the pointer object.