System and method for efficiently writing data from an in-memory database to a disk database

• US 20040267776A1
• Filed: 06/30/2003
• Published: 12/30/2004
• Est. Priority Date: 06/30/2003
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Brown, Kevin, Spicer, Michael John Elvery
• Granted Patent: US 7,113,953 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

A system and associated method write data from an in-memory database to a disk database in an efficient manner and with a relatively short lag time. The integration of data from in-memory to disk is achieved by limiting the operations of the in-memory database to insert only. The system shortens lag time by reducing the number of transactions required to transfer data from in-memory database to disk memory. The system compiles into an RDBMS, knowledge about the structure of the in-memory database, and then uses the end of the transaction callbacks from the RDBMS to keep the in-memory database and the disk memory in synch. The RDBMS includes a daemon that runs periodically to find records in the in-memory database that have not yet been written to the RDBMS, and then writes the found records to the RDBMS as part of a single transaction. If the transaction completes successfully, the in-memory database is updated to reflect which records have been “flushed” to the RDBMS. If the transaction fails, no action is taken. The present system synchronizes the in-memory database with the RDBMS.

Claims

1. A method of efficiently writing records from an in-memory database to a disk database, comprising:

linking the records by a linked list;

creating a header data structure of the linked records;

linking a new record in the in-memory database to the header data structure; and

transferring the records in the linked list and the new record from the in-memory database to the disk database using the header data structure.

2. The method of claim 1, wherein each record contains information about a single transaction.

3. The method of claim 1, wherein the header data structure comprises an entity name that identifies the record.

4. The method of claim 3, wherein the header data structure further comprises a last commit pointer.

5. The method of claim 4, wherein the header data structure further comprises a last flush pointer.

6. The method of claim 5, wherein the header data structure further comprises a head pointer.

7. The method of claim 6, wherein the header data structure further comprises a tail pointer.

8. The method of claim 7, further comprising locating a new header data structure for the new record

9. The method of claim 8, further comprising determining if the head pointer in the header data structure of the new record points to another record.

10. The method of claim 9, wherein if the head pointer in the header data structure of the new record points to another record getting the other record.

11. The method of claim 10, further comprising setting the tail pointer of the other record equal to a link value of the new record.

12. The method of claim 7, further comprising determining if the last commit pointer of the header data structure points at a record.

13. The method of claim 12, further comprising setting the last commit pointer equal to the head pointer if the last commit pointer does not point to a record.

14. The method of claim 12, further comprising getting the record to which the last commit pointer points.

15. The method of claim 14, further comprising determining whether there is a last record after the record to which the last commit pointer points.

16. The method of claim 15, further comprising writing to the disk memory all records in the link list that occur after the record to which the last commit pointer points.

17. The method of claim 16, further comprising setting a last flush pointer of the header data structure equal to the last count if the writing of the records to the disk memory ended successfully.

18. The method of claim 17, further comprising setting the last flush pointer equal to the last commit pointer if the writing of the records to the disk memory did not end successfully.

19. A computer program product having instruction codes for efficiently writing records from an in-memory database to a disk database, comprising:

a first set of instruction codes for linking the records by a linked list;

a second set of instruction codes for creating a header data structure of the linked records;

a third set of instruction codes for linking a new record in the in-memory database to the header data structure; and

a fourth set of instruction codes for transferring the records in the linked list and the new record from the in-memory database to the disk database using the header data structure.

20. The computer program product of claim 19, wherein each record contains information about a single transaction.

21. The computer program product of claim 19, wherein the header data structure comprises an entity name that identifies the record.

22. The computer program product of claim 21, wherein the header data structure further comprises a last commit pointer.

23. The computer program product of claim 22, wherein the header data structure further comprises a last flush pointer.

24. The computer program product of claim 23, wherein the header data structure further comprises a head pointer.

25. The computer program product of claim 24, wherein the header data structure further comprises a tail pointer.

26. A system for efficiently writing records from an in-memory database to a disk database, comprising:

a first set of instruction codes for linking the records by a linked list;

a second set of instruction codes for creating a header data structure of the linked records;

a third set of instruction codes for linking a new record in the in-memory database to the header data structure; and

a fourth set of instruction codes for transferring the records in the linked list and the new record from the in-memory database to the disk database using the header data structure.

27. The system of claim 26, wherein each record contains information about a single transaction.

28. The system of claim 26, wherein the header data structure comprises an entity name that identifies the record.

29. The system of claim 28, wherein the header data structure further comprises a last commit pointer.

30. The system of claim 29, wherein the header data structure further comprises a last flush pointer and a head pointer.