Simultaneous use of database and durable store in work flow and process flow systems

• US 6,014,673 A
• Filed: 05/13/1997
• Issued: 01/11/2000
• Est. Priority Date: 12/05/1996
• Status: Expired due to Term

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Hewlett-Packard Company (HP Inc.)
• Inventors: Shan, Ming Chien, Sheard, Nicholas, Davis, James W., Du, Weimin
• Primary Examiner(s): Black, Thomas G.
• Assistant Examiner(s): Rones, Charles L.
• Application Number:
  US08/853,699
• Time in Patent Office: 973 Days
• Field of Search: 707/3, 707/2, 707/102, 707/103, 707/202, 707/203, 707/204, 707/10, 395/200.53, 395/200.31, 709/201, 714/15, 705/8
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

A method and apparatus for reliable high-speed access to a database system that stores system data in a non-volatile database, stores current data in an online database object cache, the current data reflecting at least a portion of the system data in the non-volatile database and the online database object cache providing the database system with the capability of querying and updating the current data in the online database object cache, logging each message in the database system as an entry in a durable log file, and periodically committing the current data in the online database object cache to the non-volatile database.

Claims

1. A method for reliable high-speed access to a database system, comprising:

storing system data in a non-volatile database;

storing current data in an object cache, the current data reflecting at least a portion of the system data in the non-volatile database, the object cache providing the database system with the capability of querying and updating the current data;

database and the object cache, wherein (1) each message contains historical and updating data of the current data and (2) the log file allows recovery of the database system in the event of a system failure by reconstructing the current data in the object cache, wherein no database commit is required in the log file or the database when the log file logs each message; and

periodically committing the current data in the object cache to the non-volatile database.

2. The method of claim 1, wherein the step of periodically committing the current data further comprises the steps of

searching for entries in the durable log file related to modifications in the current data in the object cache;

and

executing the same modifications found in the durable log file on the system data in the non-volatile database.

3. The method of claim 2, further comprising the step of marking current data in the object cache when the current data is updated.

4. The method of claim 1, wherein the step of periodically committing the current data in the object cache further comprises the step of:

inserting a commit entry in the durable log file when the current data in the object cache is committed to the non-volatile database;

periodically pruning the log file to remove entries which temporally precede the commit entry; and

periodically pruning the log file to remove entries storing the removed entries in a historical database.

5. The method of claim 4, wherein a failure of the database system can be recovered by

locating a most recent commit entry in the durable log file;

locating the entries in the durable log file related to modifications to the current data in the object cache which temporally follow the most recent commit entry; and

performing the same modifications to the system data in the non-volatile database for each such entry found.

6. The method of claim 1, wherein the step of logging each message in the database system further comprises constructing the durable log file as a circular list;

and the step of periodically committing the current data further comprises committing the current data in the object cache before the step of logging each message overwrites uncommitted entries in the durable log file.

7. The method of claim 1, further comprising:

periodically inserting a recovery point entry in the durable log file, wherein the recovery point entry includes identification numbers for all messages waiting in volatile message queues in the database system.

8. The method of claim 7, further comprising the step of recovering from a failure of the database system by

locating a most recent recovery point entry in the durable log file;

forming a resend queue containing the messages identified in the recovery point entry;

searching the durable log file for entries subsequent to the most recent recovery point for reply messages corresponding to each of the messages in the resend queue;

removing the messages in the resend queue that have corresponding reply messages; and

resending the messages remaining in the resend queue.

9. A database system configured to support a workflow process, comprising:

a non-volatile database configured to reside in non-volatile storage to support a data structure of the workflow process and to provide the workflow process with access to a first plurality of data elements;

a volatile database configured to support the data structure of the workflow process and to provide the workflow process with query and update access to a second plurality of data elements, the second plurality of data elements being related to a portion of the first plurality of data elements;

a non-volatile and non-database log file in a non-volatile memory separate from the databases to log, as an entry, each message generated in the database system, wherein (1) each message contains historical and updating data of the second plurality of data elements and (2) the log file allows recovery of the database system in the event of a system failure by reconstructing the second plurality of data elements in the volatile database, wherein no database commit is required in the log file or the database when the log file logs each message; and

an engine configured to support the workflow process and to log an entry in the non-volatile log file for each message generated by the workflow process.

10. The database system of claim 9, wherein the engine is further configured to periodically commit the second plurality of data elements to the first plurality of elements by searching for entries in the non-volatile log file related to update transactions on the second plurality of data elements and reexecuting the entries on the first plurality of data elements.

11. The database system of claim 10, wherein the engine is further configured to insert a commit entry in the non-volatile log file to indicate which entries in the non-volatile log file have been committed.

12. The database system of claim 11, wherein the engine is further configured to recover from failure by locating a most recent commit entry in the non-volatile log file, to identify the entries in the non-volatile log file related to update transactions on the second plurality of data elements that are subsequent to the most recent commit entry, and to reexecute the identified entries on the first plurality of data elements.

13. The database system of claim 10, wherein the engine is further configured to periodically insert a recovery point entry in the log file, wherein the recovery point entry includes identification numbers for all messages waiting in volatile message queues in the engine.

14. The database system of claim 13, wherein the engine is further configured to recover from failure by locating a most recent recovery point entry in the non-volatile log file, to form a resend queue containing messages with identification numbers from the most recent recovery point entry, to remove messages from the resend queue that have corresponding reply message entries in the non-volatile log file, and to resend the messages on the resend queue.