Write behind cache with M-to-N referential integrity

• US 8,527,557 B2
• Filed: 03/29/2012
• Issued: 09/03/2013
• Est. Priority Date: 09/22/2010
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Joseph, Jojo
• Primary Examiner(s): Breene, John E
• Assistant Examiner(s): Ly, Anh
• Application Number:
  US13/434,695
• Publication Number:
  US 20120191763A1
• Time in Patent Office: 523 Days
• Field of Search: 707/812, 707/E17.005, 707/E17.044, 711/143, 711/E12.001, 711/E12.026, 709/203, 709/212, 709/214, 709/216
• US Class Current: 707/812
• CPC Class Codes: G06F 16/24552 Database cache management

Abstract

A batch of data records generated by at least one application for persistent storage is cached by a cache management device. At least one of the batch of data records includes at least one foreign key reference that each references one non-existent data record. It is determined that one of a group of batch database storage events has occurred. In response to one of the group of batch database storage events, it is iteratively determined which of the at least one of the batch of data records do not include the at least one foreign key reference that each references the one non-existent data record. Each of the cached batch of data records that do not include the at least one foreign key reference that each references the one non-existent data record is persisted to a database by a batch storage operation.

Claims

1. A method, comprising:

caching, via a processor, a batch of data records generated by at least one application for persistent storage, where at least one of the batch of data records comprises at least one foreign key reference that each references one non-existent data record;

where caching, via the processor, the batch of data records generated by the at least one application for persistent storage comprises;

maintaining a bidirectional directed graph with each data record represented as a node within the bidirectional directed graph, where each vertex in the bidirectional directed graph other than a root node represents one of a plurality of parent records and one of a plurality of child records to be persisted to the database, and each edge in the bidirectional directed graph represents a foreign key directional relationship from one child record to one associated parent record;

determining that one of a plurality of batch database storage events has occurred;

iteratively determining, in response to the one of the plurality of batch database storage events, which of the at least one of the batch of data records do not comprise the at least one foreign key reference that each references the one non-existent data record; and

persisting to a database, via a batch storage operation, each of the cached batch of data records that do not comprise the at least one foreign key reference that each references the one non-existent data record.

2. The method of claim 1, where caching, via the processor, the batch of data records generated by the at least one application for persistent storage comprises:

creating a dummy record in the cached batch of data records for each of the at least one foreign key reference that each references the one non-existent data record; and

further comprising removing each dummy record from the cache in response to caching an existing data record with a primary key that matches each dummy record.

3. The method of claim 1, where iteratively determining, in response to the one of the plurality of batch database storage events, which of the at least one of the batch of data records do not comprise the at least one foreign key reference that each references the one non-existent data record comprises:

traversing the bidirectional directed graph;

identifying each data record that needs to be committed to the database that does not comprise the at least one foreign key reference that each references the one non-existent data record;

removing each identified data record from the bidirectional directed graph;

instantiating a batch update data structure in response to identification of a first instance of each record type associated with each removed data record; and

adding each removed data record to the batch update data structure associated with the record type associated with each removed data record.

4. The method of claim 3, where persisting to the database, via the batch storage operation, each of the cached batch of data records that do not comprise the at least one foreign key reference that each references the one non-existent data record comprises:

iteratively attempting to persist each data record of each batch update data structure to the database; and

further comprising;

removing, for each persisted data record, the data record from the batch update data structure associated with each record type associated with each persisted record; and

for each record comprising a failed persistence to the database;

adding the record to the bidirectional directed graph;

incrementing a retry counter associated with each record added to the bidirectional directed graph; and

iteratively attempting to persist each record comprising a failed persistence to the database in response to at least one subsequent batch database storage event.

5. The method of claim 4, further comprising:

determining whether the retry counter associated with any record has reached a threshold number of repeated failed persistence attempts to the database; and

notifying, for each record determined to comprise the retry counter that has reached the threshold number of repeated failed persistence attempts, an application that generated the record that has reached the threshold number of repeated failed persistence attempts.

6. The method of claim 1, where the batch database storage event comprises one of a batch record count being reached event and a batch interval time expiration event.

7. The method of claim 1, where:

each data record of the batch of data records generated by the at least one application comprises an associated structured query language (SQL) statement generated by the at least one application that generated each data record; and

persisting to the database, via the batch storage operation, each of the cached batch of data records that do not comprise the at least one foreign key reference that each references the one non-existent data record comprises executing each SQL statement to the database.