Write behind cache with M-to-N referential integrity

• US 8,533,240 B2
• Filed: 09/22/2010
• Issued: 09/10/2013
• Est. Priority Date: 09/22/2010
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Joseph, Jojo
• Primary Examiner(s): Breene, John E
• Assistant Examiner(s): Ly, Anh
• Application Number:
  US12/888,030
• Publication Number:
  US 20120072470A1
• Time in Patent Office: 1,084 Days
• Field of Search: 707/812, 707/E17.044, 717/143, 717/E12.001, 717/E12.026, 709/212, 709/214, 709/216, 709/203, 711/143, 711/E12.001, 711/E12.026
• US Class Current: 707/812
• CPC Class Codes: G06F 16/24552 Database cache management

Abstract

A batch of data records generated by at least one application for persistent storage is cached by a cache management device. At least one of the batch of data records includes at least one foreign key reference that each references one non-existent data record. It is determined that one of a group of batch database storage events has occurred. In response to one of the group of batch database storage events, it is iteratively determined which of the at least one of the batch of data records do not include the at least one foreign key reference that each references the one non-existent data record. Each of the cached batch of data records that do not include the at least one foreign key reference that each references the one non-existent data record is persisted to a database by a batch storage operation.

Claims

1. A system, comprising:

a cache memory; and

a processor programmed to;

cache a batch of data records generated by at least one application for persistent storage to the cache memory, where at least one of the batch of data records comprises at least one foreign key reference that each references one non-existent data record, where the processor is programmed to;

maintain a bidirectional directed graph with each data record represented as a node within the bidirectional directed graph, where each vertex in the bidirectional directed graph other than a root node represents one of a plurality of parent records and one of a plurality of child records to be persisted to the database, and each edge in the bidirectional directed graph represents a foreign key directional relationship from one child record to one associated parent record;

determine that one of a plurality of batch database storage events has occurred;

iteratively determine, in response to the one of the plurality of batch database storage events, which of the at least one of the batch of data records do not comprise the at least one foreign key reference that each references the one non-existent data record; and

persist to a database, via a batch storage operation, each of the cached batch of data records that do not comprise the at least one foreign key reference that each references the one non-existent data record.

2. The system of claim 1, where, in being programmed to cache the batch of data records generated by the at least one application for persistent storage to the cache memory, the processor is programmed to:

create a dummy record in the cached batch of data records for each of the at least one foreign key reference that each references the one non-existent data record; and

the processor is further programmed to remove each dummy record from the cache memory in response to caching an existing data record with a primary key that matches each dummy record.

3. The system of claim 1, where, in being programmed to iteratively determine, in response to the one of the plurality of batch database storage events, which of the at least one of the batch of data records do not comprise the at least one foreign key reference that each references the one non-existent data record, the processor is programmed to:

traverse the bidirectional directed graph;

identify each data record that needs to be committed to the database that does not comprise the at least one foreign key reference that each references the one non-existent data record;

remove each identified data record from the bidirectional directed graph;

instantiate a batch update data structure in response to identification of a first instance of each record type associated with each removed data record; and

add each removed data record to the batch update data structure associated with the record type associated with each removed data record.

4. The system of claim 3, where, in being programmed to persist to the database, via the batch storage operation, each of the cached batch of data records that do not comprise the at least one foreign key reference that each references the one non-existent data record, the processor is programmed to:

iteratively attempt to persist each data record of each batch update data structure to the database; and

the processor is further programmed to;

remove, for each persisted data record, the data record from the batch update data structure associated with each record type associated with each persisted record; and

for each record comprising a failed persistence to the database;

add the record to the bidirectional directed graph;

increment a retry counter associated with each record added to the bidirectional directed graph; and

iteratively attempt to persist each record comprising a failed persistence to the database in response to at least one subsequent batch database storage event.

5. The system of claim 4, where the processor is further programmed to:

determine whether the retry counter associated with any record has reached a threshold number of repeated failed persistence attempts to the database; and

notify, for each record determined to comprise the retry counter that has reached the threshold number of repeated failed persistence attempts, an application that generated the record that has reached the threshold number of repeated failed persistence attempts.

6. The system of claim 1, where the batch database storage event comprises one of a batch record count being reached event and a batch interval time expiration event.

7. The system of claim 1, where:

each data record of the batch of data records generated by the at least one application comprises an associated structured query language (SQL) statement generated by the at least one application that generated each data record; and

in being programmed to persist to the database, via the batch storage operation, each of the cached batch of data records that do not comprise the at least one foreign key reference that each references the one non-existent data record, the processor is programmed to execute each SQL statement to the database.

8. A computer program product comprising a computer readable storage memory including computer readable program code, where the computer readable program code when executed on a computer causes the computer to:

cache a batch of data records generated by at least one application for persistent storage, where at least one of the batch of data records comprises at least one foreign key reference that each references one non-existent data record, where the computer readable program code when executed on the computer causes the computer to;

maintain a bidirectional directed graph with each data record represented as a node within the bidirectional directed graph, where each vertex in the bidirectional directed graph other than a root node represents one of a plurality of parent records and one of a plurality of child records to be persisted to the database, and each edge in the bidirectional directed graph represents a foreign key directional relationship from one child record to one associated parent record;

determine that one of a plurality of batch database storage events has occurred;

iteratively determine, in response to the one of the plurality of batch database storage events, which of the at least one of the batch of data records do not comprise the at least one foreign key reference that each references the one non-existent data record; and

persist to a database, via a batch storage operation, each of the cached batch of data records that do not comprise the at least one foreign key reference that each references the one non-existent data record.

9. The computer program product of claim 8, where in causing the computer to cache the batch of data records generated by the at least one application for persistent storage, the computer readable program code when executed on the computer causes the computer to:

create a dummy record in the cached batch of data records for each of the at least one foreign key reference that each references the one non-existent data record; and

the computer readable program code when executed on the computer further causes the computer to remove each dummy record in response to caching an existing data record with a primary key that matches each dummy record.

10. The computer program product of claim 8, where in causing the computer to iteratively determine, in response to the one of the plurality of batch database storage events, which of the at least one of the batch of data records do not comprise the at least one foreign key reference that each references the one non-existent data record, the computer readable program code when executed on the computer causes the computer to:

traverse the bidirectional directed graph;

identify each data record that needs to be committed to the database that does not comprise the at least one foreign key reference that each references the one non-existent data record;

remove each identified data record from the bidirectional directed graph;

instantiate a batch update data structure in response to identification of a first instance of each record type associated with each removed data record; and

add each removed data record to the batch update data structure associated with the record type associated with each removed data record.

11. The computer program product of claim 10, where in causing the computer to persist to the database, via the batch storage operation, each of the cached batch of data records that do not comprise the at least one foreign key reference that each references the one non-existent data record, the computer readable program code when executed on the computer causes the computer to:

iteratively attempt to persist each data record of each batch update data structure to the database; and

the computer readable program code when executed on the computer further causes the computer to;

remove, for each persisted data record, the data record from the batch update data structure associated with each record type associated with each persisted record; and

for each record comprising a failed persistence to the database;

add the record to the bidirectional directed graph;

increment a retry counter associated with each record added to the bidirectional directed graph; and

iteratively attempt to persist each record comprising a failed persistence to the database in response to at least one subsequent batch database storage event.

12. The computer program product of claim 11, where the computer readable program code when executed on the computer further causes the computer to:

determine whether the retry counter associated with any record has reached a threshold number of repeated failed persistence attempts to the database; and

notify, for each record determined to comprise the retry counter that has reached the threshold number of repeated failed persistence attempts, an application that generated the record that has reached the threshold number of repeated failed persistence attempts.

13. The computer program product of claim 8, where the batch database storage event comprises one of a batch record count being reached event and a batch interval time expiration event.

14. The computer program product of claim 8, where:

each data record of the batch of data records generated by the at least one application comprises an associated structured query language (SQL) statement generated by the at least one application that generated each data record; and

in causing the computer to persist to the database, via the batch storage operation, each of the cached batch of data records that do not comprise the at least one foreign key reference that each references the one non-existent data record, the computer readable program code when executed on the computer causes the computer to execute each SQL statement to the database.