System and method for supporting transient partition consistency in a distributed data grid

• US 8,954,391 B2
• Filed: 11/07/2012
• Issued: 02/10/2015
• Est. Priority Date: 10/15/2012
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Lee, Robert H., Gleyzer, Gene
• Primary Examiner(s): To, Baoquoc N
• Application Number:
  US13/671,408
• Publication Number:
  US 20140108358A1
• Time in Patent Office: 825 Days
• Field of Search: 707/642
• US Class Current: 707/642
• CPC Class Codes: G06F 11/16 Error detection or correction of the data by redundancy in hardware
• CPC Class Codes: G06F 11/2041 with more than one idle spare processing component
• CPC Class Codes: G06F 11/2048 where the redundant components share neither address space nor persistent storage
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: H04L 41/0668 by dynamic selection of recovery network elements, e.g. replacement by the most appropriate element after failure
• CPC Class Codes: H04L 43/0811 by checking connectivity
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Abstract

A system and method can support transient partition consistency in a distributed data grid. A cluster node in the distributed data grid can maintain a storage data structure and an index data structure. The storage data structure can store data in one or more partitions maintained on the cluster node, and the index data structure contains a plurality of indexes, wherein each index supports indexing at least one data grid operation on the one or more partitions. Furthermore, the distributed data grid ensures consistency between the storage data structure and the index data structure for the data stored in the one or more partitions maintained on the cluster node.

Claims

1. A system for supporting transient partition consistency in a distributed data grid, comprising:

a storage data structure on a cluster node in the distributed data grid, wherein the storage data structure stores data in one or more partitions on the cluster node; and

an index data structure for the data stored in the one or more partitions, wherein the index data structure contains a plurality of indexes, wherein each index supports at least one data grid operation on the one or more partitions;

wherein the distributed data grid operates to

determine whether an index in the index data structure is consistent with the one or more partitions by checking a partition version number of each said partition; and

update the index in the index data structure to maintain consistency between the index data structure and the storage data structure to support the at least one data grid operation on the one or more partitions.

2. The system according to claim 1, wherein the storage data structure is a backing map that stores data for a cache running on the distributed data grid.

3. The system according to claim 1, wherein the at least one data grid operation uses at least one of a filter and an aggregator.

4. The system according to claim 1, wherein each index is associated with an extractor that operates to perform the at least one data grid operation.

5. The system according to claim 1, wherein each index is associated with one or more partition version numbers, wherein each partition version number provides transient state information of a partition in the storage data structure.

6. The system according to claim 1, wherein the cluster node operates to receive a first request from a client for querying the data in the one or more partitions in the storage data structure based on at least one index.

7. The system according to claim 1, wherein the cluster node operates to query the data in the one or more partitions in the storage data structure before updating the index data structure.

8. The system according to claim 1, wherein the cluster node operates to process a second request from a client for querying the data in the one or more partitions before the cluster node finishes updating the index in the index data structure, if the second request does not use the index.

9. The system according to claim 1, wherein the cluster node operates to block a second request from a client for querying the data in the one or more partitions before the cluster node finishes updating the index in the index data structure, if the second request uses the index.

10. A method for supporting transient partition consistency in a distributed data grid, comprising:

providing a storage data structure on a cluster node in the distributed data grid, wherein the storage data structure stores data in one or more partitions on the cluster node;

providing an index data structure for the data stored in the one or more partitions, wherein the index data structure contains a plurality of indexes, wherein each index supports at least one data grid operation on the one or more partitions;

determining whether an index in the index data structure is consistent with for the one or more partitions by checking a partition version number of each said partition; and

updating the index in the index data structure to maintain consistency between the index data structure and the storage data structure to support the at least one data grid operation on the one or more partitions.

11. The method according to claim 10, wherein the storage data structure is a backing map that stores data for a cache running on the distributed data grid.

12. The method according to claim 10, wherein the at least one data grid operation uses at least one of a filter and an aggregator.

13. The method according to claim 10, wherein each index is associated with an extractor that operates to perform the at least one data grid operation.

14. The method according to claim 10, wherein each index is associated with one or more partition version numbers, wherein each partition version number provides transient state information of a partition in the storage data structure.

15. The method according to claim 10, wherein the cluster node receives a first request from a client for querying the data in the one or more partitions in the storage data structure based on at least one index.

16. The method according to claim 10, wherein the cluster node operates to query the data in the one or more partitions in the storage data structure before updating the index data structure.

17. The method according to claim 10, wherein after the cluster node receives a second request from a client for querying the data in the one or more partitions and before the cluster node finishes updating the index in the index data structure, the cluster node operates to process the second request, if the second request does not use the at least one index, or block the second request, if the second request uses the index.

18. A non-transitory machine readable storage medium having instructions stored thereon that when executed cause a system to perform the steps comprising:

providing a storage data structure on a cluster node in the distributed data grid, wherein the storage data structure stores data in one or more partitions on the cluster node;

providing an index data structure for the data stored in the one or more partitions, wherein the index data structure contains a plurality of indexes, wherein each index supports at least one data grid operation on the one or more partitions;

determining whether an index in the index data structure is consistent with the one or more partitions by checking a partition version number of each said partition; and

updating the index in the index data structure to maintain consistency between the index data structure and the storage data structure to support the at least one data grid operation on the one or more partitions.