PUSH REPLICATION FOR USE WITH A DISTRIBUTED DATA GRID

• US 20120197840A1
• Filed: 01/27/2012
• Published: 08/02/2012
• Est. Priority Date: 01/28/2011
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Oliver, Brian, Arliss, Noah
• Granted Patent: US 9,081,839 B2
• Time in Patent Office: Days
• US Class Current: 707/613
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation

Abstract

Push replication techniques are described for use in an in-memory data grid. When applications on a cluster perform insert, update or delete operations in the cache, a push replication provider asynchronously pushes updates from the source cluster to one or more remote destination clusters. The push replication provider includes a pluggable internal transport to send the updates to the destination cluster. This pluggable transport can be switched to employ a different communication service or protocol. A publishing transformer can chain multiple filters and apply filters on a stream of updates from source cluster to the destination cluster. A batch publisher can be used to receive batches multiple updates and replicate those batch to the destination cluster. XML based configuration can be provided to configure the push replication techniques on a cluster. A number of cluster topologies can be utilized, including active/passive, active/active, multi-site active/passive, multi-site active/active and centralized replication arrangement.

Claims

1. A system for providing push replication in distributed computing environments, said system comprising:

a source cluster of computer servers that manage a set of data and operations on said data in a distributed computing environment, said source cluster including a cache storing a set of data entries and an application residing on at least one server in said source cluster, said application executing one or more insert, update or delete operations on said data entries in the cache, wherein the source cluster maintains a maintains a connection to a remote destination cluster over a network; and

a push replication provider that asynchronously pushes updates on the data entries from the source cluster to the destination cluster, said updates being caused by the application executing the one or more insert, update or delete operations in the cache, wherein the push replication provider includes a pluggable internal transport to send the updates to the destination cluster, wherein said internal transport can be switched.

2. The system of claim 1, further comprising:

a publishing cache store residing in the source cluster, wherein the publishing cache store determines that an update has been performed on a data entry in the cache and provides the update to the push replication provider, wherein the push replication provider places the update on a topic; and

publishing service residing on the active cluster, wherein said publishing service subscribes to said topic, reads the update and transmits the update to the destination cluster.

3. The system of claim 1, wherein the push replication provider can be used to plug in any internal transport mechanism that provides store and forward semantics;

wherein the store and forward semantics can ensure that each update is kept in a queue such that in the event of a lost connection between the clusters, the update remains on the queue and is replicated once the connection is reestablished.

4. The system of claim 1, wherein the destination cluster further comprises:

a local cache publisher that reads updates received from the push replication provider, performs conflict resolution on said updates and writes said updates to a local cache on the destination cluster.

5. The system of claim 1, further comprising:

a batch publisher that receives a batch of a plurality of said updates and replicates said batch to the destination cluster.

6. The system of claim 1, further comprising:

a publishing transformer that applies at least one filter to a stream of said updates such that only the updates that pass through said filter are replicated from the source cluster to the destination cluster.

7. The system of claim 6, wherein the publishing transformer composes a chain of a plurality of filters and applies said chain of filters to the stream of updates from the source cluster to the destination cluster.

8. The system of claim 1, wherein the source cluster and the destination cluster form an active/active topology, wherein the destination cluster asynchronously pushes updates to the source cluster and the source cluster asynchronously pushes updates to the destination cluster.

9. The system of claim 1, further comprising:

a plurality of spoke clusters, wherein the active cluster is designated the hub cluster and wherein the hub cluster actively pushes the updates to the plurality of the spoke clusters and wherein each of the plurality of spoke clusters push updates only to the hub cluster.

10. The system of claim 1, further comprising:

a plurality of additional clusters wherein each of the plurality of additional clusters actively pushes updates to each other cluster.

11. A method for providing push replication in distributed computing environments, said method comprising:

storing a set of data entries in a cache located on a source cluster of computer servers that manage a set of data and operations on said data in a distributed computing environment, wherein said source cluster further includes an application residing on at least one server in said source cluster, said application executing one or more insert, update or delete operations on said data entries in the cache;

establishing a connection to a destination cluster of computer servers over a network;

detecting that the one or more insert, update or delete operations has been performed to cause an update on the data entries on the cache;

applying one or more filters to the update; and

asynchronously pushing update from the source cluster to the destination cluster if said update passes the one or more filters.

12. The method of claim 11, further comprising:

determining that an update has been performed on a data entry in the cache and providing the update to a push replication provider, wherein the push replication provider places the update on a topic; and

subscribing to the topic by a publishing service residing on the active cluster, wherein said publishing service reads the update and transmits the update to the destination cluster.

13. The method of claim 11, further comprising:

providing a pluggable internal transport mechanism to the source cluster, wherein the pluggable internal transport mechanism provides store and forward semantics;

wherein the store and forward semantics can ensure that each update is kept in a queue such that in the event of a lost connection between the clusters, the update remains on the queue and is replicated once the connection is reestablished.

14. The method of claim 11, wherein the destination cluster further comprises:

a local cache publisher that reads updates received from the source cluster, performs conflict resolution on said updates and writes said updates to a local cache on the destination cluster.

15. The method of claim 11, further comprising:

receiving a batch of a plurality of said updates by a batch publisher and replicating said batch to the destination cluster.

16. The method of claim 11, further comprising:

composing a chain of a plurality of said filters and applying said chain of filters to a stream of updates from the source cluster to the destination cluster.

17. The method of claim 11, wherein the source cluster and the destination cluster form an active/active topology, wherein the destination cluster asynchronously pushes updates to the source cluster and the source cluster asynchronously pushes updates to the destination cluster.

18. The method of claim 11, further comprising:

providing a plurality of spoke clusters, wherein the active cluster is designated the hub cluster and wherein the hub cluster actively pushes the updates to the plurality of the spoke clusters and wherein each of the plurality of spoke clusters push updates only to the hub cluster.

19. The method of claim 11, further comprising:

providing a plurality of additional clusters wherein each of the plurality of additional clusters actively pushes updates to each other cluster.

20. A non-transitory computer readable storage medium storing a set of instructions executable by one or more processors to perform a sequence of steps comprising:

storing a set of data entries in a cache located on a source cluster of computer servers that manage a set of data and operations on said data in a distributed computing environment, wherein said source cluster further includes an application residing on at least one server in said source cluster, said application executing one or more insert, update or delete operations on said data entries in the cache;

establishing a connection to a destination cluster of computer servers over a network;

detecting that the one or more insert, update or delete operations has been performed to cause an update on the data entries on the cache;

applying one or more filters to the update; and

asynchronously pushing update from the source cluster to the destination cluster if said update passes the one or more filters.