Push replication for use with a distributed data grid

• US 9,081,839 B2
• Filed: 01/27/2012
• Issued: 07/14/2015
• Est. Priority Date: 01/28/2011
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Oliver, Brian, Arliss, Noah
• Primary Examiner(s): BURKE, JEFF A
• Application Number:
  US13/360,487
• Publication Number:
  US 20120197840A1
• Time in Patent Office: 1,264 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation

Abstract

Push replication techniques are described for use in an in-memory data grid. When applications on a cluster perform insert, update or delete operations in the cache, a push replication provider asynchronously pushes updates from the source cluster to one or more remote destination clusters. The push replication provider includes a pluggable internal transport to send the updates to the destination cluster. This pluggable transport can be switched to employ a different communication service or protocol. A publishing transformer can chain multiple filters and apply filters on a stream of updates from source cluster to the destination cluster. A batch publisher can be used to receive batches multiple updates and replicate those batch to the destination cluster. XML based configuration can be provided to configure the push replication techniques on a cluster. A number of cluster topologies can be utilized, including active/passive, active/active, multi-site active/passive, multi-site active/active and centralized replication arrangement.

Claims

1. A system for providing push replication in distributed computing environments, said system comprising:

a source cluster of computer servers that manage a set of data and operations on said data in a distributed computing environment, said source cluster including a cache storing a set of data entries and an application residing on at least one server in said source cluster, said application executing one or more insert, update or delete operations on said data entries in the cache, wherein the source cluster maintains a connection to a destination cluster over a network;

a publishing cache store residing in the source cluster, wherein the publishing cache store determines that an update has been performed on a data entry in the cache and provides the update to the push replication provider, wherein the push replication provider places the update on a topic;

a publishing service residing on the source cluster, wherein said publishing service subscribes to said topic, reads the update and transmits the update to the destination cluster; and

a push replication provider that asynchronously pushes the update on the data entries from the source cluster to the destination cluster, said update being caused by the application executing the one or more insert, update or delete operations in the cache, wherein the push replication provider includes a pluggable internal transport to send the update to the destination cluster, wherein said internal transport can be switched.

2. The system of claim 1, wherein the push replication provider operates to plug in any internal transport mechanism that provides store and forward semantics;

wherein the store and forward semantics ensure that each update is kept in a queue such that in the event of a lost connection between the clusters, the update remains on the queue and is replicated once the connection is reestablished.

3. The system of claim 1, wherein the destination cluster further comprises:

a local cache publisher that reads updates received from the push replication provider, performs conflict resolution on said updates and writes said updates to a local cache on the destination cluster.

4. The system of claim 1, further comprising:

a batch publisher that receives a batch of a plurality of said updates and replicates said batch to the destination cluster.

5. The system of claim 1, further comprising:

a publishing transformer that applies at least one filter to a stream of said updates such that only the updates that pass through said filter are replicated from the source cluster to the destination cluster.

6. The system of claim 5, wherein the publishing transformer composes a chain of a plurality of filters and applies said chain of filters to the stream of updates from the source cluster to the destination cluster.

7. The system of claim 1, wherein the source cluster and the destination cluster form an active/active topology, wherein the destination cluster asynchronously pushes updates to the source cluster and the source cluster asynchronously pushes updates to the destination cluster.

8. The system of claim 1, further comprising:

a plurality of spoke clusters, wherein the source cluster is designated a hub cluster and wherein the hub cluster actively pushes the updates to the plurality of spoke clusters and wherein each of the plurality of spoke clusters push updates only to the hub cluster.

9. The system of claim 1, further comprising:

a plurality of additional clusters wherein each of the plurality of additional clusters actively pushes updates to each other cluster.

10. A method for providing push replication in distributed computing environments, said method comprising:

storing a set of data entries in a cache located on a source cluster of computer servers that manage a set of data and operations on said data in a distributed computing environment, wherein said source cluster further includes an application residing on at least one server in said source cluster, said application executing one or more insert, update or delete operations on said data entries in the cache;

establishing a connection to a destination cluster of computer servers over a network;

detecting that an update has been performed on a data entry in the cache, wherein the update is caused by the one or more insert, update or delete operations on said data entries;

providing the update to a push replication provider, wherein the push replication provider places the update on a topic;

subscribing to the topic by a publishing service residing on the source cluster, wherein said publishing service reads the update;

applying one or more filters to the update; and

asynchronously pushing, via a pluggable internal transport on the push replication provider, the update from the source cluster to the destination cluster when said update passes the one or more filters, wherein the pluggable internal transport can be switched.

11. The method of claim 10, wherein the push replication server operates to plug in any pluggable internal transport mechanism that provides store and forward semantics;

wherein the store and forward semantics ensure that each update is kept in a queue such that in the event of a lost connection between the clusters, the update remains on the queue and is replicated once the connection is reestablished.

12. The method of claim 10, wherein the destination cluster further comprises:

a local cache publisher that reads updates received from the source cluster, performs conflict resolution on said updates and writes said updates to a local cache on the destination cluster.

13. The method of claim 10, further comprising:

receiving a batch of a plurality of said updates by a batch publisher and replicating said batch to the destination cluster.

14. The method of claim 10, further comprising:

composing a chain of a plurality of said filters and applying said chain of filters to a stream of updates from the source cluster to the destination cluster.

15. The method of claim 10, wherein the source cluster and the destination cluster form an active/active topology, wherein the destination cluster asynchronously pushes updates to the source cluster and the source cluster asynchronously pushes updates to the destination cluster.

16. The method of claim 10, further comprising:

providing a plurality of spoke clusters, wherein the source cluster is designated a hub cluster and wherein the hub cluster actively pushes the updates to the plurality of spoke clusters and wherein each of the plurality of spoke clusters push updates only to the hub cluster.

17. The method of claim 10, further comprising:

providing a plurality of additional clusters wherein each of the plurality of additional clusters actively pushes updates to each other cluster.

18. A non-transitory computer readable storage medium storing a set of instructions executable by one or more processors to perform a sequence of steps comprising:

storing a set of data entries in a cache located on a source cluster of computer servers that manage a set of data and operations on said data in a distributed computing environment, wherein said source cluster further includes an application residing on at least one server in said source cluster, said application executing one or more insert, update or delete operations on said data entries in the cache;

establishing a connection to a destination cluster of computer servers over a network;

detecting that an update has been performed on a data entry in the cache, wherein the update is caused by the one or more insert, update or delete operations on said data entries;

providing the update to a push replication provider, wherein the push replication provider places the update on a topic;

subscribing to the topic by a publishing service residing on the source cluster, wherein said publishing service reads the update;

applying one or more filters to the update; and

asynchronously pushing, via a pluggable internal transport on the push replication provider, the update from the source cluster to the destination cluster when said update passes the one or more filters, wherein the pluggable internal transport can be switched.

19. The A non-transitory computer readable storage medium of claim 18, wherein the push replication server operates to plug in any pluggable internal transport mechanism that provides store and forward semantics;

wherein the store and forward semantics ensure that each update is kept in a queue such that in the event of a lost connection between the clusters, the update remains on the queue and is replicated once the connection is reestablished.

20. The A non-transitory computer readable storage medium of claim 18, wherein the destination cluster further comprises:

a local cache publisher that reads updates received from the source cluster, performs conflict resolution on said updates and writes said updates to a local cache on the destination cluster.