Synchronous data replication in a content management system

• US 10,169,440 B2
• Filed: 01/27/2014
• Issued: 01/01/2019
• Est. Priority Date: 01/27/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Chen, Ning, Ho, Mang-Rong, Kozina, Gerald E., Li, Yong L., Madani, Masoud, Shah, Mayank V., Silva, George F., Yaung, Alan T., Zientara, John T.
• Primary Examiner(s): Pham, Tuan A
• Application Number:
  US14/164,926
• Publication Number:
  US 20150213102A1
• Time in Patent Office: 1,800 Days
• Field of Search: None
• CPC Class Codes: G06F 16/275 Synchronous replication

Abstract

Methods and apparatus, including computer program products, implementing and using techniques for data replication in a cluster. In response to identifying data to replicate in a multi-node system by a first node in a cluster, the data is written concurrently to other nodes in the cluster based on a rule set in a repository. In response to detecting a criterion described in the rule set, an action is performed.

Claims

1. A computer program product for data replication during ingestion of data in a content management system, the computer program product comprising a non-transitory computer readable storage medium having program code embodied therewith, the program code being executable by a processor to:

receive, by the processor and in response to instructions by an application programming interface, blocks of data to be written into a content management system;

retrieve a replication rule from a replication rule repository, wherein the replication rule includes instructions for handling the blocks of data in a strict mode and in a tolerant mode, respectively;

replicate the blocks of data to different live replication nodes in the content management system using the strict mode in situations that require strict data consistency and no data losses in response to writing the blocks of data to the first node in the content management system by the processor and in response to instructions by the application programming interface, wherein the strict mode requires that the blocks of data be persisted synchronously to the first node and all its replica nodes before the data node returns a success status;

replicate the blocks of data to different live replication nodes in the content management system using the tolerant mode in situations that require high throughput and high availability in data ingestion, in response to writing the blocks of data to the first node in the content management system by the processor and in response to instructions by the application programming interface, wherein the tolerant mode requires that the blocks of data be persisted synchronously to a user-assigned number of replica nodes of the first data node before the first data node returns a success status and the data blocks be asynchronously persisted to any failed data nodes at a later point in time; and

receive either a success status or a failure status for persisting the data in the content management system.

2. The computer program product of claim 1, further comprising program code executable by the processor to:

receive a status back from each replication node.

3. The computer program product of claim 2, further comprising program code executable by the processor to:

determine, from the received statuses, whether to rollback the replication of data from the replication nodes.

4. The computer program product of claim 3, wherein in the strict mode the determining includes:

determining to rollback the replication data in response to receiving a status indicating failure to persist the data from at least one of the replication nodes.

5. The computer program product of claim 3, wherein in the tolerant mode the determining includes:

determining to rollback the replication data in response to receiving a status indicating failure to persist the data from a number of nodes that exceeds a predetermined quorum of nodes.

6. The computer program product of claim 3, further comprising program code executable by the processor to:

in response to determining that the replication data should not be rolled back in the tolerant mode, replicate the data to one or more responding replication nodes; and

replicate the data from the responding replication nodes to failed replication nodes asynchronously at a later point in time.

7. The computer program product of claim 1, further comprising program code executable by the processor to:

provide an application programming interface for replicating data.

8. The computer program product of claim 1, further comprising program code executable by the processor to:

return one of a success message and a fail message for the data replication to a user.

9. A system for data replication during ingestion of data in a content management system, comprising:

a processor; and

a memory storing instructions that when executed by the processor, cause the processor to;

receive, by the processor and in response to instructions by an application programming interface, blocks of data to be written into a content management system;

retrieve a replication rule from a replication rule repository, wherein the replication rule includes instructions for handling the blocks of data in a strict mode and in a tolerant mode, respectively;

replicate the blocks of data to different live replication nodes in the content management system using the strict mode in situations that require strict data consistency and no data losses in response to writing the blocks of data to the first node in the content management system by the processor and in response to instructions by the application programming interface, wherein the strict mode requires that the blocks of data be persisted synchronously to the first node and all its replica nodes before the data node returns a success status;

replicate the blocks of data to different live replication nodes in the content management system using the tolerant mode in situations that require high throughput and high availability in data ingestion, in response to writing the blocks of data to the first node in the content management system by the processor and in response to instructions by the application programming interface, wherein the tolerant mode requires that the blocks of data be persisted synchronously to a user-assigned number of replica nodes of the first data node before the first data node returns a success status and the data blocks be asynchronously persisted to any failed data nodes at a later point in time; and

receive either a success status or a failure status for persisting the data in the content management system.

10. The system of claim 9, wherein the memory further stores instructions that when executed by the processor cause the processor to:

receive a status back from each replication node.

11. The system of claim 10, wherein the memory further stores instructions that when executed by the processor cause the processor to:

determine, from the received statuses, whether to rollback the replication of data from the replication nodes.

12. The system of claim 11, wherein in the strict mode the determining includes:

determining to rollback the replication data in response to receiving a status indicating failure to persist the data from at least one of the replication nodes.

13. The system of claim 11, wherein in the tolerant mode the determining includes:

determining to rollback the replication data in response to receiving a status indicating failure to persist the data from a number of nodes that exceeds a predetermined quorum of nodes.

14. The system of claim 11, wherein the memory further stores instructions that when executed by the processor cause the processor to:

in response to determining that the replication data should not be rolled back in the tolerant mode, replicate the data to one or more responding replication nodes; and

replicate the data from the responding replication nodes to failed replication nodes asynchronously at a later point in time.

15. The system of claim 9, wherein the memory further stores instructions that when executed by the processor cause the processor to:

provide an application programming interface for replicating data.

16. The system of claim 9, wherein the memory further stores instructions that when executed by the processor cause the processor to:

return one of a success message and a fail message for the data replication to a user.