Cloud based data migration and replication

• US 9,483,540 B2
• Filed: 06/25/2015
• Issued: 11/01/2016
• Est. Priority Date: 01/04/2013
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Karve, Shrikant V., Kulkarni, Janmejay S., Patel, Sarvesh S., Pathak, Ashish R., Patil, Sandeep R.
• Primary Examiner(s): Gidado, Oluwatosin
• Application Number:
  US14/749,852
• Publication Number:
  US 20150302071A1
• Time in Patent Office: 495 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1402 Saving, restoring, recovering or retrying
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 21/31 User authentication
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/108 characterised by resources being split in blocks or fragments
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Abstract

A mechanism is provided in a data processing system for data replication. The mechanism divides a set of data to be replicated into a plurality of slices. Each slice of the plurality of slices is to be copied from a primary site to a master secondary site within a plurality of secondary sites. The plurality of secondary sites comprises the master secondary site and at least one subsidiary peer site. The mechanism copies the plurality of slices from the primary site to the plurality of secondary sites such that the master secondary site stores a first slice of the set of data and each of the at least one subsidiary peer site stores a corresponding slice of the set of data. Responsive to completion of copying the plurality of slices to the plurality of secondary sites, for each of the at least one subsidiary peer site, the mechanism initiates copying of the corresponding slice of the set of data to the master secondary site.

Claims

1. A method, in a data processing system, for data replication, comprising:

dividing a set of data to be replicated into a plurality of slices, wherein each slice of the plurality of slices is to be copied from a primary site to a master secondary site within a plurality of secondary sites, wherein the plurality of secondary sites comprises the master secondary site and at least one subsidiary peer site;

copying the plurality of slices from the primary site to the plurality of secondary sites such that the master secondary site stores a first slice of the set of data and the at least one subsidiary peer site stores at least one remaining slice of the set of data;

responsive to completion of copying the plurality of slices to the plurality of secondary sites, initiating copying of the at least one remaining slice of the set of data from the at least one subsidiary peer site to the master secondary site and enabling logic for a consolidated read view of the set of data, wherein the logic presents a consolidated read view of the set of data to a client as if the set of data is stored at the master secondary site; and

responsive to completion of copying the at least one remaining slice from the at least one subsidiary peer site to the master secondary site, disabling the logic for the consolidated read view of the set of data.

2. The method of claim 1, wherein the primary site comprises a storage cloud.

3. The method of claim 1, wherein the master secondary site comprises a storage cloud.

4. The method of claim 1, wherein the at least one subsidiary peer site comprises a storage cloud.

5. The method of claim 1, wherein dividing the set of data into the plurality of slices comprises:

determining a number of secondary sites;

determining a storage space per secondary site; and

for each given secondary site, responsive to available storage space of the given secondary site being greater than or equal to the storage space, marking a region of storage space of the given secondary site for replication storage.

6. The method of claim 5, wherein dividing the set of data into the plurality of slices further comprises:

responsive to available storage space of the given secondary site not being greater than or equal to the storage space, determining a space difference; and

responsive to a next secondary site having additional storage space, shifting a next slice to start on previous slice end and setting a new slice length for the given secondary site and the next secondary site.

7. The method of claim 1, further comprising:

establishing a partnership between the primary site and each of the plurality of secondary sites; and

establishing a partnership between each of the at least one subsidiary peer site and the master secondary site.

8. The method of claim 1, wherein initiating copying of the at least one remaining slice of the set of data to the master secondary site comprises initiating a background copy thread on each of the at least one subsidiary peer site.

9. The method of claim 1, further comprising:

verifying data from each of the at least one subsidiary peer site is copied to the master secondary site.

10. A computer program product comprising a non-transitory computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to:

divide a set of data to be replicated into a plurality of slices, wherein each slice of the plurality of slices is to be copied from a primary site to a master secondary site within a plurality of secondary sites, wherein the plurality of secondary sites comprises the master secondary site and at least one subsidiary peer site;

copy the plurality of slices from the primary site to the plurality of secondary sites such that the master secondary site stores a first slice of the set of data and the at least one subsidiary peer site stores at least one remaining slice of the set of data;

responsive to completion of copying the plurality of slices to the plurality of secondary sites, initiate copying of the at least one remaining slice of the set of data from the at least one subsidiary peer site to the master secondary site and enable logic for a consolidated read view of the set of data, wherein the logic presents a consolidated read view of the set of data to a client as if the set of data is stored at the master secondary site; and

responsive to completion of copying the at least one remaining slice from the at least one subsidiary peer site to the master secondary site, disable the logic for the consolidated read view of the set of data.

11. The computer program product of claim 10, wherein at least one of the primary site or the master secondary site comprises a storage cloud.

12. The computer program product of claim 10, wherein the at least one subsidiary peer site comprises a storage cloud.

13. The computer program product of claim 10, wherein dividing the set of data into the plurality of slices comprises:

determining a number of secondary sites;

determining a storage space per secondary site; and

for each given secondary site, responsive to available storage space of the given secondary site being greater than or equal to the storage space, marking a region of storage space of the given secondary site for replication storage.

14. The computer program product of claim 13, wherein dividing the set of data into the plurality of slices further comprises:

responsive to available storage space of the given secondary site not being greater than or equal to the storage space, determining a space difference; and

responsive to a next secondary site having additional storage space, shifting a next slice to start on previous slice end and setting a new slice length for the given secondary site and the next secondary site.

15. The computer program product of claim 10, wherein the computer readable program further causes the computing device to:

verify data from each of the at least one subsidiary peer site is copied to the master secondary site.

16. An apparatus, comprising:

a processor; and

a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to;

divide a set of data to be replicated into a plurality of slices, wherein each slice of the plurality of slices is to be copied from a primary site to a master secondary site within a plurality of secondary sites, wherein the plurality of secondary sites comprises the master secondary site and at least one subsidiary peer site;

copy the plurality of slices from the primary site to the plurality of secondary sites such that the master secondary site stores a first slice of the set of data and the at least one subsidiary peer site stores at least one remaining slice of the set of data;

responsive to completion of copying the plurality of slices to the plurality of secondary sites, initiate copying of the at last one remaining slice of the set of data from the at least one subsidiary peer site to the master secondary site and enable logic for a consolidated read view of the set of data, wherein the logic presents a consolidated read view of the set of data to a client as if the set of data is stored at the master secondary site; and

responsive to completion of copying the at least one remaining slice from the at least one subsidiary peer site to the master secondary site, disable the logic for the consolidated read view of the set of data.

17. The apparatus of claim 16, wherein the instructions further cause the processor to:

verify data from each of the at least one subsidiary peer site is copied to the master secondary site.

18. The apparatus of claim 16, wherein at least one of the primary site or the master secondary site comprises a storage cloud.

19. The apparatus of claim 16, wherein the at least one subsidiary peer site comprises a storage cloud.

20. The apparatus of claim 16, wherein dividing the set of data into the plurality of slices comprises:

determining a number of secondary sites;

determining a storage space per secondary site;

for each given secondary site, responsive to available storage space of the given secondary site being greater than or equal to the storage space, marking a region of storage space of the given secondary site for replication storage;

responsive to available storage space of the given secondary site not being greater than or equal to the storage space, determining a space difference; and

responsive to a next secondary site having additional storage space, shifting a next slice to start on previous slice end and setting a new slice length for the given secondary site and the next secondary site.