Performance of asynchronous replication in HSM integrated storage systems

• US 9,886,447 B2
• Filed: 08/22/2014
• Issued: 02/06/2018
• Est. Priority Date: 08/22/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Araki, Hiroshi, Bish, Thomas W., Iwasaki, Norie, Miyoshi, Hiroyuki, Takai, Satoshi
• Primary Examiner(s): Mofiz, Apu
• Assistant Examiner(s): HOLLAND, SHERYL L
• Application Number:
  US14/466,531
• Publication Number:
  US 20160055171A1
• Time in Patent Office: 1,264 Days
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 16/185 Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof details of archiving G06F16/11
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 3/06 Digital input from, or digital output to, record carriers , e.g. RAID, emulated record carriers or networked record carriers
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0649 Lifecycle management
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays

Abstract

A mechanism is provided in a data processing system for asynchronous replication in a hierarchical storage management integrated storage system. For a given file to be replicated from a primary storage system to a remote storage system, the primary storage system accesses the remote storage system to determine file existence and migration status at the remote storage system for the given file. Responsive to the primary storage system determining that the given file exists and has been migrated from first tier storage to second tier storage at the remote storage system, the primary storage system determines a first performance penalty for file recall and a second performance penalty for sending excess data from the primary storage system to the remote storage system. Responsive to the primary storage system determining that the first performance penalty is greater than the second performance penalty, the primary storage system sends whole file data for the given file to the remote storage system to replicate the given file at the remote storage system.

Claims

1. A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to:

for a given file to be replicated from a primary storage system to a remote storage system, access the remote storage system to determine file existence and migration status at the remote storage system for the given file;

responsive to the primary storage system determining that the given file exists at the remote storage system and has been migrated from first tier storage to second tier storage at the remote storage system, determine a first performance penalty for recall of the given file from the second tier storage to the first tier storage at the remote storage system and a second performance penalty for sending whole file data of the given file from the primary storage system to the remote storage system, wherein the primary storage system determines the first performance penalty as follows;

((T_mnt+T_seek+S_fold/R_taperd)+(T_hashpri*(S_fnew/S_hashblk))+(T_hashrem*(S_fold/S_hashblk))+((S_hash*(S_fold/S_hashblk)/R_net))),

wherein T_mnt represents a time it would take to mount the second tier storage at the remote site, wherein T_seek represents a time it would take to seek the given file in the second tier storage, wherein S_fold represents a size of the given file in the second tier storage, wherein R_taperd represents a read throughput of the second tier storage, wherein T_hashpri represents an average time to calculate a hash of a block size at the primary storage system, wherein S_fnew represents an entire file size of the given file at the primary storage system, wherein S_hashblk represents block size of the given file at the primary storage system, wherein T_hashrem represents average time to calculate a hash of a block size at the remote storage system, wherein S_hash represents size of the calculated hash, and wherein R_net represents network transfer throughput obtained from past replications; and

responsive to the primary storage system determining that the first performance penalty is greater than the second performance penalty, send whole file data for the given file from the primary storage system to the remote storage system to replicate the given file at the remote storage system.

2. The computer program product of claim 1, wherein the computer readable program further causes the computing device to:

responsive to the primary storage system determining that the first performance penalty is not greater than the second performance penalty, send incremental file data for the given file from the primary storage system to the remote storage system to replicate the given file at the remote storage system, wherein the remote storage system recalls the given file from second tier storage to first tier storage and merges the incremental file data with the recalled file to replicate the given file.

3. The computer program product of claim 2, wherein sending incremental file data comprises:

reading file data for the given file;

computing a hash for the given file;

receiving a hash for the given file from the remote storage system; and

comparing the computed hash to the received hash and identifying delta bytes based on the comparison, wherein the delta bytes are bytes within the given file to be modified to replicate the given file at the remote storage system, wherein the incremental file data comprise the delta bytes.

4. The computer program product of claim 2, wherein sending incremental file data comprises determining the incremental file data by comparing a current version of the given file at a current synchronization point and a previous version of the given file at a previous synchronization point at the primary storage system.

5. The computer program product of claim 2, wherein sending incremental file data comprises accessing a journaling feature at the primary storage system to identify a delta of the given file between two synchronization points.

6. The computer program product of claim 1, wherein the primary storage system determines the second performance penalty as follows:

(S_fnew/R_net),

wherein S_fnew represents an entire file size of the given file at the primary storage system and wherein R_net represents network transfer throughput obtained from past replications.

7. The computer program product of claim 1, wherein the primary storage system determines the first performance penalty as follows:

(T_mnt+T_seek+S_fold/R_taperd),

wherein T_mnt represents a time it would take to mount the second tier storage at the remote site, wherein T_seek represents a time it would take to seek the given file in the second tier storage, wherein S_fold represents a size of the given file in the second tier storage, and wherein R_taperd represents a read throughput of the second tier storage.

8. The computer program product of claim 1, wherein the remote storage system truncates the given file at offset 0 to ensure a recall is not triggered for the given file.

9. The computer program product of claim 1, wherein the computer readable program further causes the computing device to:

responsive to the primary storage system determining that the given file exists in first tier storage at the remote storage system, send incremental file data for the given file from the primary storage system to the remote storage system to replicate the given file at the remote storage system, wherein the remote storage merges the incremental file data with the given file in the first tier storage at the remote storage system.

10. The computer program product of claim 1, wherein the computer readable program further causes the computing device to:

responsive to the primary storage system determining that the given file does not exist at the remote storage system, send whole file data for the given file from the primary storage system to the remote storage system to replicate the given file at the remote storage system.

11. A method, in a data processing system, for asynchronous replication in a hierarchical storage management integrated storage system, the method comprising:

for a given file to be replicated from a primary storage system to a remote storage system, accessing the remote storage system to determine file existence and migration status at the remote storage system for the given file;

responsive to the primary storage system determining that the given file exists at the remote storage system and has been migrated from first tier storage to second tier storage at the remote storage system, determining a first performance penalty for recall of the given file from the second tier storage to the first tier storage at the remote storage system and a second performance penalty for sending whole file data of the given file from the primary storage system to the remote storage system, wherein the primary storage system determines the first performance penalty as follows;

((T_mnt+T_seek+S_fold/R_taperd)+(T_hashpri*(S_fnew/S_hashblk))+(T_hashrem*(S_fold/S_hashblk))+((S_hash*(S_fold/S_hashblk)/R_net))),

wherein T_mnt represents a time it would take to mount the second tier storage at the remote site, wherein T_seek represents a time it would take to seek the given file in the second tier storage, wherein S_fold represents a size of the given file in the second tier storage, wherein R_taperd represents a read throughput of the second tier storage, wherein T_hashpri represents an average time to calculate a hash of a block size at the primary storage system, wherein S_fnew represents an entire file size of the given file at the primary storage system, wherein S_hashblk represents block size of the given file at the primary storage system, wherein T_hashrem represents average time to calculate a hash of a block size at the remote storage system, wherein S_hash represents size of the calculated hash, and wherein R_net represents network transfer throughput obtained from past replications; and

responsive to the primary storage system determining that the first performance penalty is greater than the second performance penalty, sending whole file data for the given file from the primary storage system to the remote storage system to replicate the given file at the remote storage system.

12. The method of claim 11, further comprising:

responsive to the primary storage system determining that the first performance penalty is not greater than the second performance penalty, sending incremental file data for the given file from the primary storage system to the remote storage system to replicate the given file at the remote storage system, wherein the remote storage system recalls the given file from second tier storage to first tier storage and merges the incremental file data with the recalled file to replicate the given file.

13. The method of claim 12, wherein sending incremental file data comprises:

reading file data for the given file;

computing a hash for the given file;

receiving a hash for the given file from the remote storage system; and

comparing the computed hash to the received hash and identifying delta bytes based on the comparison, wherein the delta bytes are bytes within the given file to be modified to replicate the given file at the remote storage system, wherein the incremental file data comprise the delta bytes.

14. The method of claim 11, wherein the primary storage system determines the second performance penalty as follows:

(S_fnew/R_net),

wherein S_fnew represents an entire file size of the given file at the primary storage system and wherein R_net represents network transfer throughput obtained from past replications.

15. The method of claim 11, wherein the primary storage system determines the first performance penalty as follows:

(T_mnt+T_seek+S_fold/R_taperd),

wherein T_mnt represents a time it would take to mount the second tier storage at the remote site, wherein T_seek represents a time it would take to seek the given file in the second tier storage, wherein S_fold represents a size of the given file in the second tier storage, and wherein R_taperd represents a read throughput of the second tier storage.

16. An apparatus comprising:

a processor; and

a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to;

for a given file to be replicated from a primary storage system to a remote storage system, access the remote storage system to determine file existence and migration status at the remote storage system for the given file;

responsive to the primary storage system determining that the given file exists at the remote storage system and has been migrated from first tier storage to second tier storage at the remote storage system, determine a first performance penalty for recall of the given file from the second tier storage to the first tier storage at the remote storage system and a second performance penalty for sending whole file data of the given file from the primary storage system to the remote storage system, wherein the primary storage system determines the first performance penalty as follows;

((T_mnt+T_seek+S_fold/R_taperd)+(T_hashpri*(S_fnew/S_hashblk))+(T_hashrem*(S_fold/S_hashblk))+((S_hash*(S_fold/S_hashblk)/R_net))),

wherein T_mnt represents a time it would take to mount the second tier storage at the remote site, wherein T_seek represents a time it would take to seek the given file in the second tier storage, wherein S_fold represents a size of the given file in the second tier storage, wherein R_taperd represents a read throughput of the second tier storage, wherein T_hashpri represents an average time to calculate a hash of a block size at the primary storage system, wherein S_fnew represents an entire file size of the given file at the primary storage system wherein S_hashblk represents block size of the given file at the primary storage system, wherein T_hashrem represents average time to calculate a hash of a block size at the remote storage system, wherein S_hash represents size of the calculated hash, and wherein R_net represents network transfer throughput obtained from past replications; and

responsive to the primary storage system determining that the first performance penalty is greater than the second performance penalty, send whole file data for the given file from the primary storage system to the remote storage system to replicate the given file at the remote storage system.

17. The apparatus of claim 16, wherein the instructions further cause the processor to:

responsive to the primary storage system determining that the first performance penalty is not greater than the second performance penalty, send incremental file data for the given file from the primary storage system to the remote storage system to replicate the given file at the remote storage system, wherein the remote storage system recalls the given file from second tier storage to first tier storage and merges the incremental file data with the recalled file to replicate the given file.

18. The apparatus of claim 17, wherein sending incremental file data comprises:

reading file data for the given file;

computing a hash for the given file;

receiving a hash for the given file from the remote storage system; and

comparing the computed hash to the received hash and identifying delta bytes based on the comparison, wherein the delta bytes are bytes within the given file to be modified to replicate the given file at the remote storage system, wherein the incremental file data comprise the delta bytes.

19. The apparatus of claim 16, wherein the instructions further cause the processor to:

responsive to the primary storage system determining that the given file exists in first tier storage at the remote storage system, send incremental file data for the given file from the primary storage system to the remote storage system to replicate the given file at the remote storage system, wherein the remote storage merges the incremental file data with the given file in the first tier storage at the remote storage system.