OPTIMIZING DATA TRANSFER ACROSS MULTIPLE ASYNCHRONOUS DATA REPLICATION SESSIONS USING SHARED BANDWIDTH

• US 20150207874A1
• Filed: 01/21/2014
• Published: 07/23/2015
• Est. Priority Date: 01/21/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Diel, Eduard A., McBride, Gregory E., McClure, Alan G., Shackelford, David M., Crockett, Robert N., Shehab, Nadim P.
• Granted Patent: US 9,729,629 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: H04L 41/0896 Bandwidth or capacity management, i.e. automatically increasing or decreasing capacities flow or congestion control using dynamic resource allocation, e.g. in-call renegotiation, H04L47/76
• CPC Class Codes: H04L 41/147 for predicting network behaviour
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Abstract

A computer implemented method includes the steps of establishing a plurality of storage control sessions, wherein each storage control session associates at least one primary storage controller and at least one secondary storage controller with a predetermined path through a data transport network that interconnects the primary and secondary storage controllers; identifying storage control sessions that have data available to be transferred from primary storage controllers to secondary storage controllers; accumulating available data for each session and determining transfer time through the network for each data element to estimate prospective bandwidth utilization; and determining, for a predetermined time interval, whether transferring the available data would fully occupy available bandwidth of the data transport network for the predetermined time interval. When available bandwidth would be fully occupied, the available data accumulated during predetermined time intervals prior to the predetermined time interval in which fully occupied bandwidth was detected is transferred.

Claims

1. A computer-implemented method, carried out by one or more processors, for sharing bandwidth in an asynchronous data replication system having multiple storage control sessions, the method comprising the steps of:

establishing a plurality of storage control sessions, wherein each storage control session associates at least one primary storage controller and at least one secondary storage controller with a predetermined path through a data transport network that interconnects the primary and secondary storage controllers;

identifying storage control sessions that have data available to be transferred from primary storage controllers to secondary storage controllers;

accumulating available data for each session and determining transfer time through the network for each data element to estimate prospective bandwidth utilization;

determining, for a predetermined time interval, whether transferring the available data would fully occupy available bandwidth of the data transport network for the predetermined time interval; and

when available bandwidth would be fully occupied, transferring the available data accumulated during predetermined time intervals prior to the predetermined time interval in which fully occupied bandwidth was detected.

2. The method in accordance with claim 1, wherein identifying storage control sessions that have data available to be transferred further comprises the steps of:

sequentially examining journal memory locations maintained by system data movers controlling the plurality of sessions; and

when the journal memory locations for a particular session contain one or more records, identifying the particular session as a session having data available to be transferred.

3. The method in accordance with claim 2, wherein contents of the journal memory locations are available through a master data set accessible by all sessions.

4. The method in accordance with claim 1, wherein accumulating available data for each session and determining transfer time through the network further comprises the steps of:

examining each data element available to be transferred for each session and associating a corresponding data element size with each data element;

analyzing paths through the data transport network associated with each session, including data transport network nodes shared among sessions;

evaluating available bandwidth for each session, based, at least in part, upon capacity of shared nodes; and

determining transfer time through the network for each data element based, at least in part, upon data element size and available bandwidth of the session associated with the data element.

5. The method in accordance with claim 4, wherein analyzing paths through the data transport network further comprises the step of retrieving network pathways and bandwidth information from network configuration details stored in shared memory accessible to all the sessions.

6. The method in accordance with claim 4, wherein evaluating available bandwidth further comprises the step of determining capacity of the most restrictive node among nodes in a session'"'"'s path.

7. The method in accordance with claim 4, further comprising the steps of:

for each session examined for a particular predetermined time interval, accumulating all of the data that would be transferred by that session in accumulator memory associated with that session;

examining each accumulator to determine whether the data in any accumulator exceeds the capacity of a node that it touches; and

if data in any accumulator exceeds the capacity of a node that it touches, transferring the data associated with the session whose accumulator is being examined.

8. The method in accordance with claim 7, further comprising the steps of:

determining an amount of data to be transferred in a subsequent set of reads by the session whose accumulator is being examined to determine prospective data transfer; and

sharing information relating to prospective data transfer with all sessions.

9. A computer program product for sharing bandwidth in an asynchronous data replication system having multiple storage control sessions, including one or more computer-readable storage media and program instructions stored on at least one of the one or more storage media, wherein execution of the program instructions by one or more processors of a computer system causes the one or more processors to carry out the acts of:

establishing a plurality of storage control sessions, wherein each storage control session associates at least one primary storage controller and at least one secondary storage controller with a predetermined path through a data transport network that interconnects the primary and secondary storage controllers;

identifying storage control sessions that have data available to be transferred from primary storage controllers to secondary storage controllers;

accumulating available data for each session and determining transfer time through the network for each data element to estimate prospective bandwidth utilization;

determining, for a predetermined time interval, whether transferring the available data would fully occupy available bandwidth of the data transport network for the predetermined time interval; and

when available bandwidth would be fully occupied, transferring the available data accumulated during predetermined time intervals prior to the predetermined time interval in which fully occupied bandwidth was detected.

10. The computer program product in accordance with claim 9, wherein execution of the program instructions by the one or more processors of the computer system causes the one or more processors to carry out the additional acts of:

sequentially examining journal memory locations maintained by system data movers controlling the plurality of sessions; and

when the journal memory locations for a particular session contain one or more records, identifying the particular session as a session having data available to be transferred.

11. The computer program product in accordance with claim 10, wherein contents of the journal memory locations are available through a master data set accessible by all sessions.

12. The computer program product in accordance with claim 9, wherein program instructions for accumulating available data for each session and determining transfer time through the network cause the one or more processors to carry out the additional acts of:

examining each data element available to be transferred for each session and associating a corresponding data element size with each data element;

analyzing paths through the data transport network associated with each session, including data transport network nodes shared among sessions;

evaluating available bandwidth for each session, based, at least in part, upon capacity of shared nodes; and

determining transfer time through the network for each data element based, at least in part, upon data element size and available bandwidth of the session associated with the data element.

13. The computer program product in accordance with claim 12, wherein program instructions for analyzing paths through the data transport network further comprise program instructions to carry out the additional act of:

retrieving network pathways and bandwidth information from network configuration details stored in shared memory accessible to all the sessions.

14. The computer program product in accordance with claim 12, wherein program instructions for evaluating available bandwidth further comprise program instructions to carry out the additional act of:

determining capacity of the most restrictive node among nodes in a session'"'"'s path.

15. The computer program product in accordance with claim 12, wherein execution of the program instructions by the one or more processors of the computer system causes the one or more processors to carry out the additional acts of:

for each session examined for a particular predetermined time interval, accumulating all of the data that would be transferred by that session in accumulator memory associated with that session;

examining each accumulator to determine whether the data in any accumulator exceeds the capacity of a node that it touches; and

if data in any accumulator exceeds the capacity of a node that it touches, transferring the data associated with the session whose accumulator is being examined.

16. The computer program product in accordance with claim 15, wherein execution of the program instructions by the one or more processors of the computer system causes the one or more processors to carry out the additional acts of:

determining an amount of data to be transferred in a subsequent set of reads by the session whose accumulator is being examined to determine prospective data transfer; and

sharing information relating to prospective data transfer with all sessions.

17. A computer system for sharing bandwidth in an asynchronous data replication system having multiple storage control sessions, the computer system comprising:

one or more computer processors;

one or more computer-readable storage media; and

program instructions stored on the computer-readable storage media for execution by at least one of the one or more processors, the program instructions comprising;

program instructions to establish a plurality of storage control sessions, wherein each storage control session associates at least one primary storage controller and at least one secondary storage controller with a predetermined path through a data transport network that interconnects the primary and secondary storage controllers;

program instructions to identify storage control sessions that have data available to be transferred from primary storage controllers to secondary storage controllers;

program instructions to accumulate available data for each session and determine transfer time through the network for each data element to estimate prospective bandwidth utilization;

program instructions to determine, for a predetermined time interval, whether transferring the available data would fully occupy available bandwidth of the data transport network for the predetermined time interval; and

program instructions to transfer, when available bandwidth would be fully occupied, the available data accumulated during predetermined time intervals prior to the predetermined time interval in which fully occupied bandwidth was detected.

18. The computer system of claim 17, wherein the program instructions to identify storage control sessions that have data available to be transferred further comprise program instructions to:

sequentially examine journal memory locations maintained by system data movers controlling the plurality of sessions; and

when the journal memory locations for a particular session contain one or more records, identify the particular session as a session having data available to be transferred.

19. The computer system of claim 18, wherein contents of the journal memory locations are available through a master data set accessible by all sessions.

20. The computer system of claim 17, wherein the program instructions to accumulate available data for each session and determine transfer time through the network further comprise program instructions to:

examine each data element available to be transferred for each session and associate a corresponding data element size with each data element;

analyze paths through the data transport network associated with each session, including data transport network nodes shared among sessions;

evaluate available bandwidth for each session, based, at least in part, upon capacity of shared nodes; and

determine transfer time through the network for each data element based, at least in part, upon data element size and available bandwidth of the session associated with the data element.