Adaptive batch sizing for asynchronous data redundancy

• US 20040260972A1
• Filed: 06/06/2003
• Published: 12/23/2004
• Est. Priority Date: 06/06/2003
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Ji, Minwen, Wilkes, John, Veitch, Alistair
• Granted Patent: US 7,120,825 B2
• Time in Patent Office: Days
• US Class Current: 714/11
• CPC Class Codes: G06F 11/004 Error avoidance G06F11/07 and subgroups take precedence
• CPC Class Codes: G06F 11/2064 while ensuring consistency
• CPC Class Codes: G06F 11/2066 Optimisation of the communication load
• CPC Class Codes: G06F 11/2069 Management of state, configuration or failover
• CPC Class Codes: G06F 11/2074 Asynchronous techniques
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 69/40 for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection management of faults, events, alarms or notifications in data switching networks H04L41/06

Abstract

A technique for adaptive batching for asynchronous data redundancy. A sequence of write transactions are adaptively arranged into a sequence of send batches at the first storage facility. The transactions are received at a second storage facility and applied to a redundant data copy at the second storage facility. The second storage facility may arrange the write transactions according to a sequence of receive batches. The batch sizes may be adaptively adjusted or completed. The batch sizes or adaptive completion of the batches may be based on, for example, availability of a communication medium between the first storage facility and the second storage facility. Each send batch may be forwarded to the second storage facility upon completion.

Claims

1. A method for redundantly backing up data comprising:

performing a sequence of write transactions at a first storage facility;

adaptively arranging the write transactions into a sequence of send batches at the first storage facility;

forwarding send batches to the second storage facility; and

applying the write transactions in the send batches to a redundant data copy at the second storage facility.

2. The method according to claim 1, wherein said adaptively arranging comprises adaptively adjusting batch sizes.

3. The method according to claim 2, further comprising arranging the write transactions according to a sequence of receive batches at the second storage facility prior to said applying.

4. The method according to claim 2, wherein said adjusting batch sizes comprises adjusting the batch sizes based on availability of a communication medium between the first storage facility and the second storage facility.

5. The method according to claim 2, wherein said adjusting batch sizes comprises adjusting the batch sizes based on a level of traffic on a communication medium between the first storage facility and the second storage facility.

6. The method according to claim 5, wherein the send batch size is increased in response to detection of an increased level of communication traffic and decreased in response to detection of a decreased level of communication traffic.

7. The method according to claim 2, wherein said adjusting batch sizes comprises adjusting the batch sizes based on a communication bandwidth consumed by communicating the batches between the first storage facility and the second storage facility.

8. The method according to claim 7, further comprising determining a relationship between a plurality of send batch sizes and communication bandwidth consumed by forwarding a send batch of each send batch size.

9. The method according to claim 8, further comprising selecting a send batch size according to a change in slope of said relationship.

10. The method according to claim 9, wherein said change in slope is identified by an inflection in the relationship of the send batch size vs. the bandwidth consumed.

11. The method according to claim 2, wherein said adjusting batch sizes is based on an expected time between failures.

12. The method according to claim 11, wherein said adjusting batch sizes further comprises selecting an initial duration for collecting write transactions into each send batch.

13. The method according to claim 12, further comprising experimentally determining the expected time between failures for determining said initial duration.

14. The method according to claim 13, further comprising experimentally determining a new expected time between failures and adjusting the initial duration based on a difference between the expected time between failures and the new expected time between failures.

15. The method according to claim 13, determining the expected time between failures for determining said initial duration based on manufacturer'"'"'s specifications of the first storage facility.

16. The method according to claim 1, wherein said adaptively arranging comprises adaptively completing the batches.

17. The method according to claim 16, wherein each send batch is forwarded to the second storage facility upon completion.

18. The method according to claim 17, further comprising arranging the write transactions according to a sequence of receive batches at the second storage facility prior to said applying.

19. The method according to claim 16, wherein said adaptively completing the batches is based on availability of a communication medium between the first storage facility and the second storage facility.

20. The method according to claim 16, wherein said adaptively completing the batches is based on a level of traffic on a communication medium between the first storage facility and the second storage facility.

21. The method according to claim 16, wherein said adaptively completing the batches is based on the communication medium becoming idle.

22. The method according to claim 1, wherein a communication medium between the first storage facility and the second storage facility is shared.

23. The method according to claim 1, wherein a communication medium between the first storage facility and the second storage facility is dedicated.

24. The method according to claim 1, further comprising arranging the write transactions according to a sequence of receive batches at the second storage facility prior to said applying.

25. A system for redundantly backing up data comprising:

a first storage facility for adaptively arranging write transactions into a sequence of send batches; and

a second storage facility for receiving the send batches and for applying the send batches to a redundant data copy at the second storage facility

26. The system according to claim 25, wherein the first storage facility adaptively adjusts batch sizes.

27. The system according to claim 26, wherein the second storage facility arranges the write transactions according to a sequence of receive batches.

28. The system according to claim 26, wherein the first storage facility adaptively adjusts batch sizes based on availability of a communication medium between the first storage facility and the second storage facility.

29. The system according to claim 26, wherein the first storage facility adaptively adjusts batch sizes based on a level of traffic on a communication medium between the first storage facility and the second storage facility.

30. The system according to claim 26, wherein the first storage facility adaptively adjusts batch sizes based on a communication bandwidth consumed by communicating the batches between the first storage facility and the second storage facility.

31. The system according to claim 26, wherein the first storage facility adaptively adjusts batch sizes based on an expected time between failures.

32. The system according to claim 25, wherein the first storage facility adaptively completes the batches.

33. The system according to claim 32, wherein each send batch is forwarded to the second storage facility upon completion.

34. The system according to claim 32, wherein the first storage facility adaptively completes the batches based on availability of a communication medium between the first storage facility and the second storage facility.

35. The system according to claim 32, wherein the first storage facility adaptively completes the batches based on a level of traffic on a communication medium between the first storage facility and the second storage facility.

36. The system according to claim 32, wherein the first storage facility adaptively completes the batches based on the communication medium becoming idle.