Distributed storage array

• US 20070250672A1
• Filed: 04/21/2006
• Published: 10/25/2007
• Est. Priority Date: 04/21/2006
• Status: Active Grant

• Current Assignee: Valtrus Innovations Limited (f/k/a Dolya Holdco 9 Limited) (Key Patent Innovations Limited)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Stroberger, Christopher, Spence, Susan
• Granted Patent: US 7,376,805 B2
• Time in Patent Office: Days
• US Class Current: 711/162
• CPC Class Codes: G06F 11/2064 while ensuring consistency
• CPC Class Codes: G06F 11/2074 Asynchronous techniques
• CPC Class Codes: G06F 2201/855 Details of asynchronous mirroring using a journal to transfer not-yet-mirrored changes

Abstract

Systems and methods for asynchronous backup of virtual disks in a distributed storage array are disclosed. An exemplary method may comprise receiving an IO stream at the virtual disks in the distributed storage array, writing data to one or more storage cells of the virtual disks in the distributed storage array, and logging completed writes in a first batch for each of the one or more storage cells. The method also comprises establishing a consistency point in the IO stream, terminating logging to the first batch for each of the one or more storage cells at the consistency point and logging completed writes after the consistency point to a new batch for each of the one or more storage cells. The method also comprises transferring writes logged in the first batch to at least one remote storage so that the at least one remote storage is crash consistent.

Claims

1. A method for asynchronous backup of virtual disks in a distributed storage array, comprising:

receiving an IO stream at the virtual disks in the distributed storage array;

writing data to one or more storage cells of the virtual disks in the distributed storage array, and logging completed writes in a first batch for each of the one or more storage cells;

establishing a consistency point in the IO stream;

terminating logging to the first batch for each of the one or more storage cells at the consistency point and logging completed writes after the consistency point to a new batch for each of the one or more storage cells; and

transferring writes logged in the first batch to at least one remote storage so that the at least one remote storage is crash consistent.

2. The method of claim 1, wherein receiving the IO stream includes receiving at least some substantially simultaneous writes from multiple clients.

3. The method of claim 1, wherein logging completed writes is irrespective of any ordering of the writes for different regions of the virtual disks.

4. The method of claim 1, further comprising temporarily suspending all IO to the virtual disks at the consistency point.

5. The method of claim 4, wherein temporarily suspending all IO includes delaying cache commits and write acknowledgements for a predetermined time.

6. The method of claim 4, further comprising temporarily suspending the IO stream for one network roundtrip.

7. The method of claim 6, further comprising determining the time for suspending the IO stream based on the slowest storage cell response time.

8. The method of claim 6, wherein temporarily suspending the IO stream is for about half the time it takes to perform a single write.

9. The method of claim 4, further comprising temporarily suspending the IO stream based on maximum clock skew time.

10. The method of claim 9, further comprising suspending write acknowledgements by the storage cells for a time equal to the maximum skew between any two storage cell clocks in the distributed storage array.

11. The method of claim 9, further comprising determining which writes are included in the first batch based on timestamps from storage cell clocks.

12. A method for batch transition during asynchronous backup of a distributed storage array, comprising:

querying a plurality of storage cells in the distributed storage array for a current Batch ID;

broadcasting the largest Batch ID incremented by one (M+1) to the plurality of storage cells in the distributed storage array;

suspending cache commits for pending and incoming writes at the plurality of storage cells to establish a consistency point; and

setting the Batch ID at the plurality of storage cells to the largest Batch ID incremented by one (M+1) for recording the pending and incoming writes in a new batch after the consistency point.

13. The method of claim 12, further comprising resuming cache commits to the new batch after establishing the consistency point.

14. The method of claim 12, further comprising establishing the consistency point only after a quorum of storage cells acknowledges that cache commits for the pending and incoming writes are suspended.

15. The method of claim 14, resuming cache commits for the pending and incoming writes without incrementing the Batch ID if the predetermined number of storage cells fails to acknowledge that the pending and incoming writes are suspended.

16. The method of claim 14, further comprising timing out the suspending of cache commits if a batch manager fails.

17. A method for batch transition during asynchronous backup of virtual disks in a distributed storage array, comprising:

querying at least a quorum of storage cells in the distributed storage array for a current Batch ID;

broadcasting the largest Batch ID incremented by one (M+1) along with a transition time to at least the quorum of storage cells in the distributed storage array;

setting the Batch ID for at least the quorum of storage cells to the largest Batch ID incremented by one (M+1) at the transition time;

logging completed writes having a timestamp prior to the transition time in an old batch; and

logging completed writes having a timestamp at or after the transition time in a new batch.

18. The method of claim 17, further comprising suspending write acknowledgements to one or more clients for a time equal to the maximum skew between any two clocks in the distributed storage array for the transition time.

19. The method of claim 17, further comprising resuming write acknowledgements to one or more clients after the transition time.

20. The method of claim 17, further comprising acknowledging a successful batch transition after the transition time.

21. The method of claim 17, wherein logging completed writes in the new batch is only after a predetermined number of storage cells acknowledge receipt of the transition time.

22. The method of claim 21, further comprising resuming logging to the old batch if the predetermined number of storage cells fails to acknowledge receipt of the transition time.

23. A system comprising:

a distributed storage array having a plurality of storage cells;

a logging service operatively associated with the plurality of storage cells, the logging service;

logging completed writes in a first batch for each of the storage cells,

establishing a consistency point for terminating logging to the first batch, and

logging completed writes for each of the one or more storage cells after the consistency point to a new batch; and

a remote storage for recording writes logged in the first batch such that the remote storage is a mirror image of the plurality of storage cells as of the consistency point.

24. The system of claim 23, further comprising a batch collection area at the remote storage, the batch collection area for confirming receipt of the first batch before the first batch is removed from the distributed storage array.

25. The system of claim 23, further comprising a batch manager to temporarily suspend an IO stream at the consistency point.

26. The system of claim 23, wherein the batches are maintained on the same storage cells as data being logged in the batches.

27. The system of claim 26, wherein acknowledgements are made to one or more clients only after both a write is completed and logged so that the batches are valid as of the time of writing to the storage cells.

28. The system of claim 23, wherein the batches are maintained in disjointed storage cells so that the batches are transferable to remote storage even if one or more of the storage cells for recording writes fail.

29. The system of claim 23, wherein the batches are maintained as one or more virtual disk to provide redundancy for the batches.

30. A distributed storage array with asynchronous backup, comprising:

first batch means for logging completed writes for each of a plurality of storage cells;

means for terminating logging by the first batch means at a consistency point;

second batch means for logging completed writes after the consistency point; and

means for transferring the completed writes by the first batch means to a remote storage for maintaining a mirror image of the plurality of storage cells consistent to the consistency point.

31. The distributed storage array of claim 30, further comprising means for temporarily suspending all IO at the consistency point.