Flexible remote data mirroring

• US 8,762,328 B2
• Filed: 11/04/2009
• Issued: 06/24/2014
• Est. Priority Date: 11/11/1999
• Status: Expired due to Term

• Current Assignee: Intellectual Ventures Assets 78, LLC (Intellectual Ventures LLC)
• Original Assignee: Intellectual Ventures II LLC (Intellectual Ventures LLC)
• Inventors: McCabe, Ron, Church, Robert, Camp, Tracy, Card, Stuart W., Schroeder, David J.
• Primary Examiner(s): STRANGE, AARON N
• Application Number:
  US12/612,629
• Publication Number:
  US 20100049798A1
• Time in Patent Office: 1,693 Days
• Field of Search: 707/610, 707/617, 707/624, 707/625, 707/655, 714/6, 709/223
• US Class Current: 707/610
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/2058 using more than 2 mirrored copies
• CPC Class Codes: G06F 11/2064 while ensuring consistency
• CPC Class Codes: G06F 11/2069 Management of state, configuration or failover
• CPC Class Codes: G06F 11/2074 Asynchronous techniques
• CPC Class Codes: G06F 2201/835 Timestamp
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 2201/855 Details of asynchronous mirroring using a journal to transfer not-yet-mirrored changes
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0626 Reducing size or complexity of storage systems
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: H04L 1/08 by repeating transmission, e.g. Verdan system H04L1/1858 and H04L1/189 take precedence
• CPC Class Codes: H04L 1/18 Automatic repetition systems, e.g. Van Duuren systems
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Methods, systems, and configured storage media are provided for flexible data mirroring. In particular, the invention provides many-to-one data mirroring, including mirroring from local servers running the same or different operating systems and/or file systems at two or more geographically dispersed locations. The invention also provides one-to-many data mirroring, mirroring with or without a dedicated private telecommunications link, and mirroring with or without a dedicated server or another server at the destination(s) to assist the remote mirroring unit(s). In addition, the invention provides flexibility by permitting the use of various combinations of one or more external storage units and/or RAID units to hold mirrored data. Spoofing, SCSI and other bus emulations, and further tools and techniques are used in various embodiments of the invention.

Claims

1. A system for storing data, comprising:

an interface configured to receive multiple data changes, the multiple data changes being created in a particular order, wherein at least one of the data changes;

indicates corresponding data to be stored on a storage volume of a host; and

indicates at least one block of the storage volume on where the corresponding data is to be stored;

a meta-data generator configured to, for each of the multiple data changes;

generate meta-data indicating the data change'"'"'s position in the order; and

associate the qenerated meta-data with the data change;

a local buffer configured to store one or more of the multiple data changes and the meta-data associated with the one or more of the multiple data changes;

a local mirror divided into a plurality of units;

a comparator configured to, for the one or more of the multiple data changes;

determine that mirror data in a unit of the local mirror that corresponds to the data change is different from the data indicated by the data change, and

in response to the determining, store the data change in the local buffer with the meta-data associated with the data change; and

a communications link coupled to the local buffer wherein the communications link is configured to send to a remote buffer;

(A) the one or more of the multiple data changes and (B) the meta-data associated with the one or more of the multiple data changes,

wherein the remote buffer is configured to receive and store the sent data changes and the meta-data associated with the sent data changes.

2. The system of claim 1, wherein the meta-data includes a timestamp and a sequence number.

3. The system of claim 1, further comprising:

a communications link monitor configured to monitor a status of the communications link;

wherein the local buffer is configured to pause transmission of the data changes and associated meta-data if the communications link monitor detects an interruption of the communications link.

4. The system of claim 3, wherein the local buffer is further configured to resume transmission of the data changes and associated meta-data if the communications link monitor detects that the communications link is restored.

5. The system of claim 1, further comprising:

a communications link monitor configured to monitor a status of the communications link;

wherein the local buffer is configured to adjust a rate of transmission of the data changes and associated meta-data in response to the communications link monitor detecting a change in available bandwidth.

6. The system of claim 1, wherein the remote buffer is coupled to a remote mirror including all of the data stored on the storage volume at an associated point in time;

and

wherein the remote buffer is configured to update the remote mirror using the data changes.

7. The system of claim 6, wherein the remote buffer is configured to store in the remote mirror an oldest data change of the data changes stored in the remote buffer according to the associated meta-data and remove the oldest data change from the remote buffer.

8. The system of claim 6, further comprising:

a volume reconstructor configured to generate data of the storage volume from a previous point in time from the remote mirror, and at least one of the data changes and the meta-data associated with the at least one data change.

9. The system of claim 6 wherein the remote buffer is configured to, upon receiving a particular one of the data changes and associated meta-data, send an ACK signal to the local buffer,

and wherein the local buffer is configured to, upon receiving the ACK signal, send an ACK-ACK signal to the remote buffer,

and wherein the remote buffer is configured to update the remote mirror using the particular data change only upon receiving the ACK-ACK signal.

10. The system of claim 9 wherein the remote buffer is configured to compute a first signature for the content of the particular data change stored by the remote buffer,

and wherein the remote buffer is configured to include the first signature with the ACK signal it sends to the local buffer,

and wherein the local buffer is configured to compute a second signature for the content of the particular data change stored by the local buffer,

and wherein the local buffer is configured to send an ACK-ACK signal to the remote buffer only if the second signature computed by the local buffer matches the first signature computed by the remote buffer.

11. The system of claim 1,

wherein the local buffer is configured to maintain a particular data change in the local buffer until after an acknowledgement indicating that the particular data change was stored in the remote buffer is received by the local buffer.

12. The system of claim 11, wherein the local mirror is configured to store the data changes and wherein the local mirror includes all of the data stored on the storage volume at a particular point in time;

and

wherein the interface is further configured to transmit the data changes to the local buffer and the local mirror.

13. The system of claim 1, wherein the remote buffer is coupled to a remote mirror including all of the data stored on the storage volume at a point in time;

and

wherein the local buffer is configured to maintain a particular data change in the local buffer until after an acknowledgement indicating that the particular data change was stored in the remote mirror is received by the local buffer.

14. The system of claim 13, wherein the local buffer is configured to remove the particular data change from the local buffer when the acknowledgement indicating that the particular data change was stored in the remote mirror is received by the local buffer.

15. The system of claim 13, wherein the local buffer is configured to remove the particular data change from the local buffer when a signature associated with the particular data change and received by the local buffer is validated.

16. The system of claim 1, wherein the remote buffer is coupled to a remote mirror including all of the data stored on the storage volume at a particular point in time;

and

wherein the remote buffer is configured to store the data changes in the remote mirror corresponding to the order.

17. The system of claim 1, further comprising:

a plurality of hosts, each host including a primary storage volume;

a plurality of first mirroring units, each mirroring unit coupled to an associated host and configured to mirror the primary storage volume of the associated host, such that each host is coupled its own first mirroring unit; and

a second mirroring unit coupled to the first mirroring units, the second mirroring unit configured to mirror the mirrored primary storage volumes of the first mirroring units;

wherein;

a first mirroring unit of the plurality of first mirroring units includes the interface, the meta-data generator, and the local buffer, where the first mirroring unit is coupled to its corresponding host through the interface.

18. The system of claim 17, wherein the second mirroring unit locks access to a mirror of a mirrored primary storage volume in response to an access by an associated first mirroring unit.

19. The system of claim 1, wherein the remote buffer is coupled to a remote mirror divided into a plurality of sections, the sections of the remote mirror corresponding to the blocks of the storage volume on a one-to-one basis;

and

wherein the remote buffer is configured to store the data changes in the corresponding sections of the remote mirror.

20. The system of claim 1, wherein:

the interface is a standard storage subsystem bus interface; and

the data changes are write requests received through the standard storage subsystem bus interface.

21. The system of claim 20 wherein the local mirror is coupled to the interface through a second standard storage subsystem bus interface;

and

wherein the interface is configured to provide a first copy of the write requests to the local mirror through the second standard storage subsystem bus interface and to provide a second copy of the write requests to the local buffer.

22. The system of claim 1 wherein the remote buffer stores the data changes and associated meta-data in a nonvolatile storage device.

23. A system for storing data, comprising:

a remote buffer comprising a memory that stores remote buffer elements, the remote buffer coupled to a communications link and configured to receive via the communications link and store a plurality of data changes created in a particular order, wherein one or more the data changes;

indicates corresponding data to be stored on a storage volume of a host,

indicates at least one block of the storage volume on which the corresponding data is to be stored, and

is associated with meta-data indicating the data change'"'"'s position in the particular order; and

a remote mirror;

wherein;

the remote buffer is configured to, upon receiving a particular one of the data changes and associated meta-data, send an ACK signal to a local buffer from which the remote buffer receives the data changes, and

the remote buffer is configured to update the remote mirror using the particular data change only upon receiving an ACK-ACK signal from the local buffer subsequent to sending the ACK signal.

24. The system of claim 23, wherein the remote mirror includes all of the data stored on the storage volume at an associated point in time,

wherein the remote buffer is configured to update the remote mirror using the data changes.

25. The system of claim 23 wherein the remote buffer stores the data changes and associated meta-data in a nonvolatile storage device.

26. The system of claim 23 wherein the remote buffer is configured to compute a signature for the content of the particular data change stored by the remote buffer,

and wherein the remote buffer is configured to include the signature with the ACK signal it sends to the local buffer.

27. A computer-readable storage device having contents configured to cause a computing system to perform operations for storing data, the operations comprising:

receiving a plurality of data changes created in a particular order, wherein at least one of the data changes;

indicates corresponding data to be stored on a storage volume of a host, and

indicates at least one block of the storage volume on which the corresponding data is to be stored;

associating with one or more of the data changes, meta-data indicating the data change'"'"'s position in the particular order; and

for the one or more data changes;

determining that data in the block indicated by the data change is different from the data indicated by the data change, and

in response to the determining, storing the data change in a local buffer with the meta-data associated with the data change.

28. The computer-readable storage device of claim 27, the operations further comprising:

updating a remote mirror using the received and stored data changes.

29. The computer-readable storage device of claim 27 where storing the received data change with the associated meta-data comprises storing the data change in a nonvolatile storage device.

30. The computer-readable storage device of claim 29, the operations further comprising:

upon receiving a specific one of the data changes and associated meta-data, sending an ACK signal to the local buffer from which the data changes and meta-data are received,

and wherein the remote mirror is updated using the specific data change only upon receiving an ACK-ACK signal from the local buffer subsequent to sending the ACK signal.

31. The computer-readable storage device of claim 30, the operations further comprising:

computing a signature for the content of the stored specific data change,

and wherein the ACK signal sent to the local buffer includes the signature.

32. A system for conveying data, comprising:

a memory;

a processor;

an interface coupled to the memory, the interface and memory configured to receive and store a plurality of data changes created in a particular order, wherein at least one of the changes;

indicates corresponding data to be stored on a storage volume of a host, and

indicates at least one block of the storage volume on which the indicated data is to be stored;

a meta-data generator configured to;

generate meta-data indicating the data change'"'"'s position in the order; and

associate the generated meta-data with the data change;

a local buffer configured to (1) store one or more of the data changes and the meta-data associated with the one or more of the multiple data changes, (2) transmit, to a remote buffer via a communications link coupled to the local buffer, the data changes and the meta-data associated with the one or more of the multiple data changes stored in the local buffer, and (3) upon receiving an ACK signal signifying the remote buffer'"'"'s receipt of a particular one of the data changes and associated meta-data block, send an ACK-ACK signal to the remote buffer.

33. The system of claim 32 wherein the received ACK signal includes a first signature computed by the remote buffer for the content of the particular data change stored by the remote buffer,

and wherein the local buffer is configured to compute a second signature for the content of the particular data change stored by the local buffer,

and wherein the local buffer is configured to send an ACK-ACK signal to the remote buffer only if the second signature computed by the local buffer matches the first signature computed by the remote buffer.

34. A method in a computing system for conveying data, comprising:

receiving, from a processor, a plurality of data changes in a particular order, wherein one or more of the data changes;

indicates corresponding data to be stored on a storage volume of a host, and;

indicates at least one block of the storage volume on which the indicated data is to be stored;

associating with at least one of the data changes, based on the order, generated meta-data indicating the data change'"'"'s position in the order;

storing, in a local buffer, at least one of the received data changes and meta-data associated with the at least one of the multiple data changes;

transmitting the stored data changes and the meta-data the meta-data associated with the stored data changes to a remote buffer via a communications link; and

upon receiving an ACK signal signifying the remote buffer'"'"'s receipt of a particular one of the data changes and associated meta-data, sending an ACK-ACK signal to the remote buffer.

35. The method of claim 34 wherein the received ACK signal includes a first signature computed by the remote buffer for the content of the particular data change stored by the remote buffer, the method further comprising:

computing a second signature for the content of the particular data change stored by the local buffer; and

sending an ACK-ACK signal to the remote buffer only if the second signature computed by the local buffer matches the first signature computed by the remote buffer.