Reducing latency of access requests in distributed storage systems having a shared data set

• US 7,921,179 B1
• Filed: 01/15/2008
• Issued: 04/05/2011
• Est. Priority Date: 01/15/2008
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Palekar, Amber, Zheng, Cindy, Khona, Rahul
• Primary Examiner(s): Chan; Wing F
• Assistant Examiner(s): Yohannes; Tesfay
• Application Number:
  US12/014,453
• Time in Patent Office: 1,176 Days
• Field of Search: 709/213, 707/83, 707/100, 707/201
• US Class Current: 709/213
• CPC Class Codes: G06F 16/176 Support for shared access to files; File sharing support
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Described are methods and apparatus for reducing latency of read and write requests for a set of storage system sites having a shared data set. An owner site may directly write to the shared data set and contains current data regarding the shared data set. The remote sites may experience substantial latency when accessing the shared data set stored at the owner site. Synchronizing and caching methods may reduce overall read latency experienced at remote sites by periodically transmitting images of the shared data set to the remote sites. Also, a migration method may be used to change ownership of the shared data set from a current owner site (that may be receiving a relatively low number of read/write requests) to a new owner site (that may be receiving a relatively high number of read/write requests) to reduce the overall read and write latency experienced in the sites.

Claims

1. A system for changing storage of a shared data set from a current owner storage system to a new owner storage system, each storage system accessing the shared data set, the system comprising:

at least one computer processor;

a memory device;

the current owner storage system configured for storing the shared data set and receiving and performing write requests on the shared data set, the current owner storage system comprising;

a current owner migration module stored in the memory and configured for;

at time t0, beginning transmitting an image of the shared data set to the new owner storage system over a network;

after beginning transmitting the image at time t0, between time t0 and time t1, receiving and performing write requests on the shared data set and producing a change log for each write request that describes a change to the shared data set caused by performance of the write request, time t1 being after time t0;

at time t1, receiving a first message from the new owner storage system, time t1 comprising an approximate point in time that transmission of the image to the new owner storage system is completed;

in response to receiving the first message at time t1, beginning transmitting the change logs, produced between time t0 and time t1, to the new owner storage system over the network;

after beginning transmitting the change logs, between time t1 and time t2, receiving write requests to the shared data set, not performing the received write requests on the shared data set, and producing a record of each received write request, time t2 being after time t1;

at time t2, receiving a second message from the new owner storage system, time t2 comprising an approximate point in time that transmission of the change logs to the new owner storage system is completed; and

in response to receiving the second message at time t2, transmitting the records of write requests, produced between time t1 and time t2, to the new owner storage system over the network.

2. The system of claim 1, further comprising:

the new owner storage system comprising;

a new owner migration module configured for;

beginning receiving and storing the image of the shared data set;

when receiving of the image is complete, sending the first message to the current owner storage system; and

beginning receiving the change logs from the current owner storage system; and

a network connecting the current owner storage system and the new owner storage system.

3. The system of claim 2, wherein:

the new owner migration module is further configured for;

when receiving of the change logs is complete, sending the second message to the current owner storage system.

4. The system of claim 3, wherein the new owner migration module is further configured for:

integrating each change log into the shared data set stored at the new owner storage system, wherein;

a change log for a write request comprises a pointer to changed data in the shared data set stored at the current owner storage system caused by the write request; and

integrating each change log into the shared data set stored at the new owner storage system comprises using the pointer in the change log to retrieve changed data from the current owner storage system and writing the retrieved data into the shared data stored to new owner storage system; and

receiving records of write requests to the shared data set from the current owner storage system; and

performing write requests on the shared data set based on the received records of the write requests.

5. The system of claim 2, further comprising a remote storage system that accesses the shared data set, the remote storage system being connected with the new owner storage system via the network, wherein:

the new owner migration module is further configured for;

when receiving of the image is complete, sending the first message to the remote storage system; and

when receiving of the change logs is complete, sending the second message to the remote storage system; and

the remote storage system comprises a remote migration module configured for;

in response to receiving the first message from the new owner storage system, receiving write requests to the shared data set, not performing the received write requests, and producing a record of each received write request; and

in response to receiving the second message from the new owner storage system, transmitting records of received write requests to the new owner storage system.

6. The system of claim 1, wherein transmitting the image is performed at predetermined time intervals automatically without human initiation or intervention.

7. The system of claim 1, wherein the image of the shared data set comprises a delta image comprising changes to the shared data set since a previous image was transmitted to the new owner storage system.

8. A method for changing storage of a shared data set from a current owner storage system to a new owner storage system, each storage system accessing the shared data set, the method comprising:

at the current owner storage system, storing the shared data set and receiving and performing write requests on the shared data set;

at the current owner storage system, at time t0, beginning transmitting an image of the shared data set to the new owner storage system over a network;

at the current owner storage system, after beginning transmitting the image at time t0, between time t0 and time t1, receiving and performing write requests on the shared data set and producing a change log for each write request that describes a change to the shared data set caused by performance of the write request, time t1 being after time t0;

at the current owner storage system, at time t1, receiving a first message from the new owner storage system, time t1 comprising an approximate point in time that transmission of the image to the new owner storage system is completed;

at the current owner storage system, in response to receiving the first message at time t1, beginning transmitting the change logs, produced between time t0 and time t1, to the new owner storage system over the network;

at the current owner storage system, after beginning transmitting the change logs, between time t1 and time t2, receiving write requests to the shared data set, not performing the received write requests on the shared data set, and producing a record of each received write request, time t2 being after time t1;

at the current owner storage system, at time t2, receiving a second message from the new owner storage system, time t2 comprising an approximate point in time that transmission of the change logs to the new owner storage system is completed; and

at the current owner storage system, in response to receiving the second message at time t2, transmitting the records of write requests, produced between time t1 and time t2, to the new owner storage system over the network.

9. The method of claim 8, further comprising:

at the new owner storage system, beginning receiving and storing the image of the shared data set;

at the new owner storage system, when receiving of the image is complete, sending the first message to the current owner storage system, and at the new owner storage system, beginning receiving the change logs from the current owner storage system.

10. The method of claim 9, further comprising:

at the new owner storage system, when receiving of the change logs is complete, sending the second message to the current owner storage system.

11. The method of claim 10, further comprising:

at the new owner storage system, integrating each change log into the shared data set stored at the new owner storage system;

at the new owner storage system, receiving records of write requests to the shared data set from the current owner storage system; and

at the new owner storage system, performing write requests on the shared data set based on the received records of the write requests.

12. The method of claim 8, wherein transmitting the image is performed at predetermined time intervals automatically without human initiation or intervention.

13. The method of claim 8, wherein the image of the shared data set comprises a delta image comprising changes to the shared data set since a previous image was transmitted to the new owner storage system.

14. The method of claim 8, further comprising:

at the new owner storage system, when receiving of the image is complete, sending the first message to a remote storage system that accesses the shared data set, the remote storage system being connected with the new owner storage system via the network;

at the new owner storage system, when receiving of the change logs is complete, sending the second message to the remote storage system;

at the remote storage system, in response to receiving the first message from the new owner storage system, receiving write requests to the shared data set, not performing the received write requests, and producing a record of each received write request; and

at the remote storage system, in response to receiving the second message from the new owner storage system, transmitting records of received write requests to the new owner storage system.

15. A non-transitory computer readable storage medium having instructions stored thereon when executed by a processor, changes storage of a shared data set from a current owner storage system to a new owner storage system, each storage system accessing the shared data set, the computer readable medium comprising sets of instructions for:

at the current owner storage system, storing the shared data set and receiving and performing write requests on the shared data set;

at the current owner storage system, at time t0, beginning transmitting an image of the shared data set to the new owner storage system over a network;

at the current owner storage system, after beginning transmitting the image at time t0, between time t0 and time t1, receiving and performing write requests on the shared data set and producing a change log for each write request that describes a change to the shared data set caused by performance of the write request, time t1 being after time t0;

at the current owner storage system, at time t1, receiving a first message from the new owner storage system, time t1 comprising an approximate point in time that transmission of the image to the new owner storage system is completed;

at the current owner storage system, in response to receiving the first message at time t1, beginning transmitting the change logs, produced between time t0 and time t1, to the new owner storage system over the network;

at the current owner storage system, after beginning transmitting the change logs, between time t1 and time t2, receiving write requests to the shared data set, not performing the received write requests on the shared data set, and producing a record of each received write request, time t2 being after time t1;

at the current owner storage system, at time t2, receiving a second message from the new owner storage system, time t2 comprising an approximate point in time that transmission of the change logs to the new owner storage system is completed; and

at the current owner storage system, in response to receiving the second message at time t2, transmitting the records of write requests, produced between time t1 and time t2, to the new owner storage system over the network.

16. The non-transitory computer readable storage medium of claim 15, further comprising sets of instructions for:

at the new owner storage system, beginning receiving and storing the image of the shared data set;

at the new owner storage system, when receiving of the image is complete, sending the first message to the current owner storage system, and

at the new owner storage system, beginning receiving the change logs from the current owner storage system.

17. The non-transitory computer readable storage medium of claim 16, further comprising sets of instructions for:

at the new owner storage system, when receiving of the change logs is complete, sending the second message to the current owner storage system.

18. The non-transitory computer readable storage medium of claim 17, further comprising sets of instructions for:

at the new owner storage system, integrating each change log into the shared data set stored at the new owner storage system;

at the new owner storage system, receiving records of write requests to the shared data set from the current owner storage system; and

at the new owner storage system, performing write requests on the shared data set based on the received records of the write requests.

19. The non-transitory computer readable storage medium of claim 15, wherein the set of instructions for transmitting the image is performed at predetermined time intervals automatically without human initiation or intervention.

20. The non-transitory computer readable storage medium of claim 15, wherein the image of the shared data set comprises a delta image comprising changes to the shared data set since a previous image was transmitted to the new owner storage system.

21. The non-transitory computer readable storage medium of claim 15, further comprising sets of instructions for:

at the new owner storage system, when receiving of the image is complete, sending the first message to a remote storage system that accesses the shared data set, the remote storage system being connected with the new owner storage system via the network;

at the new owner storage system, when receiving of the change logs is complete, sending the second message to the remote storage system;

at the remote storage system, in response to receiving the first message from the new owner storage system, receiving write requests to the shared data set, not performing the received write requests, and producing a record of each received write request; and

at the remote storage system, in response to receiving the second message from the new owner storage system, transmitting records of received write requests to the new owner storage system.