Asynchronous file replication and migration in a storage network

• US 7,571,168 B2
• Filed: 07/25/2006
• Issued: 08/04/2009
• Est. Priority Date: 07/25/2005
• Status: Active Grant

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Parascale Incorporated (Hitachi, Ltd.)
• Inventors: Bahar, Cameron, Hopfield, Joseph, Nalam, Naveen, Zafman, David B., Oskouy, Rasoul M.
• Primary Examiner(s): Robinson, Greta L
• Assistant Examiner(s): Wilcox, James J
• Application Number:
  US11/459,909
• Publication Number:
  US 20070022122A1
• Time in Patent Office: 1,106 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 2209/508 Monitor
• CPC Class Codes: G06F 9/5011 the resources being hardware resources other than CPUs, Servers and Terminals
• CPC Class Codes: G06F 9/5016 the resource being the memory
• CPC Class Codes: H04L 61/50 Address allocation
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1017 based on a round robin mechanism
• CPC Class Codes: H04L 67/1029 using data related to the state of servers by a load balancer
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/563 Data redirection of data network streams
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

File system independent techniques and mechanisms for replicating files on multiple devices are provided, migrating files from one device to another (for purposes of reliability, increased bandwidth, load balancing, capacity expansion, or reduced cost), and propagating updates from a master copy to remote replicas. The mechanisms involve work queues and asynchronous file migration daemons that operate independently from and in parallel with the primary client-server and network protocol to on-disk storage data paths.

Claims

1. A computer-implemented method of performing remote file operations, the method comprising:

receiving a file task description that identifies a remote file, an operation to be performed on the remote file, and a current status associated with the file task description;

providing a set of persistent work queues associated with a plurality of storage server computers that include one or more file task descriptions;

providing an elected work manager elected by the plurality of storage server computers and configured to accept remote file operation requests and to translate the requests into one or more corresponding tasks to be placed in the set of work queues;

providing a membership manager to determine membership of a control server computer and storage server computers that join or leave a storage system;

providing a background replication agent associated with each storage server computer in the plurality of storage server computers, the background replication agent configured to monitor a work queue in the set of work queues for the associated storage server computer, to accept requests in the work queue from the elected work manager, to perform the accepted tasks using file system independent mechanisms, and to update the completion status of each accepted task, one of the requests including a migration task in which the elected work manager places a remote copy request on a work queue of a first storage server computer and the background replication agent of the first storage server computer performs a remote copy operation and provides a confirmation to the elected work manager, and after receipt of the confirmation, the elected work manager places a remote delete request on a work queue of a second storage server computer and the background replication agent of the second storage server computer performs a remote delete operation and provides a confirmation to the elected work manager; and

generating a first token corresponding to a lease for a file system operation on the remote file, the first token including a first access time and volume information, in response to a first request from the first storage server computer to access the remote file, the first token allowing the first storage server computer to access the remote file based on the first access time, the first storage server computer including a timer set to a value shorter than the first access time, and wherein the first storage server computer determines whether to renew the first token when the timer times out.

2. The method of claim 1 wherein a file task description associated with replication is implemented by performing a remote copy operation with respect to the remote file, awaiting confirmation, and updating a file location map to reflect the existence of a new copy of the remote file.

3. The method of claim 1 wherein a file task description associated with the migration task is implemented by performing the remote copy operation with respect to the remote file, awaiting confirmation, updating a file location map to reflect a new file location for the remote file, and performing the remote delete operation of the original remote file.

4. The method of claim 1 wherein a file task description associated with file update propagation is implemented by performing a remote copy operation to update each existing replica of the remote file.

5. The method of claim 4 wherein a file task description associated with delete propagation is implemented by performing a remote delete operation on each existing replica.

6. A storage system comprising:

a plurality of storage nodes each comprising a storage server configured to store a plurality of files in a file system, wherein a first storage node in the plurality of storage nodes corresponds to a master storage server elected by the plurality of storage servers and including an elected work manager configured to receive a file name that identifies a remote file, an operation to be performed on the remote file, and a current status associated with the operation, and to generate a corresponding task to be placed in a set of work queues, and wherein each storage node includes a background replication agent to communicate with the elected work manager via a work queue in the set of work queues for the associated storage node, to accept tasks in the work queue, to perform the accepted tasks using file system independent mechanisms, and to update the completion status of each accepted task, wherein one of the requests is a migration task in which the elected work manager places a remote copy request on a work queue of a first storage node and the background replication agent of the first storage node performs a remote copy operation and provides a confirmation to the elected work manager, and after receipt of the confirmation, the elected work manager places a remote delete request on a work queue of a second storage node and the background replication agent of the second storage node performs a remote delete operation and provides a confirmation to the elected work manager.

7. The storage system of claim 6, wherein the operation to be performed on the remote file includes replication to be implemented by performing a remote copy operation with respect to the remote file, and wherein the elected work manager is to await confirmation and update a file location map to reflect the existence of a new copy of the remote file.

8. The storage system of claim 6, wherein the operation to be performed on the remote file includes migration to be implemented by performing the remote copy operation, and wherein the elected work manager is to await confirmation, update a file location map to reflect a new file location for the remote file, and perform the remote delete operation of the original remote file.

9. The storage system of claim 6, wherein the operation to be performed on the remote file includes file update propagation to be implemented by performing a remote copy operation to update each existing replica of the remote file.

10. The storage system of claim 9, wherein the operation to be performed on the remote file includes delete propagation to be implemented by performing a remote delete operation on each existing replica.

11. An article comprising a machine-accessible storage medium including instructions that when executed cause a system to:

receive a file task description that identifies a remote file, an operation to be performed on the remote file, and a current status associated with the file task description;

provide a set of persistent work queues associated with a plurality of storage servers that include one or more file task descriptions;

provide an elected work manager elected by the plurality of storage servers and configured to accept remote file operation requests and to translate the requests into one or more corresponding tasks to be placed in the set of work queues;

provide a background replication agent associated with each storage server in the plurality of storage servers, the background replication agent configured to monitor a work queue in the set of work queues for the associated storage server, to accept requests in the work queue from the elected work manager, to perform the accepted tasks using file system independent mechanisms, and to update the completion status of each accepted task, including a migration task in which the elected work manager places a remote copy request on a work queue of a first storage server and the background replication agent of the first storage server performs a remote copy operation and provides a confirmation to the elected work manager, and after receipt of the confirmation, the elected work manager places a remote delete request on a work queue of a second storage server and the background replication agent of the second storage server performs a remote delete operation and provides a confirmation to the elected work manager;

generate a first token corresponding to a lease for a file system operation on the remote file, the first token including a first access time and volume information, in response to a first request from the first storage server to access the remote file, the first token allowing the first storage server to access the file based on the first access time, the first storage server including a timer set to a value shorter than the first access time; and

determine whether to renew the first token when the timer times out.

12. The article of claim 11, further comprising instructions that when executed enable the system to perform a remote copy operation with respect to the remote file, await confirmation, and update a file location map to reflect the existence of a new copy of the remote file.

13. The article of claim 11, further comprising instructions that when executed enable the system to perform a remote copy operation with respect to the remote file, await confirmation, update a file location map to reflect a new file location for the remote file, and perform a remote delete of the original remote file.

14. The article of claim 11, further comprising instructions that when executed enable the system to perform a remote copy operation to update each existing replica of the remote file.

15. The article of claim 14, further comprising instructions that when executed enable the system to perform a remote delete operation on each existing replica.