Scalable clustered storage system

• US 7,574,443 B2
• Filed: 07/25/2006
• Issued: 08/11/2009
• Est. Priority Date: 07/25/2005
• Status: Active Grant

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Parascale Incorporated (Hitachi, Ltd.)
• Inventors: Bahar, Cameron, Hopfield, Joseph, Nalam, Naveen, Zafman, David B., Oskouy, Rasoul
• Primary Examiner(s): Robinson, Greta L
• Assistant Examiner(s): Wilcox, James J
• Application Number:
  US11/459,902
• Publication Number:
  US 20070022087A1
• Time in Patent Office: 1,113 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 2209/508 Monitor
• CPC Class Codes: G06F 9/5011 the resources being hardware resources other than CPUs, Servers and Terminals
• CPC Class Codes: G06F 9/5016 the resource being the memory
• CPC Class Codes: H04L 61/50 Address allocation
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1017 based on a round robin mechanism
• CPC Class Codes: H04L 67/1029 using data related to the state of servers by a load balancer
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/563 Data redirection of data network streams
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

A scalable clustered storage system includes a control node and a plurality of storage nodes. The control node stores object identifiers and location information associated with a plurality of files. The plurality of storage nodes stores the plurality of files. A storage node in the plurality of storage nodes receives a request for a file in a file system from a client. The storage node then communicates with the control node to determine an object identifier and location information associated with the file. The storage node accesses the file in the file system to service the request from the client using the object identifier and location information associated with the file.

Claims

1. A storage system comprising:

a control node comprising a first server including a processor and configured to store object identifiers and location information associated with a plurality of files in a file system, the object identifiers each corresponding to a cluster-wide globally unique identifier used to name and store a file in the storage system and the location information each associated with at least one of a logical and physical location of the file in the file system;

a plurality of storage nodes each comprising a storage server including a processor and configured to store the plurality of files in the file system, wherein a first storage node in the plurality of storage nodes corresponds to a master storage server configured to advertise a virtual IP address for the storage system and to receive a request for a file from a client, to communicate with the control node to determine an object identifier and location information associated with the file, and to access the file from an object storage of the master storage server using the object identifier and location information associated with the file if the object identifier and the location information indicates that the object storage of the master storage server includes the file, and to otherwise request the file from a second storage node based on the object identifier and the location information, to service the request from the client; and

a token manager configured to coordinate access to the file, where the file is stored at a first location of the file system and at a second location of the file system;

wherein the token manager is configured to generate a first token corresponding to a lease for a file system operation on the file, the first token including a first access time and volume information, in response to a first request from a first storage node to access the file, the first token allowing the first storage node to access the file in the first location based on the first access time, the first storage node including a timer set to a value shorter than the first access time, and wherein the first storage node is to determine whether to renew the first token when the timer times out.

2. The storage system of claim 1 wherein the control node is further configured to store namespace and metadata associated with the plurality of files.

3. The storage system of claim 1 wherein the control node is configured to store the object identifiers associated with the plurality of files using a file system at the control node.

4. The storage system of claim 1 wherein the control node is configured to store the location information associated with the plurality of files using a database.

5. The storage system of claim 1 wherein the first storage node is configured to use the object identifier and location information associated with the file to access the file from a remote object storage available on the second storage node in the plurality of storage nodes.

6. The storage system of claim 3 wherein the first storage node is configured to use the object identifier and location information associated with the file to access the file from a portion of the file system located at the first storage node.

7. The storage system of claim 1,

wherein the token manager is configured to generate a second token including a second access time in response to a second request from the second storage node to access the file, the second token allowing the second storage node to access the file in the second location based on the second access time.

8. A computer implemented method of servicing requests for files in a file system of a storage system, the method comprising:

storing object identifiers and location information associated with a plurality of files;

receiving a request for a file from a client in a master storage server of a plurality of storage servers of the file system, the master storage server to receive client requests for the file system via a globally unique virtual IP address advertised by the master storage server;

determining an object identifier and location information associated with the file in a control node of the storage system responsive to a communication from the master storage server to the control node;

accessing the file using the object identifier and location information associated with the file to service the request from the client;

generating a first token corresponding to a lease for a file system operation on the file including a first access time and volume information in response to a first request from a first storage node to access the file, the first token allowing the first storage node to access the file in a first location of the file system based on the first access time; and

generating a second token including a second access time in response to a second request from a second storage node to access the file, the second token allowing the second storage node to access the file in a second location of the file system based on the second access time.

9. The method of claim 8 further comprising storing metadata associated with the plurality of files.

10. The method of claim 8 wherein storing the object identifiers associated with the plurality of files comprises storing the object identifiers using the file system.

11. The method of claim 8 wherein storing the location information associated with the plurality of files comprises storing the location information using a database.

12. The method of claim 8 wherein accessing the file using the object identifier and location information associated with the file to service the request from the client comprises accessing the file by the first storage node from a remote object storage available on the second storage node.

13. The method of claim 8 wherein accessing the file using the object identifier and location information associated with the file to service the request from the client comprises accessing the file from a portion of the file system located at a storage node.

14. The method of claim 8 further comprising determining whether to renew the first token when a timer set at a value shorter than the first access time of the first token times out.

15. A computer readable storage medium comprising instructions that when executed by a computer system having a processor, enable the computer system to:

store metadata for a plurality of files of a file system in a control node of a storage system, the metadata including object identifiers and location information associated with the plurality of files;

receive a request for a file from a client in a master storage server of a plurality of storage servers, the master storage server elected to receive client requests for the plurality of storage servers, wherein the plurality of storage servers store the files of the file system separated from the metadata;

determine based at least in part on statistics of the plurality of storage servers a selected one of the storage servers to handle the client request;

communicate with the control node regarding the client request to determine an object identifier and location information associated with the file;

access the file in one of the plurality of storage servers using the object identifier and location information associated with the file to service the request from the client, wherein the file is stored at a first location of the file system and at a second location of the file system;

generate a first token corresponding to a lease for a file system operation on the file including a first access time and volume information in response to a first request from a first storage node to access the file, the first token allowing the first storage node to access the file in the first location based on the first access time, wherein the first storage node includes a timer set to a value shorter than the first access time of the first token, and is to determine whether to renew the first token when the timer times out;

generate a second token corresponding to a lease for a file system operation on the file including a second access time and volume information in response to a second request from a second storage node to access the file, the second token allowing the second storage node to access the file in the second location based on the second access time; and

wherein the instructions reside on the computer readable storage medium.

16. The computer readable storage medium of claim 15 further comprising instructions that enable the computer system to store metadata associated with the plurality of files.

17. The computer readable storage medium of claim 15 wherein the instructions to store the object identifiers associated with the plurality of files comprises code for storing the object identifiers using the file system.

18. The computer readable storage medium of claim 15 wherein the instructions to store the location information associated with the plurality of files comprises code for storing the location information using a database.

19. The computer readable storage medium of claim 15 wherein the instructions to access the file using the object identifier and location information associated with the file to service the request from the client comprises code for accessing the file by a first storage node from a remote object storage available on the second storage node.

20. The computer readable storage medium of claim 15 wherein the instructions to access the file using the object identifier and location information associated with the file to service the request from the client comprises code for accessing the file from a portion of the file system located at a storage node.