Methods and apparatus for load balancing storage nodes in a distributed network attached storage system

• US 7,509,645 B2
• Filed: 02/13/2003
• Issued: 03/24/2009
• Est. Priority Date: 10/17/2002
• Status: Expired due to Fees

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Nalam, Naveen, Coates, Joshua L., Landrum, Alfred Gary, Mattis, Peter D., Bozeman, Patrick E., Roselli, Drew S.
• Primary Examiner(s): An; Meng-Ai
• Assistant Examiner(s): Truong; Camquy
• Application Number:
  US10/367,541
• Publication Number:
  US 20040078466A1
• Time in Patent Office: 2,231 Days
• Field of Search: 718/104, 718/105, 709/223, 709/248, 709/226, 707/204, 707/10, 711/111
• US Class Current: 718/105
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 2206/1012 Load balancing
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1012 based on compliance of requirements or conditions with available server resources
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/288 Distributed intermediate devices, i.e. intermediate devices for interaction with other intermediate devices on the same level
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

A distributed data storage system stores a single image file system across a plurality of physical storage volumes. The physical storage may be direct attached storage, or may be coupled through a storage area network (“SAN”). One or more clients communicate with a plurality of storage nodes through a network. A client of the distributed data storage system transmits a request over the network for a file identified in the file system. A load-balancing switch selects one of the storage nodes to process the request. The storage node accesses at least one of the physical volumes and transmits a response for the storage operation to the client.

Claims

1. A distributed data storage system comprising:

a plurality of physical storage volumes for storing files of data identified by a single file system;

a network;

a plurality of storage nodes, coupled to said network, each one of said storage nodes for conducting storage operations including read and write operation on said files of data stored on said physical storage volumes, the plurality of files of data being distributed across the plurality of physical storage volumes by distributing both namespace and data across the plurality of storage nodes, each of the plurality of storage nodes for managing a subset of inode numbers, each inode number identifying a different inode uniquely identifying a different one of the files of data in the single file system, each storage node storing a list of the subset of inode number managed by each of the plurality of storage nodes to provide the single file system across each of the plurality of storage nodes; and

a load balancing switch, coupled to said network and to said storage nodes, for receiving a request from a client to conduct a selected one of the storage operations on an identified one of the plurality of files in said single file system and for selecting one of said storage nodes to process said selected one of the storage operations, the one of said storage nodes to process said selected one of the storage operations also being to perform a mounting of the distributed file system, the mounting including obtaining a respective inode to a root directory and generating a client file handle to the root directory and mount table, wherein said selected node globally and atomically updates mount table of the plurality of storage nodes, the mounting also including transmitting to the client the file handle to the root directory, the respective inode uniquely identifying the root directory.

2. The distributed data storage system as set forth in claim 1, wherein a respective storage node comprises:

a distributed file system manager for processing a plurality of client requests for storage operations; and

a volume manager for accessing said physical storage volumes.

3. The distributed data storage system as set forth in claim 2, wherein said distributed file system manager of a first storage node for communicating to a volume manager of a second storage node to access a desired file stored on a physical volume attached to said second storage node.

4. The distributed data storage system of claim 2, wherein said volume manager to extend to a logical volume comprising a plurality of said physical volumes across said storage nodes.

5. A method for selecting storage nodes in a distributed data storage system, said method comprising the steps of:

storing a plurality of files of data identified by a single file system in a plurality of physical storage volumes;

coupling said physical storage volumes to a plurality of storage nodes;

coupling said storage nodes to a load balancing switch;

coupling said load balancing switch to a network;

receiving a request from a client at said load balancing switch to conduct a selected one of a plurality of storage operations on an identified one of the plurality of files in said plurality of files of data, the plurality of files being distributed across the plurality of physical storage volumes by distributing both namespace and data across the plurality of storage nodes;

managing, by each of the plurality of storage nodes, a subset of inode numbers, each inode number identifying a different inode uniquely identifying a different one of the files of data in the single file system;

storing, in each storage node, a list of the subset of inode numbers managed by each of the plurality of storage nodes to provide said single file system across each of the plurality of storage nodes;

selecting, at said load balancing switch, one of said storage nodes to process said selected one of the storage operations on an identified one of the plurality of files in said single file system and for selecting one of said storage nodes to process said selected one of the storage operations, the one of said storage nodes to process said selected one of the storage operations also being to perform a mounting of the distributed file system, the mounting including obtaining a respective inode to a root directory and generating a client file handle to the root directory and mount table, wherein said selected node globally and atomically updates mount table of the plurality of storage nodes, the mounting also including transmitting to the client the file handle to the root directory, the respective inode uniquely identifying the root directory; and

conducting the plurality of storage operations including read and write operations on said files of said file system stored on said physical storage volumes.

6. The method as set forth in claim 5, wherein the step of conducting the plurality of storage operations on said files of said file single system stored on said physical storage volumes comprise the steps of:

processing a plurality of client requests for storage operations in a distributed file system manager; and

accessing said physical volumes from a volume manager.

7. The method as set forth in claim 6, further comprising the steps of:

communicating from a distributed file system manager of a first storage node to a volume manager of a second storage nodes; and

accessing a respective file stored on a physical volume attached to said second storage node.

8. The method of claim 6, wherein said volume manager to extend a logical volume comprising a plurality of said physical volumes across said storage nodes.

9. A computer readable medium for storing a plurality of instructions, which when executed by a computer system, cause the computer to perform the steps of:

storing a plurality of files of data identified by a singe file system in a plurality of physical storage volumes;

coupling said physical storage volumes to a plurality of storage nodes;

coupling said storage nodes to a load balancing switch;

coupling said load balancing switch to a network;

receiving a request from a client at said load balancing switch to conduct a selected one of a plurality of storage operations on an identified one of the files in said plurality of files of data, the plurality of file of data being distributed across the plurality of physical storage volumes by distributing both namespace and data across the plurality of storage nodes;

managing, by each of the plurality of storage nodes, a subset of inode numbers, each inode number identifying a different inode uniquely identifying a different one of the plurality of files of data in the single file system;

storing, in each storage node, a list of the subset of inode numbers managed by each of the plurality of storage nodes to provide said single file system across each of the plurality of storage nodes;

selecting, at said load balancing switch, one of said storage nodes to process said selected one of the storage operations on an identified one of the plurality of files in said single file system and for selecting one of said storage nodes to process said selected one of the storage operations, the one of said storage nodes to process said selected one of the storage operations also being to perform a mounting of the distributed file system, the mounting including obtaining a respective inode to a root directory and generating a client file handle to the root directory and mount table, wherein said selected node globally and atomically updates mount table of the plurality of storage nodes, the mounting also including transmitting to the client the file handle to the root directory, the respective inode uniquely identifying the root directory; and

conducting the plurality of storage operations including read and write operations on said plurality of files of said file system stored on said physical storage volumes.

10. The computer readable medium as set forth in claim 9, wherein the step of conducting storage operations on said plurality of files of said file single system stored on said physical storage volumes comprises the steps of:

processing said plurality of client requests for storage operations in a distributed file system manager; and

accessing said physical volumes from a volume manager.

11. The computer readable medium as set forth in claim 10, further comprising the steps of:

communicating from distributed file system manager of a first storage node to a volume manager of a second storage node; and

accessing a desired file stored on a physical volume attached to said second storage node.

12. The computer readable medium of claim 10, wherein said volume manger to extend a logical volume comprising a plurality of said physical volumes across said storage nodes.