Automatic load spreading in a clustered network storage system

• US 20080046538A1
• Filed: 08/21/2006
• Published: 02/21/2008
• Est. Priority Date: 08/21/2006
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Inventors: Susarla, Sai Rama Krishna, Eisler, Michael R.
• Granted Patent: US 8,046,422 B2
• Time in Patent Office: Days
• US Class Current: 709/217
• CPC Class Codes: G06F 11/2092 Techniques of failing over between control units
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1019 Random or heuristic server selection
• CPC Class Codes: H04L 67/1023 based on a hash applied to IP addresses or costs
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

A cache is used in a network storage system that includes a plurality of data storage nodes in a storage cluster, to automatically spread read and write access load, by a plurality of storage clients, for file system data and metadata, across the plurality of data storage nodes.

Claims

1. A method comprising:

using a cache, in a network storage system that includes a plurality of data storage nodes in a cluster, to automatically spread read and write access load for file system data and metadata, by a plurality of storage clients, across the plurality of data storage nodes.

2. A method as recited in claim 1, wherein the method comprises:

operating a plurality of caches collectively to automatically spread read and write access load, by the plurality of storage clients, for file system data and metadata, across the plurality of data storage nodes.

3. A method as recited in claim 2, further comprising:

operating a plurality of clustered storage server modules, wherein each of the caches is associated with a separate one of the storage server modules, and wherein each cache caches a copy of data managed by one or more of the storage server modules other than the storage module with which the cache is associated.

4. A method as recited in claim 1, wherein using the cache comprises:

caching a plurality of copies of a dataset in different ones of the data storage nodes;

receiving from a client a data access request relating to the dataset; and

selecting one of the cached copies of the dataset from which to service the request, based on an update frequency of the dataset.

5. A method as recited in claim 4, wherein:

caching a plurality of copies of the dataset comprises

storing a read-only copy of the dataset in at least one of the plurality of data storage nodes; and

storing a read-write copy of the dataset in only one of the plurality of data storage nodes.

6. A method as recited in claim 5, wherein the data access request is a read request, and wherein selecting one of the cached copies of the dataset from which to service the request comprises:

allowing the read request to be serviced from one of the read-only copies if the dataset is not considered to be a frequently updated dataset; and

servicing the read request only from the read-write copy if the dataset is considered to be a frequently updated dataset.

7. A method as recited in claim 1, wherein the cluster contains a plurality of volumes of data, the method further comprising:

selectively enabling or disabling said using a cache to automatically spread read and write access load, on a volume-by-volume basis.

8. A method as recited in claim 1, wherein using the cache comprises write caching data in a volume, including:

creating only one read-write copy of each file of a plurality of individual files of the volume; and

assigning the read-write copy of each file to a particular one of the data storage nodes, such that the read-write copy of two or more files of the volume are assigned to different data storage nodes.

9. A method as recited in claim 8, wherein using the cache further comprises read caching data in the volume, including:

creating at least one read-only copy of at least one file of the volume, and

for at least one file of the volume, assigning a read-only copy of the file to different ones of the data storage nodes.

10. A method as recited in claim 9, wherein using the cache comprises:

creating only one read-write copy of each directory of the volume;

assigning the read-write copy of each directory of the volume to a particular one of the data storage nodes, such that the read-write copy of different directories of the volume are assigned to different data storage nodes;

creating at least one read-only copy of at least one directory of the volume, and

for at least one directory of the volume, assigning a read-only copy of the directory to different ones of the data storage nodes.

11. A method as recited in claim 8, further comprising accessing said read-only copies according to an update frequency of the file.

12. A method as recited in claim 1, further comprising:

caching portions of a file in different ones of the plurality of data storage nodes.

13. A method as recited in claim 1, wherein using the cache comprises:

receiving a write request directed to a dataset stored in the cluster;

determining a hash value based on a dataset identifier associated with the write request; and

determining a particular one of the data storage nodes in which to access the dataset in response to the write request, according to the hash value.

14. A method as recited in claim 13, wherein the plurality of data storage nodes contain only a single writeable copy of the dataset, and said determining a particular one of the data storage nodes in which to access the dataset comprises determining which of the data storage nodes contains the writeable copy of the dataset.

15. A method as recited in claim 1, further comprising:

dividing a volume stored in the network storage system into a first plurality of subsets;

spreading the first plurality of subsets across a first level of caches which includes a plurality of caches;

identifying a particular subset of the first plurality of subsets as being frequently accessed; and

in response to identifying the particular subset as being frequently accessed, dividing the particular subset into a second plurality of subsets and spreading the second plurality of subsets across a second level of caches in the network storage system.

16. A method comprising:

maintaining a plurality of N-modules and a plurality of data storage nodes in a network storage system, each of the data storage nodes including a separate D-module;

maintaining, in each of the of N-modules, a data structure indicating a mapping between D-modules and data identifiers of data stored in the network storage system; and

spreading read and write accesses across the plurality of data storage nodes, including

receiving at one of the N-modules a write request directed to a first dataset;

determining one of the D-modules at which to access the first dataset, based on the mapping and a data identifier indicated by the write request;

directing the write request to said one of the D-modules;

receiving at one of the N-modules a read request directed to a second dataset; and

selecting one of the D-modules at which to access the second dataset, based on an actual or expected update frequency of the second dataset.

17. A method as recited in claim 16, wherein said spreading further comprises:

directing all writes to a particular file to the same D-module, regardless of which N-module receives the write request to said file; and

directing reads of the file to a D-module selected based on an observed update frequency of the particular file.

18. A method as recited in claim 16, further comprising:

selectively enabling or disabling said spreading on a volume-by-volume basis.

19. A network storage system comprising:

a storage cluster including a plurality of data storage nodes;

a plurality of write-back caches, each associated with a separate one of the data storage nodes; and

a plurality of network server modules to receive data access requests from a plurality of clients via a network, and to use the set of caches to automatically spread file system data and metadata access load from the plurality of clients across the plurality of data storage nodes.

20. A network storage system as recited in claim 19, wherein the network storage system stores a volume containing a plurality of individual files, the network storage system maintaining one or more read-only copies of each file of the volume and only one read-write copy of each file of the volume, wherein the one read-write copy of each file is stored in only one of the data storage nodes, and wherein two or more read-only copies of a particular file of the volume are stored in different ones of the plurality of data storage nodes.

21. A network storage system as recited in claim 20, wherein the volume contains a plurality of individual directories, the network storage system maintaining one or more read-only copies of each directory of the volume and only one read-write copy of each directory of the volume, wherein the one read-write copy of each directory is stored in one of the data storage nodes, and wherein two or more read-only copies of a particular directory of the volume are stored in different ones of the plurality of data storage nodes.

22. A network storage system as recited in claim 19, wherein each of the network server modules comprises an N-module and each of the plurality of data storage nodes comprises a D-module.

23. A network storage system as recited in claim 22, wherein each of the N-modules maintains a mapping of data identifiers to D-modules, and each of the N-modules directs write requests that it receives to an appropriate one of the D-modules based on the mapping and a data identifier associated with each of the write requests.

24. A network storage system as recited in claim 22, wherein all write requests to any particular file are directed to the same D-module, regardless of which N-module receives the write request, said D-module containing a writeable copy of the particular file.

25. A network storage system as recited in claim 22, wherein each of the N-modules directs any read requests that it receives for a particular file to a D-module selected based on an update frequency associated with the file.

26. A network storage system as recited in claim 19, wherein two or more of the D-modules are configured as cluster-failover partners.

27. A system comprising:

a plurality of storage nodes, each including a mass storage subsystem and a D-module associated therewith to manage back-end storage functions for the mass storage subsystem, each of the D-modules to implement a cache for primary data stored in one or more of the mass storage subsystems other than the mass storage subsystem with which the D-module is associated; and

a plurality of N-modules, each to receive data access requests via a network from one or more of a plurality of clients, such that each data access request received by any of the N-modules is mapped to a particular one of the D-modules for servicing so as to spread file system data and metadata access load across the plurality of data storage nodes.

28. A system as recited in claim 27, wherein each of the D-modules comprises a cache to store data of which a primary copy is stored in one or more of the mass storage subsystems other than the mass storage subsystem with which the D-module is associated.

29. A system as recited in claim 27, wherein each of the N-modules maintains a mapping of data identifiers to D-modules, and each of the N-modules directs write requests which it receives to an appropriate one of the D-modules based on the mapping and a data identifier in each of the write requests.

30. A system as recited in claim 29, wherein all write requests for any particular file are directed to the same D-module regardless of which N-module receives the write request.

31. A system as recited in claim 30, wherein each of the N-modules directs any read requests that it receives for a particular file to a D-module selected based on an update frequency of the file.

32. A system as recited in claim 27, wherein the plurality of data storage nodes collectively store a volume containing a plurality of individual files, including one or more read-only copies of each file of the volume and only one read-write copy of each file of the volume, wherein the one read-write copy of each file is stored in the cache in one of the data storage nodes, and wherein two or more read-only copies of a particular file of the volume are stored in the caches in different ones of the plurality of data storage nodes.

33. A system as recited in claim 27, wherein two or more of the D-modules are configured as cluster-failover partners.