Networked storage system employing information lifecycle management in conjunction with a distributed global file system

• US 7,711,711 B1
• Filed: 03/29/2006
• Issued: 05/04/2010
• Est. Priority Date: 03/29/2006
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: D'Orazio, Daniel, Linnell, J. Andrew
• Primary Examiner(s): Ali; Mohammad
• Assistant Examiner(s): LAWSON, DONALD L
• Application Number:
  US11/393,185
• Time in Patent Office: 1,497 Days
• Field of Search: 707/204, 707/10, 709/219, 709/223, 714/7
• US Class Current: 707/652
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/0649 Lifecycle management
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A networked storage system includes network-attached file system (FS) nodes implementing a distributed global file system. Each FS node includes a file system server and at least one storage system including storage resources (such as disk drives) representing portions of a global file system storage space. The FS nodes are organized according to respective cost-performance characteristics of the storage systems, generally ranging from a high-cost, high-performance characteristic to a low-cost, low-performance characteristic. A storage management system performs information lifecycle management (ILM) which includes allocating the FS nodes for storing data according to a mapping between an ILM-based data classification scheme and the cost-performance characteristics of the storage devices, as well as dynamically managing the placement of data among the storage devices according to the ILM-based classification of the data. The storage management system also performs global file system management including allocating a set of identifiers (e.g. segment identifiers) of the distributed global file system among the FS nodes, and dynamically adjusting the allocation of the identifiers among the FS nodes in response to the addition and removal of whole storage systems and/or storage resources within each of the storage systems.

Claims

1. A networked storage system, comprising:

a plurality of network-attached file system (FS) nodes each including a file system server and at least one storage system coupled to the file system server, each storage system including a storage controller having a respective processor and one or more data storage devices, the FS nodes being arranged into cost-performance groups ranging from a first cost-performance group of FS nodes providing high performance at high cost to a second cost-performance group of FS nodes providing low performance at low cost; and

a storage management system coupled to the FS nodes, the storage management system having a processor and being operative in conjunction with the FS nodes to;

define a distributed global file system across the FS nodes, the distributed global file system being divided into segments for storing data files and having a single namespace of segment identifiers for the segments as visible from host computers utilizing the networked storage system, the segment identifiers lacking any identification of the FS nodes;

monitor frequency of access of the data files and based on the monitoring assign the data files to respective information lifecycle management (ILM) classes, the ILM classes ranging from an active class for data files having a high and recent access frequency to an inactive class for data files having a low and less recent access frequency; and

assigning the FS nodes to store respective segments of the distributed global file system according to a mapping between the cost-performance characteristics of the FS nodes and the ILM classes of the data files contained in the segments, such that segments containing data files assigned to the active class are stored in the FS nodes of the first cost-performance group and segments containing data files assigned to the inactive class are stored in the FS nodes of the second cost-performance group,

wherein the active and inactive ILM classes are major ILM classes and each includes a plurality of sub-classes, the sub-classes of the active class including active-high and active-low sub-classes distinguished by relatively higher and lower access frequencies, and the sub-classes of the inactive class including inactive-recent and inactive-older sub-classes distinguished by an amount of time that data files have been assigned to the respective sub-classes.

2. A networked storage system according to claim 1, wherein the storage management system and FS nodes are further operative to dynamically adjust the allocation of the segment identifiers among the FS nodes to achieve balanced loading of the FS nodes.

3. A networked storage system according to claim 2, wherein the storage management system and FS nodes employ a common set of mechanisms for dynamically adjusting the allocation of the segment identifiers among the FS nodes for both load-balancing purposes and ILM-related placement of data.

4. A networked storage system according to claim 1, wherein the storage devices include at least one integrated cached disk array (ICDA) having the high-cost, high-performance characteristic and at least one archival storage system having the low-cost, low-performance characteristic.

5. A networked storage system according to claim 1, wherein each FS node is physically packaged as (1) a rack-mountable or blade type of server and (2) one or more rack-mountable enclosures each containing a corresponding storage system of the FS node.

6. A networked storage system according to claim 1, wherein the inactive classification includes an archival classification for end-of-lifetime data, and the corresponding relatively low cost-performance storage devices include archival storage devices having high storage density, slow access times, and low cost per unit of storage.

7. A networked storage system according to claim 1, wherein the segment identifiers are consecutively ordered alphabetic or numeric values.

8. A method of operating a networked storage system including a plurality of network-attached file system (FS) nodes, each FS node including a file system server and at least one storage system coupled to the file system server, each storage system including a storage controller having a processor and one or more data storage devices, the method comprising:

arranging the FS nodes into cost-performance groups ranging from a first cost-performance group of FS nodes providing high performance at high cost to a second cost-performance group of FS nodes providing low performance at low cost; and

in a storage management system in conjunction with the FS nodes, the storage management system having a processor;

defining a distributed global file system across the FS nodes, the distributed global file system being divided into segments for storing data files and having a single namespace of segment identifiers for the segments as visible from host computers utilizing the networked storage system, the segment identifiers lacking any identification of the FS nodes;

monitoring frequency of access of the data files and based on the monitoring assigning the data files to respective information lifecycle management (ILM) classes, the ILM classes ranging from an active-high class for data files having a high and recent frequency of access to an inactive-low class for data files having a low and less recent frequency of access; and

assigning the FS nodes to store respective segments of the distributed global file system according to a mapping between the cost-performance characteristics of the FS nodes and the ILM classes of the data files contained in the segments, such that segments containing data files assigned to the active-high class are stored in the FS nodes of the first cost-performance group and segments containing data files assigned to the inactive-low class are stored in the FS nodes of the second cost-performance group,

wherein the active and inactive ILM classes are major ILM classes and each includes a plurality of sub-classes, the sub-classes of the active class including active-high and active-low sub-classes distinguished by relatively higher and lower access frequencies, and the sub-classes of the inactive class including inactive-recent and inactive-older sub-classes distinguished by an amount of time that data files have been assigned to the respective sub-classes.

9. A method according to claim 8, further comprising dynamically adjusting the allocation of the segment identifiers among the FS nodes to achieve balanced loading of the FS nodes.

10. A method according to claim 9, further comprising employing a common set of mechanisms in the storage management system and FS nodes for dynamically adjusting the allocation of the segment identifiers among the FS nodes for both load-balancing purposes and ILM-related placement of data.

11. A method according to claim 8, wherein the inactive classification includes an archival classification for end-of-lifetime data, and the corresponding relatively low cost-performance storage devices include archival storage devices having high storage density, slow access times, and low cost per unit of storage.

12. A method according to claim 8, wherein the segment identifiers are consecutively ordered alphabetic or numeric values.