Systems and methods for providing a distributed file system incorporating a virtual hot spare

• US 20060277432A1
• Filed: 08/11/2006
• Published: 12/07/2006
• Est. Priority Date: 08/03/2001
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Isilon Systems Incorporated (Dell Technologies Inc.)
• Inventors: Schack, Darren P., Passey, Aaron J., Mikesell, Paul A., Patel, Sujal M.
• Granted Patent: US 7,509,524 B2
• Time in Patent Office: Days
• US Class Current: 714/6
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1662 the resynchronized component or unit being a persistent storage device re-synchronization of failed mirror storage G06F11/2082; rebuild or reconstruction of parity RAID storage G06F11/1008
• CPC Class Codes: G06F 11/2056 by mirroring
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 2211/104 Metadata, i.e. metadata associated with RAID systems with parity

Abstract

The intelligent distributed file system enables the storing of file data among a plurality of smart storage units which are accessed as a single file system. The intelligent distributed file system utilizes a metadata data structure to track and manage detailed information about each file, including, for example, the device and block locations of the file'"'"'s data blocks, to permit different levels of replication and/or redundancy within a single file system, to facilitate the change of redundancy parameters, to provide high-level protection for metadata, to replicate and move data in real-time, and to permit the creation of virtual hot spares among the smart storage units without the need to idle any single smart storage unit in the intelligent distributed file system.

Claims

1. A method for providing a virtual hot spare in an intelligent distributed storage system having a plurality of smart storage units, the method comprising:

determining available storage space remaining across a cluster of smart storage units, wherein the cluster comprises at least two of the plurality of smart storage units, and wherein the available storage space is not currently allocated and open to write recovered data and failure protected data;

providing an indication that the cluster is full when the percentage of available storage space is approximately equal to or less than a predetermined level;

upon failure of one of the plurality of smart storage units, dynamically allocating storage space within the available storage space remaining on the cluster for the recovered data, wherein the recovered data is recovered from data stored on the failed one of the plurality of smart storage units; and

wherein the virtual hot spare is the storage space which has not been assigned to any predetermined set of unallocated physical locations within the available free storage space within the smart storage units of the cluster and thereby allows for maximum flexibility for independent physical placement of data within each smart storage unit in the cluster as the new data or recreated data comes in to be stored.

2. A system for storing data comprising:

a plurality of data storage units, each data storage unit having a processor module associated therewith, each of said plurality of said data storage units having sufficient information about where data is stored on all of said plurality of data storage units to respond to and implement write commands on behalf of the system including all of said plurality of data storage units, and the plurality of data storage units having free space which is not allocated and which is available to store at least a portion of an incoming file;

wherein said system is configured to ordinarily store an incoming file across more than one of said plurality of data storage units;

a failure recovery module configured to recreate lost data, said failure recovery module in communication with said plurality of data storage units;

wherein said plurality of data storage units are configured to dynamically adjust the allocation of free space among said plurality of storage units for storing said recreated data.