SYSTEMS AND METHODS FOR A DISTRIBUTED FILE SYSTEM WITH DATA RECOVERY

• US 20080243773A1
• Filed: 06/09/2008
• Published: 10/02/2008
• Est. Priority Date: 08/03/2001
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Isilon Systems Incorporated (Dell Technologies Inc.)
• Inventors: Schack, Darren P., Passey, Aaron J., Mikesell, Paul A., Patel, Sujal M.
• Granted Patent: US 7,962,779 B2
• Time in Patent Office: Days
• US Class Current: 707/2
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1662 the resynchronized component or unit being a persistent storage device re-synchronization of failed mirror storage G06F11/2082; rebuild or reconstruction of parity RAID storage G06F11/1008
• CPC Class Codes: G06F 11/2056 by mirroring
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 2211/104 Metadata, i.e. metadata associated with RAID systems with parity

Abstract

The intelligent distributed file system enables the storing of file data among a plurality of smart storage units which are accessed as a single file system. The intelligent distributed file system utilizes a metadata data structure to track and manage detailed information about each file, including, for example, the device and block locations of the file'"'"'s data blocks, to permit different levels of replication and/or redundancy within a single file system, to facilitate the change of redundancy parameters, to provide high-level protection for metadata, to replicate and move data in real-time, and to permit the creation of virtual hot spares among the smart storage units without the need to idle any single smart storage unit in the intelligent distributed file system.

Claims

1. A distributed file storage system configured to provide data recovery comprising:

a plurality of storage modules in communication via a network, each of the plurality of storage modules configured to process at least one of read and write requests on behalf of the entire distributed storage system;

file metadata that identifies the location of data blocks comprising a file stored on the storage system, wherein the data blocks are distributed across at least some of the plurality of storage modules, and wherein the file metadata is stored on at least two, but not all of the plurality of storage modules; and

a data recovery module configured to reconstruct portions of the file corresponding to data blocks on failed storage modules, wherein the reconstructed data blocks are stored in available data storage locations on the storage modules, and wherein the file metadata is updated to identify the new data storage locations.

2. The distributed file storage system of claim 1, wherein the file is stored in a hierarchical file system.

3. The distributed file storage system of claim 1, further comprising directory metadata configured to reference the file metadata.

4. The distributed file storage system of claim 3, wherein the file metadata is stored on a different storage module than at least a portion of the directory metadata that references the location of the file.

5. The distributed file storage system of claim 1, further comprising an allocation module configured to flexibly allocate the data blocks and the file metadata across the storage modules of the storage system to improve the storage system'"'"'s performance.

6. The distributed file storage system of claim 5, wherein the allocation module further comprises changing the allocation of the data blocks and/or the file metadata across the storage modules of the storage system as the storage and/or retrieval demands change over time.

7. The distributed file storage system of claim 1, wherein the available data storage locations comprise a portion of a virtual hot spare.

8. The distributed file storage system of claim 1, wherein the virtual hot spare includes at least one of the following:

a per stripe virtual hot spare, a per file virtual hot spare, and a per data virtual hot spare.