File system having a hybrid file system format

• US 7,870,172 B1
• Filed: 12/22/2005
• Issued: 01/11/2011
• Est. Priority Date: 12/22/2005
• Status: Expired due to Fees

• Current Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Original Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Inventors: Sarma, Joydeep
• Primary Examiner(s): Jalil; Neveen Abel
• Assistant Examiner(s): MINCEY, JERMAINE A
• Application Number:
  US11/317,940
• Time in Patent Office: 1,846 Days
• Field of Search: None
• US Class Current: 707/822
• CPC Class Codes: G06F 16/17 Details of further file system functions
• CPC Class Codes: G06F 16/1805 Append-only file systems, e.g. using logs or journals to store data

Abstract

Embodiments of this invention disclose a file system having a hybrid file system format. The file system maintains certain data in two formats, thereby defining a hybrid file system format. In one exemplary application, the first format has properties favorable to write operations, e.g. a log-structured file system format, while the second format has properties favorable to read operations, e.g. an extent-based file system format. The data is stored in the first file system format and then asynchronously stored in the second file system format. The data stored in the second file system format are also updated asynchronously.

Claims

1. A system comprising:

a storage subsystem having a plurality of mass storage devices, each of the plurality of mass storage devices having a first contiguous region and a second contiguous region, the storage subsystem including a single hybrid file system to store data in a log-structured file system format, in which a data block is written to a new physical location when it is modified, in the first contiguous region of a mass storage device of the storage subsystem and to store a replica of the data in an extent-based file system format, in which a data block is written to a physical location on which it was previously stored when it is modified, in the second contiguous region of the mass storage device, wherein the hybrid file system assigns a single file block number to the data, the file block number having a mapping to a first volume block number dictated by the log-structured file system format and a mapping to a second volume block number dictated by the extent-based file system format, and wherein the hybrid file system uses the mappings to determine whether to access the first volume block number or the second volume block number to fulfill a read request;

a memory including instructions for a hybrid file system manager; and

a processor, connected with the memory, to execute the instructions for the hybrid file system manager, wherein the instructions cause the processor to store the replica subsequent to changes in the data stored in the log-structured file system format in response to detecting that the data stored in the log-structured file system format exceeds a threshold of potential fragmentation, the instructions further to cause the processor to receive a read request for the data, to make a determination to read the data in the log-structured file system format if the replica is not stored in the extent-based file system format, and to make a determination to read the replica of the data in the extent-based file system format if the replica of the data is stored in the extent-based file system format.

2. The system of claim 1, further comprising the instructions to cause the processor to update the replica in response to a system condition which is independent of operations on the data stored in the log-structured file system format, wherein the system condition comprises at least one of:

idling of a mass storage device storing at least a subset of the replica to be updated, imminent clearing of a volatile memory storing changes to the replica, receiving a request to read a non-updated portion of the replica or reaching a threshold number of operations waiting to update the replica.

3. The system of claim 1, wherein the data comprises a file, a subset of a file, a logical unit number, or a block.

4. The system of claim 1, further comprising a data structure to indicate data blocks to be modified to update the replica.

5. The system of claim 1, wherein the first region is an inner tracks region and the second region is an outer tracks region, and wherein the inner tracks region stores the data in the log-structured file system format and the outer tracks region stores the replica of the data in the extent-based file system format.

6. The system of claim 1, wherein the data stored in the log-structured file system format is parity protected, and wherein the replica of the data stored in the extent-based file system format is not parity protected.

7. The system of claim 1, wherein the first region is an outer tracks region and the second region is an inner tracks region, and wherein the outer tracks region stores the data in the log-structured file system format and the inner tracks region stores the replica of the data in the extent-based file system format.

8. A method of storing data comprising:

receiving data by a storage server that executes a hybrid file system, the hybrid file system having a log-structured file system format, in which a data block is written to a new physical location when it is modified, and an extent-based file system format, in which a data block is written to a physical location on which it was previously stored when it is modified;

assigning, by the storage server, a single file block number to the data, the file block number having a mapping to a first volume block number dictated by the first log-structured file system format and a mapping to a second volume block number dictated by the extent-based file system format;

storing data in the log-structured file system format in a first location associated with the first volume block number, the first location being in a first contiguous region of a mass storage device, and a replica of the data in the extent-based file system format in a second location associated with the second volume block number, the second location being in a second contiguous region of the mass storage device, wherein the extent-based file system format enables faster reads of the replica than the log-structured file system format enables reads of the data, and wherein the replica of the data is stored in the extent-based file system format upon detecting that the data in the log-structured file system format exceeds a threshold potential fragmentation;

asynchronously updating the replica following changes to the data stored in the first log-structured file system format;

receiving a request to read the data;

reading the data in the log-structured file system format if the replica is not stored in the extent-based file system format; and

reading the replica of the data in the extent-based file system format if the replica of the data is stored in the extent-based file system format.

9. The method of claim 8, wherein the threshold potential fragmentation is based on data indicating typical fragmentation of certain types of files.

10. The method of claim 8, wherein determining that the data has a threshold potential fragmentation comprises one or more of analyzing a user-defined data storage policy, determining whether the data comprises a certain type of data, determining whether the data is a database index file, or determining whether the data is an inode file.

11. The method of claim 8, further comprising identifying, in a bitmap, blocks in the second contiguous region to store the replica.

12. The method of claim 11, wherein asynchronously updating the replica following changes to the data stored in the log-structured file system comprises, in response to a system condition which is independent of operations on the data in the log-structured file system format, writing at least a subset of the replica to a subset of the blocks identified in the bitmap.

13. The method of claim 12, wherein the system condition comprises any one or more of idling of a mass storage device mapped to the subset of the blocks, imminent clearing of a volatile memory storing the subset of the replica, receiving a request to read at least a subset of the replica, or reaching a threshold number of operations waiting to write to the second storage space.

14. The method of claim 8, wherein updating the replica comprises processing at least a subset of write operations associated with the updating as a batch job.

15. The method of claim 8, further comprising deleting the replica when the replica reaches a predetermined age.

16. The method of claim 8, wherein the replica of the data in the extent-based file system format has no parity, compression or mirroring data protection.

17. A method of reading data comprising:

receiving a request to read data by a storage server that executes a hybrid file system, wherein the hybrid file system maintains first information in an extent-based file system format, in which a data block is written to a physical location on which it was previously stored when it is modified, in a first virtual block number space in a first contiguous region of a mass storage device and maintains second information in a log-structured file system format, in which a data block is written to a new physical location when it is modified, in a second virtual block number space in a second contiguous region of the mass storage device, wherein the first virtual block number space and the second virtual block number space are both mapped to a single file block number space, and wherein the first information is stored in the extent-based file system format upon detecting that the second information stored in the log-structured file system format exceeds a threshold potential fragmentation;

checking a volatile memory for the data by the storage server;

if the data is in the volatile memory, reading the data from the volatile memory;

if the data is not in the volatile memory, determining if an extent-based file system format version of the data is available in a storage subsystem;

if the extent-based version of the data is available, reading the extent-based version of the data; and

if the extent-based version of the data is not available, reading a log-structured version of the data stored in the storage subsystem.

18. The method of claim 17, wherein determining if the extent-based version is available comprises determining if the data was previously stored in the extent-based file system format, analyzing a bitmap, determining if the extent-based version is up-to-date, determining if the extent-based version is corrupt, determining if devices storing the extent-based version is in a degraded mode, determining if a checksum of the extent-based version is valid, determining if one or more devices storing the extent-based version is busy processing other operations, or a combination thereof.

19. The method of claim 17, further comprising, when the extent-based version of the data is not available, subsequently satisfying the request to read the data using the log-structured version of the data.

20. A method of updating data comprising:

receiving, by a storage server that executes a hybrid file system that maintains data in a log-structured file system format, in which a data block is written to a new physical location when it is modified, in a first contiguous region of a mass storage device on a storage subsystem and maintains a replica of at least a portion of the data in an extent-based file system format, in which a data block is written to a physical location on which it was previously stored when it is modified, in a second contiguous region of the mass storage device on the storage subsystem, an update of data stored in a first storage space of the first contiguous region associated with a first volume block number space in the log-structured file system format;

updating the data in the first storage space;

determining, by the storage server, if a replica of the data is stored in a second storage space of the second contiguous region associated with a second volume block number space in the extent-based file system format, wherein the first volume block number space and the second volume block number space are both mapped to a single file block number space;

when a replica of the data is stored in the second storage space, writing to a data structure to indicate blocks in the second storage space that are to be modified to propagate the update; and

in response to detecting a system condition which is independent of operations on the data stored in the log-structured file system format, modifying at least a subset of the blocks indicated in the data structure.

21. The method of claim 20, wherein modifying at least a subset of the blocks indicated in the data structure comprises performing write operations as a batch job.

22. The method of claim 20, further comprising, marking the blocks as invalid until the block is modified to propagate the update.

23. A storage server comprising:

a network interface through which to communicate with one or more clients over a network;

a storage interface through which to communicate with a mass storage device;

a processor to implement a hybrid file system to store data in the mass storage device, the hybrid file system including a log-structured file system format, in which a data block is written to a new physical location when it is modified, and an extent-based file system format, in which a data block is written to a physical location on which it was previously stored when it is modified; and

a memory storing instructions which, when executed by the processor, cause the server to perform a set of operations comprising;

storing data in the log-structured file system format in a first location in a first contiguous region of the mass storage device, the first location being associated with a first volume block number, wherein the first volume block number is mapped to a file block number;

storing a replica of the data in the extent-based file system format in a second location in a second contiguous region of the mass storage device, the second location being associated with a second volume block number, wherein the second volume block number is also mapped to the file block number, wherein the replica of the data is stored in the extent-based file system format upon determining that the data stored in the log-structured file system format exceeds a threshold potential fragmentation; and

asynchronously updating the replica following changes to the data stored in the first file system.