Segregating data and metadata in a file system

• US 9,355,121 B1
• Filed: 06/28/2013
• Issued: 05/31/2016
• Est. Priority Date: 06/28/2013
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Scheer, Michael, Rashid, Ahsan, Kachmar, Maher, Clark, Morgan, Vempati, Srinivasa R., DeSouter, Marc, Sethi, Pranit
• Primary Examiner(s): Alam, Hosain
• Assistant Examiner(s): Ahmed, Saba
• Application Number:
  US13/930,506
• Time in Patent Office: 1,068 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/185 Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof details of archiving G06F16/11

Abstract

A technique for managing a file system includes dividing the addressable space of a file system into multiple subspaces: a data subspace for user data; an inode subspace for inodes describing the user data; and an MDB (metadata block) subspace for indirect blocks and/or directory structures pertaining to the user data. In response to storage requests to write user files to the file system, the file system stores user data of the files in the data subspace, stores inodes pertaining to the user files in the inode subspace, and stores indirect blocks and/or directories in the MDB subspace.

Claims

1. A method of managing a file system having an address space, comprising:

providing multiple subspaces within the address space of the file system, each subspace having defined boundaries within the address space and having provisioned thereto a respective, distinct set of storage elements derived from one or more physical storage devices,

the multiple subspaces including;

a data subspace having provisioned thereto a set of data storage elements on which host data of the file system are stored, but on which no inodes of the file system or MDBs (metadata blocks, including indirect blocks and directories) of the file system are stored;

an inode subspace having provisioned thereto a set of inode storage elements on which inodes of the file system are stored, but on which no host data of the file system or MDBs of the file system are stored; and

an MDB subspace having provisioned thereto a set of MDB storage elements on which indirect blocks and/or directories of the file system are stored, but on which no host data of the file system or inodes of the file system are stored, and

further comprising, for a set of user data;

provisioning a new data storage element to the data subspace of the file system;

storing the set of user data in the new data storage element;

storing a set of inodes pertaining to the set of user data in an inode storage element within the inode subspace of the file system; and

storing a set of indirect blocks and/or directory structures pertaining to the set of user data in an MDB storage element within the MDB subspace of the file system,

wherein the file system is supported by a sparse volume, and wherein provisioning a new data storage element to the data subspace of the file system includes (i) provisioning the new data storage element to the sparse volume from a storage pool and (ii) provisioning the new data storage element to the data subspace of the file system from the sparse volume.

2. The method of claim 1, further comprising:

establishing an allocation unit for each subspace of the file system, each allocation unit specifying a minimum addressable amount of storage in the respective subspace,

wherein, upon establishing the allocation unit for each subspace, the allocation unit for the data subspace is made larger than the allocation unit for the inode subspace and the allocation unit for the data subspace is made larger than the allocation unit for the MDB subspace.

3. The method of claim 1, wherein the data subspace stores no metadata of the file system.

4. The method of claim 1, wherein providing multiple subspaces further includes providing a VBM (virtual block map) subspace having provisioned thereto a set of VBM storage elements on which VBMs of the file system are stored, but on which no user data of the file system, inodes of the file system, or MDBs of the file system are stored.

5. The method of claim 1, wherein providing multiple subspaces further includes providing at least one additional subspace for storing some other type of metadata not already stored in the other subspaces.

6. The method of claim 1, further comprising:

receiving a request to provision a new storage element to a first subspace of the multiple subspaces of the file system;

accessing a policy that specifies tiers of storage to be used for provisioning storage elements to different subspaces of the file system, each tier of storage providing a different quality of service;

in response to the request to provision the new storage element and in accordance with the accessed policy, selecting the new storage element from a tier of storage that the policy specifies for the first subspace of the file system; and

provisioning the selected storage element to the first subspace.

7. The method of claim 6, further comprising:

receiving a request to provision a new storage element to a second subspace of the multiple subspaces of the file system;

in response to the request to provision a new storage element to the second subspace and in accordance with the policy, selecting the new storage element for the second subspace from a tier of storage that the policy specifies for the second subspace of the file system; and

provisioning the selected new storage element for the second subspace to the second subspace,

wherein the new storage element provisioned to the second subspace provides a higher quality of service than the new storage element provisioned to the first subspace.

8. The method of claim 7, wherein the first subspace is the data subspace and the second subspace is the inode subspace.

9. The method of claim 4, wherein each of the provisioned storage elements of the inode subspace, the MDB subspace, and the VBM subspace includes multiple descriptors that describe the contents of the respective storage element.

10. The method of claim 9,

wherein the data storage elements of the data subspace include no descriptors that describe the content of the respective storage elements, and

wherein providing multiple subspaces within the address space of the file system further includes providing a shadow subspace, the shadow subspace including multiple shadow storage elements, each shadow storage element storing multiple descriptors describing the contents of at least one storage element of the data subspace.

11. The method of claim 10, wherein each shadow storage element in the shadow subspace stores multiple descriptors describing the contents of multiple data storage elements of the data subspace.

12. The method of claim 11, further comprising, in response to receiving a request to store new host data:

creating a new descriptor describing the contents of the new host data; and

storing the new descriptor in a shadow storage element of the shadow subspace at a location in the shadow subspace algorithmically determined based on a location of the new host data in the data subspace.

13. The method of claim 12, further comprising, in response to receiving a request to delete host data:

algorithmically determining a location of a descriptor of the host data to be deleted in the shadow subspace based on a location of the host data to be deleted in the data subspace; and

marking as deleted the host data to be deleted in the descriptor of the host data to be deleted in the shadow subspace.

14. The method of claim 13, wherein the multiple descriptors describing the contents of multiple data storage elements of the data subspace include CG headers and BMDs, wherein the CG headers include bitmaps of free and allocated resources in the data subspace, and wherein the BMDs include metadata describing the contents of data storage elements in the data subspace.

15. The method of claim 2, further comprising maintaining a count of free resources in each of the multiple subspaces.

16. The method of claim 15, further comprising requesting a new storage element for any of the subspaces in response to the number of free resources in that subspace falling below a predetermined minimum level.

17. The method of claim 15, wherein maintaining a count of free resources in each of the multiple subspaces includes:

maintaining a count of free inodes in the inode subspace;

maintaining a count of free data allocation units in the data subspace; and

maintaining a count of free MDB blocks in the MDB subspace.

21. The method of claim 1, further comprising provisioning another new data storage element to the inode subspace of the file system, wherein provisioning the other new data storage element to the inode subspace of the file system includes (i) provisioning the other new data storage element to the sparse volume from the storage pool and (ii) provisioning the other new data storage element to the inode subspace of the file system from the sparse volume.

18. A data storage apparatus, comprising:

a set of storage processors configured to receive requests from a set of host computing devices for reading and/or writing files; and

a set of storage devices, coupled to the set of storage processors and configured to store files accessible to the set of hosts via the set of storage processors,

wherein the set of storage processors includes a set of processing units coupled to memory, the memory storing executable instructions, which when executed by the set of processing units cause the set of processing units to perform a method of managing a file system, the method comprising;

providing multiple subspaces within an address space of the file system, each subspace having defined boundaries within the address space and having provisioned thereto a respective, distinct set of storage elements derived from one or more physical storage devices,

the multiple subspaces including;

a data subspace having provisioned thereto a set of data storage elements on which host data of the file system are stored, but on which no inodes of the file system or MDBs (metadata blocks, including indirect blocks and directories) of the file system are stored;

an inode subspace having provisioned thereto a set of inode storage elements on which inodes of the file system are stored, but on which no host data of the file system or MDBs of the file system are stored; and

an MDB subspace having provisioned thereto a set of MDB storage elements on which indirect blocks and directories of the file system are stored, but on which no host data of the file system or inodes of the file system are stored, and

further comprising, for a set of user data;

provisioning a new data storage element to the data subspace of the file system;

storing the set of user data on the new data storage element;

storing a set of inodes pertaining to the set of user data in an inode storage element within the inode subspace of the file system; and

storing a set of indirect blocks and/or directory structures pertaining to the set of user data in an MDB storage element within the MDB subspace of the file system,

wherein the file system is supported by a sparse volume, and wherein provisioning a new data storage element to the data subspace of the file system includes (i) provisioning the new data storage element to the sparse volume from a storage pool and (ii) provisioning the new data storage element to the data subspace of the file system from the sparse volume.

19. The data storage apparatus of claim 18, wherein the method further comprises:

receiving a request to provision a new storage element to a first subspace of the multiple subspaces of the file system;

accessing a policy that specifies tiers of storage to be used for provisioning storage elements to different subspaces of the file system, each tier of storage providing a different quality of service;

in response to the request to provision the new storage element and in accordance with the accessed policy, selecting the new storage element from a tier of storage that the policy specifies for the first subspace of the file system;

provisioning the selected storage element to the first subspace;

receiving a request to provision a second new storage element of storage to a second subspace of the multiple subspaces of the file system;

in response to the request to provision the second new storage element of storage and in accordance with the policy, selecting the second new storage element from a tier of storage that the policy specifies for the second subspace of the file system; and

provisioning the selected second new storage element to the second subspace,

wherein the new storage element provisioned to the second subspace provides a higher quality of service than the new storage element provisioned to the first subspace.

20. A non-transitory computer-readable medium including instructions which, when executed by a set of processing units of a set of storage processors of a data storage system, cause the set of processing units to perform a method of managing a file system, the method comprising:

providing multiple subspaces within an address space of the file system, each subspace having defined boundaries within the address space and having provisioned thereto a respective, distinct set of storage elements derived from one or more physical storage devices,

the multiple subspaces including;

a data subspace having provisioned thereto a set of data storage elements on which host data of the file system are stored, but on which no inodes of the file system or MDBs (metadata blocks, including indirect blocks and directories) of the file system are stored;

an inode subspace having provisioned thereto a set of inode storage elements on which inodes of the file system are stored, but on which no host data of the file system or MDBs of the file system are stored; and

an MDB subspace having provisioned thereto a set of MDB storage elements on which indirect blocks and directories of the file system are stored, but on which no host data of the file system or inodes of the file system are stored, and

further comprising, for a set of user data;

provisioning a new data storage element to the data subspace of the file system;

storing a set of user data on the new data storage element;

storing a set of inodes pertaining to the set of user data in an inode storage element within the inode subspace of the file system; and

storing a set of indirect blocks and/or directory structures pertaining to the set of user data in an MDB storage element within the MDB subspace of the file system,

wherein the file system is supported by a sparse volume, and wherein provisioning a new data storage element to the data subspace of the file system includes (i) provisioning the new data storage element to the sparse volume from a storage pool and (ii) provisioning the new data storage element to the data subspace of the file system from the sparse volume.