Reclaiming storage from a file system in a file server

• US 8,566,371 B1
• Filed: 08/15/2011
• Issued: 10/22/2013
• Est. Priority Date: 06/30/2007
• Status: Active Grant

• Current Assignee: EMC Corporation (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Bono, Jean-Pierre, Hayden, John M., Veeraswamy, Sairam, Mullick, Sachin
• Primary Examiner(s): Pham, Khanh
• Assistant Examiner(s): JOHNSON, JOHNESE T
• Application Number:
  US13/210,334
• Time in Patent Office: 799 Days
• Field of Search: 707/822
• US Class Current: 707/822
• CPC Class Codes: G06F 16/119 Details of migration of file systems migration mechanisms in storage systems G06F3/0647
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/185 Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof details of archiving G06F16/11

Abstract

A file server provides access to a file system built on a logical volume of data storage. The file system includes multiple cylinder groups of file system blocks, and the logical volume includes slices of the cylinder groups. The file server has a storage reorganizer for identifying allocated file system blocks in a selected slice, and evacuating the identified allocated file system blocks from the selected slice. The selected slice is evacuated by reverse mapping each identified allocated block to identify a file to which the identified allocated block is allocated, allocating a new block in another slice, copying data from the identified allocated block to the new block, and substituting the new block for the identified allocated block in the file.

Claims

1. A computer-implemented method of operating a file server, the file server including data storage and a data processor, the data storage storing a logical volume and a file system built on the logical volume, the file system including multiple cylinder groups, each of the cylinder groups including file system blocks, the file system blocks including allocated blocks of the data storage allocated to files in the file system, and free blocks of the data storage that are not allocated to the files in the file system, the logical volume including slices of the data storage, and each of the slices of the data storage storing a number of the cylinder groups, the method comprising the data processor executing computer instructions stored on a non-transitory computer-readable storage medium to perform the steps of:

(a) selecting one of the slices of the data storage for evacuation; and

then

(b) evacuating the selected one of the slices of the data storage by identifying allocated blocks of the data storage in the selected one the slices of the data storage, and for each of the identified allocated blocks of the data storage in the selected one the slices of the data storage, (1) reverse mapping said each of the identified allocated blocks to identify a file to which said each of the identified allocated blocks is allocated, (2) allocating a new block of the data storage in another one of the slices of the data storage in the logical volume upon which the file system is built, (3) copying data from said each of the identified allocated blocks to the allocated new block of the data storage in another one of the slices of the data storage in the logical volume upon which the file system is built, and (4) in the identified file to which said each of the identified allocated blocks is allocated, substituting the allocated new block of the data storage for said each of the identified allocated blocks.

2. The method as claimed in claim 1, wherein step (a) further includes stopping new allocations of file system blocks from the cylinder groups in the selected one of the slices of data storage;

and which further includes changing each cylinder group in the selected one of the slices of data storage from a read-only state to an inactive state once said each cylinder group in the selected one of the slices of data storage is empty of allocated file system blocks.

3. The method as claimed in claim 1, which further includes a step (c) of removing the selected one of the slices of the data storage from the logical volume once the selected one of the slices of the data storage is empty of allocated file system blocks.

4. The method as claimed in claim 1, wherein the logical volume is a sparse metavolume including slices of logical storage, and the method further includes maintaining in the data storage a respective state for each of the slices of logical storage, the respective state for each of the slices of logical storage indicating whether said each of the slices of logical storage is provisioned or not with a respective slice of the data storage, and when said each of the slices of logical storage is provisioned with a respective slice of the data storage, also indicating whether or not said each of the slices of logical storage has been selected for evacuation.

5. The computer-implemented method as claimed in claim 1, wherein the substituting of the allocated new block of the data storage for said each of the identified allocated blocks includes substituting a block number of the allocated new block of the data storage for a block number of said each of the identified allocated blocks in an inode or indirect block of the identified file to which said each of the identified allocated blocks is allocated.

6. The computer-implemented method as claimed in claim 1, which includes:

maintaining per-block metadata in the data storage, the per-block metadata indicating, for each of the allocated blocks of the data storage allocated to files in the file system, a list of the files in the file system to which said each of the allocated blocks of the file system has been allocated, and

accessing the per-block metadata during the reverse mapping of said each of the identified allocated blocks to identify the files to which said each of the identified allocated blocks is allocated, and

in each of the identified files to which said each of the identified allocated blocks is allocated, substituting the allocated new blocks of the data storage for said each of the identified allocated blocks.

7. The computer-implemented method as claimed in claim 1, which further includes the file server providing read access and write access to the identified file to which said each of the identified allocated blocks is allocated during the step (b) of evacuating the selected one of the slices.

8. The computer-implemented method as claimed in claim 7, which includes blocking client write access to said each of the identified allocated blocks only when data of said each of the identified allocated blocks is being copied from said each of the identified allocated blocks to the allocated new block of the data storage in another one of the slices of the data storage in the logical volume upon which the file system is built.

9. The computer-implemented method as claimed in claim 8, which includes:

setting an indication, in a block number field of the inode or indirect block of the identified file to which said each of the identified allocated blocks is allocated, when data of said each of the identified allocated blocks is being copied from said each of the identified allocated blocks to the allocated new block of the data storage in another one of the slices of the data storage in the logical volume upon which the file system is built, and

during client write access to said each of the identified allocated blocks, inspecting the indication, in the block number field of the inode or indirect block of the identified file to which said each of the identified allocated blocks is allocated, in order to block client write access to said each of the identified allocated blocks only when data of said each of the identified allocated blocks is being copied from said each of the identified allocated blocks to the allocated new block of the data storage in another one of the slices of the data storage in the logical volume upon which the file system is built.

10. The method as claimed in claim 1, wherein:

the files in the file system include inodes, and some of the inodes are in the selected one of the slices of the data storage, and

the step (b) of evacuating the selected one of the slices of the data storage includes relocating each of the inodes in the selected one of the slices from an original inode number at an original location in the selected one of the slices of the data storage to a new inode number at a new location in the data storage, and recording, in a database in the file server, the new inode number in association with the original inode number, and

the method further includes responding to a request to lookup one of the inodes having a specified inode number by indexing the database with the specified inode number to read a new inode number associated with the specified inode number, and then using the new inode number to lookup said one of the inodes having the specified inode number.

11. A file server comprising:

data storage; and

a data processor coupled to the data storage for accessing data stored in the data storage;

the data storage storing a logical volume and a file system built on the logical volume, the file system including multiple cylinder groups, each of the cylinder groups including file system blocks, the file system blocks including allocated blocks of the data storage allocated to files in the file system, and free blocks of the data storage that are not allocated to the files in the file system, the logical volume including slices of the data storage, and each of the slices of the data storage storing a number of the cylinder groups; and

the data processor having a non-transitory computer-readable storage medium storing computer instructions that, when executed by the data processor, perform the steps of;

(a) selecting one of the slices of the data storage for evacuation; and

then

(b) evacuating the selected one of the slices of the data storage by identifying allocated blocks of the data storage in the selected one the slices of the data storage, and for each of the identified allocated blocks of the data storage in the selected one the slices of the data storage;

(1) reverse mapping said each of the identified allocated blocks to identify a file to which said each of the identified allocated blocks is allocated, (2) allocating a new block of the data storage in another one of the slices of the data storage in the logical volume upon which the file system is built, (3) copying data from said each of the identified allocated blocks to the allocated new block of the data storage in another one of the slices of the data storage in the logical volume upon which the file system is built, and (4) in the identified file to which said each of the identified allocated blocks is allocated, substituting the allocated new block of the data storage for said each of the identified allocated blocks.

12. The file server as claimed in claim 11, wherein step (a) further includes stopping new allocations of file system blocks from the cylinder groups in the selected one of the slices of data storage;

and which further includes changing each cylinder group in the selected one of the slices of data storage from a read-only state to an inactive state once said each cylinder group in the selected one of the slices of data storage is empty of allocated file system blocks.

13. The file server as claimed in claim 11, wherein the computer instructions, when executed by the data processor, further perform a step (c) of removing the selected one of the slices of the data storage from the logical volume once the selected one of the slices of the data storage is empty of allocated file system blocks.

14. The file server as claimed in claim 11, wherein the logical volume is a sparse metavolume including slices of logical storage, and wherein the computer instructions, when executed by the data processor, further maintain in the data storage a respective state for each of the slices of logical storage, the respective state for each of the slices of logical storage indicating whether said each of the slices of logical storage is provisioned or not with a respective slice of the data storage, and when said each of the slices of logical storage is provisioned with a respective slice of the data storage, also indicating whether or not said each of the slices of logical storage has been selected for evacuation.

15. The file server as claimed in claim 11, wherein the substituting of the allocated new block of the data storage for said each of the identified allocated blocks includes substituting a block number of the allocated new block of the data storage for a block number of said each of the identified allocated blocks in an inode or indirect block of the identified file to which said each of the identified allocated blocks is allocated.

16. The file server as claimed in claim 11, wherein the computer instructions, when executed by the data processor, further maintain per-block metadata in the data storage, the per-block metadata indicating, for each of the allocated blocks of the data storage allocated to files in the file system, a list of the files in the file system to which said each of the allocated blocks of the file system has been allocated, and access the per-block metadata during the reverse mapping of said each of the identified allocated blocks to identify the files to which said each of the identified allocated blocks is allocated, and in each of the identified files to which said each of the identified allocated blocks is allocated, substitute the allocated new blocks of the data storage for said each of the identified allocated blocks.

17. The file server as claimed in claim 11, wherein the computer instructions, when executed by the data processor, further provide read access and write access to the identified file to which said each of the identified allocated blocks is allocated during the step (b) of evacuating the selected one of the slices.

18. The file server as claimed in claim 17, wherein the computer instructions, when executed by the data processor, further block client write access to said each of the identified allocated blocks only when data of said each of the identified allocated blocks is being copied from said each of the identified allocated blocks to the allocated new block of the data storage in another one of the slices of the data storage in the logical volume upon which the file system is built.

19. The file server as claimed in claim 18, wherein the computer instructions, when executed by the data processor, further set an indication, in a block number field of the inode or indirect block of the identified file to which said each of the identified allocated blocks is allocated, when data of said each of the identified allocated blocks is being copied from said each of the identified allocated blocks to the allocated new block of the data storage in another one of the slices of the data storage in the logical volume upon which the file system is built, and during client write access to said each of the identified allocated blocks, inspect the indication, in the block number field of the inode or indirect block of the identified file to which said each of the identified allocated blocks is allocated, in order to block client write access to said each of the identified allocated blocks only when data of said each of the identified allocated blocks is being copied from said each of the identified allocated blocks to the allocated new block of the data storage in another one of the slices of the data storage in the logical volume upon which the file system is built.

20. The file server as claimed in claim 11, wherein:

the files in the file system include inodes, and some of the inodes are in the selected one of the slices of the data storage, and

the step (b) of evacuating the selected one of the slices of the data storage includes relocating each of the inodes in the selected one of the slices from an original inode number at an original location in the selected one of the slices of the data storage to a new inode number at a new location in the data storage, and recording, in a database in the file server, the new inode number in association with the original inode number, and

the computer instructions, when executed by the data processor, further respond to a request to lookup one of the inodes having a specified inode number by indexing the database with the specified inode number to read a new inode number associated with the specified inode number, and then using the new inode number to lookup said one of the inodes having the specified inode number.