Storage space management in union mounted file systems

• US 10,657,102 B2
• Filed: 01/03/2017
• Issued: 05/19/2020
• Est. Priority Date: 01/03/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bhagwat, Deepavali M., Eshel, Marc, Hildebrand, Dean, Naik, Manoj P., Sawdon, Wayne A., Schmuck, Frank B., Tewari, Renu
• Primary Examiner(s): Gofman, Alex
• Assistant Examiner(s): Mian, Umar
• Application Number:
  US15/397,614
• Publication Number:
  US 20180189309A1
• Time in Patent Office: 1,232 Days
• Field of Search: None
• CPC Class Codes: G06F 16/1727 Details of free space management performed by the file system saving storage space on storage systems G06F3/0608; management of blocks in storage devices G06F3/064
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

One embodiment provides a method for re-balancing data and metadata across multiple sub-file systems of a file system. The method includes determining sub-file systems including an amount of data that exceeds a threshold. At least one cell in the sub-file systems is identified as a candidate for re-balancing. A re-balance process is performed on the at least one cell that includes performing a flush operation to flush dirty data from file system buffers, copying an inode table for an independent set of files in the at least one cell to a destination sub-file system, notifying an allocation manager for the destination sub-file system of a new storage pool to manage, and performing an un-quiesce operation on I/O operations to each element in the at least one cell.

Claims

1. A method for re-balancing data and metadata across multiple sub-file systems of a file system, the method comprising:

determining sub-file systems including an amount of data that exceeds a threshold;

identifying at least one cell in the sub-file systems as a candidate for re-balancing;

performing a re-balance process on the at least one cell comprising;

performing a flush operation to flush data from file system buffers;

copying an inode table plus any additional metadata for an independent set of files in the at least one cell to a destination sub-file system;

notifying an allocation manager for the destination sub-file system of a new storage pool to manage; and

performing an un-quiesce operation on I/O operations to each element of the at least one cell; and

upon a request by a particular sub-file system for more inodes without any inodes available, revoking inode numbers from at least one other sub-file system having unused inodes;

wherein each directory in the multiple sub-file systems is named using an inode number of a directory.

2. The method of claim 1, wherein at least one cell comprises logical and physical constructs, and a namespace in a top file system portion of the file system and pointers to each of the multiple sub-file systems include a name of a directory in a sub-file system that is its inode number in the top file system portion.

3. The method of claim 1, wherein a range of inode numbers for each of the sub-file systems is unique, and inodes within the at least one cell are unique across all of the sub-file systems of the at least one cell.

4. The method of claim 1, further comprising:

performing a quiesce operation on input/output (I/O) operations to each element in the at least one cell,

wherein the quiesce operation is initiated to stop new incoming I/O to a set of files of the at least one cell.

5. The method of claim 4, wherein the re-balance process further comprising:

performing another flush operation of data from file system buffers.

6. The method of claim 5, wherein the re-balance process further comprising:

disassociating the set of files is disassociated from a parent sub-file system;

associating the set of files with the destination sub-file system that is smaller in size than at least one sub-file system and has a size to hold the at least one cell; and

upon a determination that a candidate set of files of the at least one cell has a size too large to be migrated to another sub-file system, creating a new sub-file system for the candidate set of files.

7. The method of claim 1, further comprising:

updating pointers or links in a top-file system portion of the file system to a new location of the at least one cell.

8. A computer program product for re-balancing data and metadata across multiple sub-file systems of a file system, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to:

determine, by the processor, sub-file systems including an amount of data that exceeds a threshold;

identify, by the processor, at least one cell in the sub-file systems as a candidate for re-balancing;

perform, by the processor, a re-balance process on the at least one cell comprising;

performing a flush operation to flush data from file system buffers;

copying an inode table plus associated metadata for an independent set of files in the at least one cell to a destination sub-file system;

notifying an allocation manager for the destination sub-file system of a new storage pool to manage; and

performing an un-quiesce operation on I/O operations to each element of the at least one cell; and

upon a request by a particular sub-file system for more inodes without any inodes available, revoking, by the processor, inode numbers from at least one other sub-file system having unused inodes;

wherein each directory in the multiple sub-file systems is named using an inode number of a directory.

9. The computer program product of claim 8, further comprising program instructions executable by the processor to cause the processor to:

request, by the processor, a unique range of inode numbers, wherein the request is made from a top-file system portion or individual sub-file systems of the file system, and a namespace in the top file system portion of the file system and pointers to each of the multiple sub-file systems include a name of a directory in a sub-file system that is its inode number in the top file system portion.

10. The computer program product of claim 8, wherein a range of inode numbers for each of the sub-file systems is unique, and inodes within the at least one cell are unique across all of the sub-file systems of the at least one cell.

11. The computer program product of claim 8, wherein the re-balance process further comprises performing a quiesce operation on input/output (I/O) operations to each element in the at least one cell, and the quiesce is initiated to stop new incoming I/O to a set of files of the at least one cell.

12. The computer program product of claim 11, wherein the re-balance process further comprises:

performing another flush operation of data from file system buffers.

13. The computer program product of claim 12, wherein the re-balance process further comprising:

disassociating the set of files is disassociated from a parent sub-file system;

associating the set of files with the destination sub-file system that is smaller in size than at least one sub-file system and has a size to hold the at least one cell; and

upon a determination that a candidate set of files of the at least one cell has a size too large to be migrated to another sub-file system, creating a new sub-file system for the candidate set of files.

14. The computer program product of claim 8, further comprising program instructions executable by the processor to cause the processor to:

update, by the processor, pointers or links in a top-file system portion of the file system to a new location of the at least one cell.

15. An apparatus comprising:

a memory storing instructions; and

one or more processors executing the instructions to;

determine sub-file systems including an amount of data that exceeds a threshold;

identify at least one cell in the sub-file systems as a candidate for re-balancing;

perform a re-balance process on the at least one cell by the one or more processors executing the instructions to;

perform a flush operation to flush data from file system buffers;

copy an inode table for an independent set of files in the at least one cell to a destination sub-file system;

notify an allocation manager for the destination sub-file system of a new storage pool to manage; and

perform an un-quiesce operation on I/O operations to each element of the at least one cell; and

upon a request by a particular sub-file system for more inodes without any inodes available, revoking inode numbers from at least one other sub-file system having unused inodes;

wherein each directory in the multiple sub-file systems is named using an inode number of a directory.

16. The apparatus of claim 15, wherein the at least one cell comprises logical and physical constructs, and a namespace in a top file system portion of the file system and pointers to each of the sub-file systems include a name of a directory in a sub-file system that is its inode number in the top file system portion.

17. The apparatus of claim 16, wherein a range of inode numbers for each of the sub-file systems is unique, and inodes within the at least one cell are unique across all of the sub-file systems of the at least one cell.

18. The apparatus of claim 15, wherein the re-balance process further comprises the one or more processors further executing the instructions to:

perform a quiesce operation on input/output (I/O) operations to each element in the at least one cell, wherein the quiesce operation is initiated to stop new incoming I/O to a set of files of the at least one cell.

19. The apparatus of claim 18, wherein the re-balance process further comprises the one or more processors further executing the instructions to:

perform another flush operation of data from file system buffers;

disassociate the set of files is disassociated from a parent sub-file system;

associate the set of files with the destination sub-file system that is smaller in size than at least one sub-file system and has a size to hold the at least one cell; and

upon a determination that a candidate set of files of the at least one cell has a size too large to be migrated to another sub-file system, create a new sub-file system for the candidate set of files.

20. The apparatus of claim 15, wherein the one or more processors further executing the instructions to:

update any pointers or links in a top-file system portion of the file system to a new location of the at least one cell.