METHOD FOR DIRECTORY ENTRIES SPLIT AND MERGE IN DISTRIBUTED FILE SYSTEM
First Claim
1. A plurality of MDSs (metadata servers) in a distributed storage system which includes data servers storing file contents to be accessed by clients, each MDS having a processor and a memory and storing file system metadata to be accessed by the clients,wherein directories of a file system namespace are distributed to the MDSs based on a hash value of inode number of each directory, each directory is managed by a MDS as a master MDS of the directory, and a master MDS may manage one or more directories;
- wherein when a directory grows with a high file creation rate that is greater than a preset split threshold, the master MDS of the directory constructs a consistent hashing overlay with one or more MDSs as slave MDSs and splits directory entries of the directory to the consistent hashing overlay based on hash values of file names under the directory;
wherein the consistent hashing overlay has a number of MDSs including the master MDS and the one or more slave MDSs, the number being calculated based on the file creation rate;
wherein when the directory continues growing with a file creation rate that is greater than the preset split threshold, the master MDS adds a slave MDS into the consistent hashing overlay and splits directory entries of the directory to the consistent hashing overlay with the added slave MDS based on hash values of file names under the directory.
1 Assignment
0 Petitions
Accused Products
Abstract
A distributed storage system has MDSs (metadata servers). Directories of file system namespace are distributed to the MDSs based on hash value of inode number of each directory. Each directory is managed by a master MDS. When a directory grows with a file creation rate greater than a preset split threshold, the master MDS constructs a consistent hashing overlay with one or more slave MDSs and splits directory entries of the directory to the consistent hashing overlay based on hash values of file names under the directory. The number of MDSs in the consistent hashing overlay is calculated based on the file creation rate. When the directory continues growing with a file creation rate that is greater than the preset split threshold, the master MDS adds a slave MDS into the consistent hashing overlay and splits directory entries to the consistent hashing overlay based on hash values of file names.
-
Citations
20 Claims
-
1. A plurality of MDSs (metadata servers) in a distributed storage system which includes data servers storing file contents to be accessed by clients, each MDS having a processor and a memory and storing file system metadata to be accessed by the clients,
wherein directories of a file system namespace are distributed to the MDSs based on a hash value of inode number of each directory, each directory is managed by a MDS as a master MDS of the directory, and a master MDS may manage one or more directories; -
wherein when a directory grows with a high file creation rate that is greater than a preset split threshold, the master MDS of the directory constructs a consistent hashing overlay with one or more MDSs as slave MDSs and splits directory entries of the directory to the consistent hashing overlay based on hash values of file names under the directory; wherein the consistent hashing overlay has a number of MDSs including the master MDS and the one or more slave MDSs, the number being calculated based on the file creation rate; wherein when the directory continues growing with a file creation rate that is greater than the preset split threshold, the master MDS adds a slave MDS into the consistent hashing overlay and splits directory entries of the directory to the consistent hashing overlay with the added slave MDS based on hash values of file names under the directory. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)
-
-
15. A method of distributing directory entries to a plurality of MDSs (metadata servers) in a distributed storage system which includes clients and data servers storing file contents to be accessed by the clients, each MDS storing file system metadata to be accessed by the clients, the method comprising:
-
distributing directories of a file system namespace to the MDSs based on a hash value of inode number of each directory, each directory being managed by a MDS as a master MDS of the directory, wherein a master MDS may manage one or more directories; when a directory grows with a high file creation rate that is greater than a preset split threshold, constructing a consistent hashing overlay with one or more MDSs as slave MDSs and splits directory entries of the directory to the consistent hashing overlay based on hash values of file names under the directory, wherein the consistent hashing overlay has a number of MDSs including the master MDS and the one or more slave MDSs, the number being calculated based on the file creation rate; and when the directory continues growing with a file creation rate that is greater than the preset split threshold, adding a slave MDS into the consistent hashing overlay and splits directory entries of the directory to the consistent hashing overlay with the added slave MDS based on hash values of file names under the directory. - View Dependent Claims (16, 17, 18, 19, 20)
-
Specification