Technique to scale out namespace

• US 10,152,481 B1
• Filed: 12/03/2015
• Issued: 12/11/2018
• Est. Priority Date: 12/03/2015
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Singh, Pranay
• Primary Examiner(s): Vo, Truong V
• Application Number:
  US14/958,469
• Time in Patent Office: 1,104 Days
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/128 Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion use of snapshots for error detection or correction G06F11/14, G06F11/16
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/182 Distributed file systems

Abstract

A first data is stored in a first filesystem data storage, wherein the first filesystem data storage is part of a first cluster node that comprises a common filesystem metadata. A second data is stored in a second filesystem data storage, wherein the second filesystem data storage is part of a second cluster node coupled with the first cluster node that comprises the common filesystem metadata. The first cluster node and the second cluster node are associated with a cluster for shared storage.

Claims

1. A system, comprising:

a first computer cluster node including a first filesystem data storage system;

a second computer cluster node coupled with the first computer cluster node and including a second filesystem data storage system;

a common filesystem metadata that is configured to be shared by the first computer cluster node and the second computer cluster node;

wherein the first computer cluster node and the second computer cluster node are associated with a cluster for shared storage;

wherein the common filesystem metadata is structured as a tree having a root node, a plurality of intermediate nodes, a plurality of leaf nodes, wherein the root node and each intermediate node are replicated across the first computer cluster node and the second computer cluster node and stored in their respective filesystem data storage system, wherein each of the plurality of leaf nodes resides only on one computer cluster node;

wherein the tree includes one or more subtrees, with a top level of the tree maintained on a parent computer node, which determines whether each of the one or more subtrees is migrated from one node to another node based at least in part on a migration policy, wherein the migration policy tends toward a self-balancing tree.

2. The system of claim 1, wherein the common filesystem metadata offers a dynamic partitioning of a global namespace for the cluster.

3. The system of claim 1, wherein the tree is a B+tree.

4. The system of claim 3, wherein the B+tree has a depth of three levels.

5. The system of claim 3, wherein the B+tree is constrained such that a directory cannot be split across two or more cluster nodes.

6. The system of claim 1, wherein the common filesystem metadata and each filesystem data storage system is associated with a filesystem.

7. The system of claim 6, wherein the common filesystem metadata comprises a file manager layer for the filesystem.

8. The system of claim 6, wherein the common filesystem metadata comprises a directory manager layer for the filesystem.

9. The system of claim 6, where each filesystem data storage system comprises one or more of the following for the filesystem:

a content store, a segment store, an index layer, a container manager layer, a storage layer, and a RAID layer.

10. The system of claim 6, wherein the filesystem is a deduplicating filesystem.

11. The system of claim 6, wherein the first cluster node is a parent computer node for the filesystem.

12. The system of claim 11, wherein an inode is stored in a leaf level of the tree.

13. The system of claim 1, wherein the tree is one of a plurality of trees for a common library.

14. The system of claim 13, wherein the tree has an associated snapshot tree for a common library.

15. The system of claim 1, wherein the parent computer node includes a plurality of top nodes.

16. The system of claim 1, when a number of intermediate nodes on either the first computer cluster node or the second computer cluster node exceeds a threshold, the intermediate nodes on the computer cluster node are moved to another computer cluster node.

17. The system of claim 1, wherein when a directory in the tree spans two more intermediate nodes, the two or more intermediate nodes are to be placed in a same computer cluster node.

18. A method, comprising:

storing a first data in a first filesystem data storage system, wherein the first filesystem data storage system is part of a first computer cluster node that shares a common filesystem metadata;

storing a second data in a second filesystem data storage system, wherein the second filesystem data storage system is part of a second computer cluster node coupled with the first cluster node that shares the common filesystem metadata; and

wherein the first computer cluster node and the second computer cluster node are associated with a cluster for shared storage, wherein the common filesystem metadata is structured as a tree having a root node, a plurality of intermediate nodes, a plurality of leaf nodes, wherein the root node and each intermediate node are replicated across the first computer cluster node and the second computer duster node and stored in their respective filesystem data storage system, wherein each of the plurality of leaf nodes resides only on one computer cluster node;

wherein the tree includes one or more subtrees, with a top level of the tree maintained on a parent computer node, which determines whether each of the one or more subtrees is migrated from one node to another node based at least in part on a migration policy, wherein the migration policy tends toward a self-balancing tree.

19. A computer program product, the computer program product being embodied in a tangible computer readable storage medium and comprising computer instructions for:

storing a first data in a first filesystem data storage system, wherein the first filesystem data storage system is part of a first computer cluster node that shares a common filesystem metadata;

storing a second data in a second filesystem data storage system, wherein the second filesystem data storage system is part of a second computer cluster node coupled with the first cluster node that shares the common filesystem metadata;

wherein the first computer cluster node and the second computer cluster node are associated with a cluster for shared storage, wherein the common filesystem metadata is structured as a tree having a root node, a plurality of intermediate nodes, a plurality of leaf nodes, wherein the root node and each intermediate node are replicated across the first computer cluster node and the second computer cluster node and stored in their respective filesystem data storage system, wherein each of the plurality of leaf nodes resides only on one computer cluster node;

wherein the tree includes one or more subtrees, with a top level of the tree maintained on a parent computer node, which determines whether each of the one or more subtrees is migrated from one node to another node based at least in part on a migration policy, wherein the migration policy tends toward a self-balancing tree.