Building a meta file system from file system cells

• US 20020138502A1
• Filed: 03/20/2001
• Published: 09/26/2002
• Est. Priority Date: 03/20/2001
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Gupta, Uday K.
• Granted Patent: US 6,775,679 B2
• Time in Patent Office: Days
• US Class Current: 707/200
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface

Abstract

File system cells are linked together to form a meta file system that appears to a user or application program to be a single file system. Each file system cell may have a conventional file system format, and can be indistinguishable from a conventional file system except for information, such as directory entry attributes, indicating one or more links to other file system cells. These external links may include direct links that are hidden from the user or application program and define a hierarchy of the meta file system cells, and indirect links that appear in the user-visible file system. The meta file system substantially reduces crash recovery time because each file system cell functions as a consistency unit that can be repaired only if needed. The meta file system also permits the file system cells to be accessed concurrently by multiple processors in a file server.

Claims

1. A method of building a meta file system, said method comprising:

selecting a file subsystem in a first file system cell;

removing the selected file subsystem from the first file system cell, converting the selected file subsystem into a second file system cell, and inserting a link in the first file system cell from a directory entry for the selected file subsystem to a root directory of the second file system cell.

2. The method as claimed in claim 1, wherein the converting of the selected file subsystem into the second file system cell includes adding file system metadata to the selected file subsystem.

3. The method as claimed in claim 1, which includes creating a hierarchy of multiple file system cells in such a way that the hierarchy appears to a user or application program as a single file system.

4. The method as claimed in claim 1, which includes hiding the link from view of a user or application program so that the combination of the first file system cell and the second file system cell appears to the user or application program to be one file system.

5. The method as claimed in claim 1, which includes providing the directory entry for the selected file with an attribute indicating that the entry is for an external link.

6. The method as claimed in claim 1, wherein the link is inserted by inserting a link file in the first file system cell, the link file including information for locating the second file system cell.

7. The method as claimed in claim 1, wherein the first file system cell includes a link between a portion of the first file system cell outside the selected file subsystem to the inside of the selected file subsystem, and the method further includes providing the link with a target specification that does not change when the method of claim 1 is repeated upon the file system cells.

8. The method as claimed in claim 1, wherein the first file system cell includes a link between a portion of the first file system cell outside the selected file subsystem to the inside of the selected file subsystem, and the method further includes providing a user or application with a view of objects in the first and second file system cells that does not change from before and after performing the method of claim 1, and providing the link with a target specification which includes a name of a path in the view of objects in the first and second file system cells.

9. The method as claimed in claim 1, wherein the method includes accessing each file system cell by checking whether said each file system cell has a meta file system attribute indicating that the first file system cell has at least one external link to another file system cell, and when said each file system cell does not have a meta file system attribute, invoking at least one file system manager routine that does not recognize external links, and when said each file system does have a meta file system attribute, invoking at least one file system manager routine that recognizes external links.

10. The method as claimed in claim 1, which includes recovering from a system crash by checking whether each file system cell has a file system attribute indicating whether or not said each file system cell needs to be checked for consistency after the system crash.

11. The method of claim 1, which further includes operating a first processor to access the first file system cell to check the first file system cell for consistency after a system crash and concurrently operating a second processor to access the second file system cell to check the second file system cell for consistency after the system crash.

12. The method of claim 1, which further includes operating a first processor to manage user or application program access to the first file system cell and concurrently operating a second processor to manage user or application program access to the second file system cell.

13. The method of claim 12, wherein the selecting of the file subsystem is based on loading of the processors in order to balance loading upon the processors.

15. The meta file system as claimed in claim 14, wherein the direct links include hidden objects in the file system cells, respective directory entries for the hidden objects are included in the file system cells, and the respective directory entries for the hidden objects each include an attribute indicating that the entry is for an external link.

16. The meta file system as claimed in claim 14, wherein each file system cell that includes at least one direct link to another file system cell in the hierarchy also includes file system meta data including an attribute indicating that said each file system cell includes at least one direct link to another file system cell in the hierarchy.

17. The meta file system as claimed in claim 14, wherein each direct link from a first file system cell in the meta file system to a second file system cell in the meta file system includes a link file in the first file system cell, and the link file contains information for locating the second file system cell.

18. The meta file system as claimed in claim 14, further including at least one indirect link from a first file system cell in the meta file system to a second file system cell in the meta file system, the indirect link being visible to the user or application program, and the indirect link including a target specification that is a name of a path in said single file system visible to the user or application program.

20. The method as claimed in claim 19, wherein the file system cells are linked in a hierarchy and said each data mover computer accesses a hidden root directory in the hierarchy of file system cells in order to direct the request to the data mover computer that has exclusive management of access to a respective one of the plurality of file system cells containing the object.

21. The method as claimed in claim 20, wherein the hidden root directory is contained in a file system cell that is replicated in each of the data mover computers.

22. The method as claimed in claim 19, wherein after a system crash, each data mover performs a consistency check upon the file system cells in said each data mover'"'"'s respective subset of file system cells.

23. The method of claim 19, which further includes load balancing upon the data movers by reconfiguring the file system cells in order to move objects from file system cells managed by more heavily loaded data movers to file system cells managed by less heavily loaded data movers.

24. The method as claimed in claim 23, wherein each data mover computer monitors loading upon said each data mover to determine whether said each data mover computer is heavily loaded, and upon determining that said each data mover computer is heavily loaded, said each data mover computer polls other data mover computers to search for a less heavily loaded data mover computer.

25. The method as claimed in claim 23, wherein the objects are moved by shifting file subsystems.

26. The method as claimed in claim 25, wherein file subsystems are shifted by removing the selected file subsystem from a first file system cell, converting the selected file subsystem into a second file system cell, and inserting a link in the first file system cell from a directory entry for the selected file subsystem to a root directory of the second file system cell.

27. The method of claim 19, which further includes load balancing upon the data movers by moving ownership of file system cells from more heavily loaded data movers to less heavily loaded data movers.

29. The network file server as claimed in claim 28, further including at least one dedicated communication link interconnecting the data mover computers for conveying the request from the data mover computer that has exclusive management of access to a respective one of the file system cells containing the object.

30. The network file server as claimed in claim 28, wherein the file system cells are linked in a hierarchy and said each data mover computer is programmed to access a hidden root directory in the hierarchy of file system cells in order to direct the request to the data mover computer that has exclusive management of access to a respective one of the plurality of file system cells containing the object.

31. The network file server as claimed in claim 30, wherein the hidden root directory is contained in a file system cell that is replicated in each of the data mover computers.

32. The network file server as claimed in claim 31, wherein the file system cell containing the hidden root directory contains no user-visible objects.

33. The network file server as claimed in claim 28, wherein each data mover is programmed to recover from a system crash by performing a consistency check upon the file system cells in said each data mover'"'"'s respective subset of file system cells.

34. The network file server as claimed in claim 28, wherein the data movers are programmed for load balancing by reconfiguring the file system cells in order to move objects from file system cells managed by more heavily loaded data movers to file system cells managed by less heavily loaded data movers.

35. The network file server as claimed in claim 34, wherein each data mover computer is programmed for monitoring loading upon said each data mover to determine whether said each data mover computer is heavily loaded, and upon determining that said each data mover computer is heavily loaded, said each data mover computer polls other data mover computers to search for a less heavily loaded data mover computer.

36. The network file server as claimed in claim 34, wherein each data mover computer is programmed to move objects by shifting file subsystems.

37. The network file server as claimed in claim 36, wherein each data mover computer is programmed to shift file subsystems by removing the selected file subsystem from a first file system cell, converting the selected file subsystem into a second file system cell, and inserting a link in the first file system cell from a directory entry for the selected file subsystem to a root directory of the second file system cell.

38. The network file server as claimed in claim 28, wherein the data movers are programmed for load balancing by changing ownership of file system cells from more heavily loaded data movers to less heavily loaded data movers.

40. The program storage device as claimed in claim 39, wherein the meta file system manager is executable for accessing each file system cell by checking whether said each file system cell has a meta file system attribute indicating that said each file system cell has external links, and when said each file system cell does not have a meta file system attribute, calling at least one file system manager routine that does not recognize external links, and when said each file system does have a meta file system attribute, executing at least one meta file system manager routine that recognizes external links.

41. The program storage device as claimed in claim 40, wherein the meta file system manager is executable to recover from a system crash by checking whether said each file system cell has a file system attribute indicating whether or not said each file system cell needs to be checked for consistency after the system crash.

14. A meta file system contained in data storage, the meta file system including a multiplicity of file system cells linked in a hierarchy by direct links in the file system cells, the direct links being hidden from a user or application program in such a way that the hierarchy appears to the user or application program as a single file system.

19. A method of operating a network file server to provide users in the network with concurrent access to objects in a user-visible file system, the objects being distributed over a plurality of file system cells in data storage of the network file server, the network file server having a plurality of data mover computers for responding to users requesting access to the data storage, the method comprising:

(a) configuring the data mover computers so that each data mover computer has exclusive management of access to a respective subset of the file system cells; and

(b) when each data mover computer receives a user request for access to an object in the user-visible file system, said each data mover computer directs the request to the data mover computer that has exclusive management of access to a respective one of the file system cells containing the object.

28. A network file server comprising:

data storage; and

a plurality of data mover computers adapted for coupling to a data network and coupled to the data storage to provide users in the network with concurrent access to objects in a user-visible file system, the objects being distributed over a plurality of file system cells in the data storage of the network file server;

wherein the data mover computers are configured so that each data mover computer has exclusive management of access to a respective subset of the file system cells; and

each data mover computer is programmed to receive a user request for access to an object in the user-visible file system, and directs the request to the data mover computer that has exclusive management of access to a respective one of the file system cells containing the object.

39. A program storage device containing a program executable by a network file server, the network file server being adapted for coupling to at least one client for access of the client to a user-visible file system in storage in the network file server, objects in the user-visible file system being contained in a plurality of file system cells, said program including a meta file system manager for receiving file system access calls intended for a file system manager, and the meta file system manager being executable for acting as an intermediary between the client and the file system manager in order to follow hidden links between the file system cells so that the file system cells appear to the client as one file system.