Virtual file system

• US 20030115218A1
• Filed: 12/19/2001
• Published: 06/19/2003
• Est. Priority Date: 12/19/2001
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Friedman, Marc T., Doll, Stephan A., Mullally, Joseph P., Bobbitt, Jared E., Lau, Patrick Wing Sang
• Granted Patent: US 7,024,427 B2
• Time in Patent Office: Days
• US Class Current: 707/200
• CPC Class Codes: G06F 16/188 Virtual file systems
• CPC Class Codes: G06F 2003/0697 device management, e.g. handlers, drivers, I/O schedulers
• CPC Class Codes: G06F 3/0601 Interfaces specially adapted for storage systems

Abstract

A virtual file system and method. The system architecture enables a plurality of underlying file systems running on various file servers to be “virtualized” into one or more “virtual volumes” that appear as a local file system to clients that access the virtual volumes. The system also enables the storage spaces of the underlying file systems to be aggregated into a single virtual storage space, which can be dynamically scaled by adding or removing file servers without taking any of the file systems offline and in a manner transparent to the clients. This functionality is enabled through a software “virtualization” filter on the client that intercepts file system requests and a virtual file system driver on each file server. The system also provides for load balancing file accesses by distributing files across the various file servers in the system, through migration of data files between servers.

Claims

1. A method for virtualizing a plurality of underlying file systems hosted on one or more file servers, wherein each file system includes a storage space in which a plurality of data files may be stored comprising:

aggregating the storage spaces of said plurality of file systems into a single virtual storage space that is exported as a virtual volume that appears to clients as local file system;

creating a virtual volume directory and file name hierarchy including a virtual pathname for each data file stored in the virtual volume;

providing a software virtualization layer including both client-side and server-side software components that cooperatively enable clients to access data files stored in the virtual volume through reference to the virtual pathnames for those data files,

wherein clients that access data files stored in the virtual volume do not need to know the file servers and pathnames under which those data files are actually stored.

2. The method of claim 1, wherein said one or more file servers are accessed via a local area network (LAN).

3. The method of claim 1, wherein at least one of said one or more file servers is accessed via a wide area network (WAN).

4. The method of claim 1, wherein said plurality of underlying file systems include at least two file systems that comprise different local file system types.

5. The method of claim 1, further comprising dynamically scaling the single virtual storage space by adding a new underlying file system to the virtual file system without having to take any existing underlying file system offline.

6. The method of claim 1, further comprising migrating files initially stored on a first underlying file system to a second underlying file system, wherein the files are migrated without having to take any of the underlying file systems offline and in a manner that is transparent to the clients.

7. The method of claim 1, further comprising storing file mapping data that is used to route client file system access requests that reference a virtual file name to a file server and location at which the data file corresponding to the virtual pathname is actually stored.

8. The method of claim 1, wherein data describing the virtual volume directory and file name hierarchy is stored in a master logical volume corresponding to one of said underlying file systems and the data files corresponding to the virtual pathnames in the virtual volume are stored in one or more slave logical volumes corresponding to at least one of said plurality of underlying file systems.

9. The method of claim 8, further comprising maintaining a copy of system configuration information that identifies the file server used to host the master logical volume and the file servers hosting the slave logical volumes on each client that accesses the virtual file system.

10. The method of claim 8, further comprising maintaining a copy of system configuration information that identifies the file server used to host the master logical volume and the file servers hosting the slave logical volumes on each of said one or more file servers.

11. The method of claim 8, wherein one of the underlying file systems hosts both the master logical volume and a slave logical volume.

13. The method of claim 1, wherein data describing the virtual file system directory and file name hierarchy is stored in a master volume corresponding to one of said underlying file systems and the data files corresponding to the virtual pathnames in the virtual volume are stored in one or more slave volumes corresponding to at least one of said plurality of underlying file systems.

12. A method for creating a virtual file system that provides clients access to files stored on a plurality of underlying file systems hosted on one or more file servers, comprising:

exporting a virtual file system view to a client comprising a virtual directory and file hierarchy that includes a virtual pathnames for each data file that may be accessed by the client;

maintaining mapping information that maps the virtual pathname for each file to a file server and physical pathname under which the file is stored on that file server;

enabling a data file to be accessed by the client through referencing the file by its virtual pathname;

providing a client-side filter that intercepts file system access requests made by the client that reference the virtual pathname to access a file; and

routing the file system request to the server and pathname under which the file is actually stored based on the mapping information for the file.

14. The method of claim 12, further comprising maintaining a copy of system configuration information that identifies the file server used to host the master volume and the file servers hosting the slave volumes on each of said one or more file servers.

15. The method of claim 14, wherein the data describing the virtual directory and file name hierarchy comprises a master directory and file hierarchy structure stored on the master logical volume containing directories and files that are named and hierarchically configured to mirror the virtual directory and file name hierarchy.

16. A method for providing a virtual file system that aggregates storage space that is partitioned across a plurality of underlying file systems hosted on one or more file servers, comprising:

enabling clients to create a virtual directory and file name hierarchy in which data files that may be accessed via the virtual file system are identified by a corresponding virtual pathname;

storing the data files on one or more slave logical volumes hosted by said plurality of underlying file systems; and

providing a software file system virtualization layer including both client-side and server-side software components that cooperatively enable clients to access data files stored in the single virtual storage space through reference to the virtual pathnames for those data files by intercepting client file system requests and routing the file system request to the actual file server and pathname under which the file is stored.

17. The method of claim 16, further comprising storing file mapping data on a master logical volume hosted by one of said plurality of underlying file systems, wherein the file mapping data contains information for mapping the virtual pathname for a given data file to the actual file server and pathname under which the file is stored.

18. The method of claim 17, wherein the file mapping data includes data describing the virtual file system directory and file name hierarchy that is stored in the master logical volume and the data files are stored in one or more slave logical volumes corresponding to at least one of said plurality of underlying file systems.

19. The method of claim 18, wherein the data describing the virtual directory and file name hierarchy comprises a master directory and file hierarchy structure stored on the master logical volume containing directories and files that are named and hierarchically configured to mirror the virtual directory and file name hierarchy.

20. The method of claim 19, wherein each file in the master directory and file hierarchy structure comprises a pointer file that includes data that identifies on what slave logical volume the data file corresponding to that file is physically stored.

21. The method of claim 20, wherein the pointer file includes a first GUID (global unique identifier) that identifies the slave logical volume on which a data file is stored and a second GUID that is used to identify where on the slave logical volume the data file is stored.

22. The method of claim 21, wherein the data file is stored in the underlying file system corresponding to the slave logical volume identified by its second GUID through use of a physical pathname that is derived from its second GUID.

23. The method of claim 16, wherein a copy of system configuration information that identifies the file servers that host each of the master logical volume and the slave logical volumes is maintained on each client that accesses the virtual file system.

24. The method of claim 16, wherein creation of a new virtual directory in the virtual directory and file name hierarchy comprises creating a new directory in the master directory and file hierarchy structure having a hierarchical position and name corresponding to a hierarchical position and name of the new directory in the virtual directory and file name hierarchy.

25. A virtual file system comprising:

a computer network that connects a plurality of clients to a plurality of file servers;

a plurality of underlying file systems hosted on the plurality of servers in which a plurality of data files are stored;

a server-side software virtualization component running on each file server;

a client-side software virtualization component running on each client,

wherein the client-side and server-side software virtualization components cooperatively enable the clients to create a virtual directory and file name hierarchy including a virtual pathname for each data file that is used to access that data file by intercepting client file system requests that reference the virtual pathname of the data file and routing the file system request to the actual file server and pathname under which the data file is stored.

26. The virtual file system of claim 25, wherein data describing the virtual directory and file name hierarchy is stored in a master logical volume corresponding to one of said underlying file systems and the data files are stored in one or more slave logical volumes corresponding to at least one of said plurality of underlying file systems.

27. The virtual file system of claim 25, wherein a primary set of system configuration information that maps each of the master logical volume and the slave logical volumes to the file server that hosts that master or slave logical volume is stored on one of said plurality of file servers and a local copy of the system configuration information is maintained on each of the files servers and each of the clients.

28. The virtual file system of claim 27, wherein the primary set of system configuration information is changed in response to a change in a configuration of the plurality file servers and/or underlying file systems, and wherein each client includes an agent that updates the local copy of the system configuration information stored on the client when the system configuration information is changed

29. The virtual file system of claim 25, wherein the primary set of system configuration information is changed in response to a change in a configuration of the plurality file servers and/or underlying file systems, and wherein each client includes an agent that updates the local copy of the system configuration information stored on the client when the system configuration information is changed

30. The virtual file system of claim 25, wherein the computer network comprises a wide area network (WAN).