Virtual file system

• US 7,024,427 B2
• Filed: 12/19/2001
• Issued: 04/04/2006
• Est. Priority Date: 12/19/2001
• Status: Expired due to Term

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Friedman, Marc T., Doll, Stephan A., Mullally, Joseph P., Lau, Patrick Wing Sang, Bobbitt, Jared E.
• Primary Examiner(s): Coby, Frantz
• Assistant Examiner(s): ALAUBAIDI, HAYTHIM J
• Application Number:
  US10/025,005
• Publication Number:
  US 20030115218A1
• Time in Patent Office: 1,567 Days
• Field of Search: 707/200, 707/201, 707/205, 707/100, 707/102, 707/1, 707/10
• US Class Current: 1/1
• CPC Class Codes: G06F 16/188 Virtual file systems
• CPC Class Codes: G06F 2003/0697 device management, e.g. handlers, drivers, I/O schedulers
• CPC Class Codes: G06F 3/0601 Interfaces specially adapted for storage systems

Abstract

A virtual file system and method. The system architecture enables a plurality of underlying file systems running on various file servers to be “virtualized” into one or more “virtual volumes” that appear as a local file system to clients that access the virtual volumes. The system also enables the storage spaces of the underlying file systems to be aggregated into a single virtual storage space, which can be dynamically scaled by adding or removing file servers without taking any of the file systems offline and in a manner transparent to the clients. This functionality is enabled through a software “virtualization” filter on the client that intercepts file system requests and a virtual file system driver on each file server. The system also provides for load balancing file accesses by distributing files across the various file servers in the system, through migration of data files between servers.

Claims

1. A method for virtualizing a plurality of underlying file systems hosted on one or more file servers, wherein each file system includes a storage space in which a plurality of data files may be stored comprising:

aggregating the storage spaces of said plurality of file systems into a single virtual storage space that is exported as a virtual volume that appears to clients as local file system;

creating a virtual volume directory and file name hierarchy including a virtual pathname for each data file stored in the virtual volume;

providing a software virtualization layer including both client-side and server-side software components that cooperatively enable clients to access data files stored in the virtual volume through reference to the virtual pathnames for those data files; and

maintaining a copy of system configuration information that identifies the file server used to host a master logical volume and one or more file servers hosting one or more slave logical volumes on each client that accesses the virtual file system,

wherein clients that access data files stored in the virtual volume do not need to know the file servers and pathnames under which those data files are actually stored;

for each file in the virtual name directory and file hierarchy, data separate from the file system data of the underlying file system is associated with the virtual directory and file name hierarchy, the data including a metadata and a pointer to the file contents stored in one or more underlying file systems;

data describing the virtual volume directory and file name hierarchy is stored in the master logical volume corresponding to one of said underlying file systems; and

the data files corresponding to the virtual pathnames in the virtual volume are stored in the one or more slave logical volumes corresponding to at least one of said plurality of underlying file systems.

2. A method for creating a virtual file system that provides clients access to files stored on a plurality of underlying file systems hosted on one or more file servers, comprising:

exporting a virtual file system view to a client comprising a virtual directory and file name hierarchy that includes a virtual pathnames for each data file that may be accessed by the client;

maintaining mapping information that maps the virtual pathname for each file to a file server and physical pathname under which the file is stored on that file server;

maintaining a copy of system configuration information that identifies the file server used to host a master volume and the file servers hosting one or more slave volumes on each of said one or more file servers;

enabling a data file to be accessed by the client through referencing the file by its virtual pathname;

providing a client-side filter that intercepts file system access requests made by the client that reference the virtual pathname to access a file; and

routing the file system request to the server and pathname under which the file is actually stored based on the mapping information for the file;

wherein for each file in the virtual directory and file name hierarchy, data separate from the file system data of the underlying file system is associated with the virtual directory and file name hierarchy, the data including a metadata and a pointer to the file contents stored in one or more underlying file systems; and

data describing the virtual file system directory and file name hierarchy is stored in the master volume corresponding to one of said underlying file systems and the data files corresponding to the virtual pathnames in the virtual volume are stored in the one or more slave volumes corresponding to at least one of said plurality of underlying file systems.

3. The method of claim 2, wherein the data describing the virtual directory and file name hierarchy comprises a master directory and file hierarchy structure stored on the master logical volume containing directories and files that are named and hierarchically configured to mirror the virtual directory and file name hierarchy.

4. A method for providing a virtual file system that aggregates storage space that is partitioned across a plurality of underlying file systems hosted on one or more file servers, comprising:

enabling clients to create a virtual directory and file name hierarchy in which data files that may be accessed via the virtual file system are identified by a corresponding virtual pathname;

storing the data files on one or more slave logical volumes hosted by said plurality of underlying file systems;

providing a software file, system virtualization layer including both client-side and server-side software components that cooperatively enable clients to access data files stored in the single virtual storage space through reference to the virtual pathnames for those data files by intercepting client file system requests and routing the file system request to the actual file server and pathname under which the file is stored; and

storing file mapping data on a master logical volume hosted by one of said plurality of underlying file systems;

wherein for each file in the virtual directory and file name hierarchy, data separate from the file system data of the underlying file system is associated with the virtual directory and file name hierarchy, the data including a metadata and a pointer to the file contents stored in one or more underlying file systems;

the file mapping data contains information for mapping the virtual pathname for a given data file to the actual file server and pathname under which the file is stored; and

a copy of system configuration information that identifies the file servers that host each of the master logical volume and the slave logical volumes is maintained on each client that accesses the virtual file system.

5. A virtual file system comprising:

a computer network that connects a plurality of clients to a plurality of file servers;

a plurality of underlying file systems hosted on the plurality of servers in which a plurality of data files are stored;

a server-side software virtualization component running on each file server;

a client-side software virtualization component running on each client,

wherein the client-side and server-side software virtualization components cooperatively enable the clients to create a virtual directory and file name hierarchy including a virtual pathname for each data file that is used to access that data file by intercepting client file system requests that reference the virtual pathname of the data file and routing the file system request to the actual file server and pathname under which the data file is stored;

for each file in the virtual directory and file name hierarchy, data separate from the file system data of the underlying file system is associated with the virtual directory and file name hierarchy, the data including a metadata and a pointer to the file contents stored in one or more underlying file systems;

data describing the virtual directory and file name hierarchy is stored in a master logical volume corresponding to one of said underlying file systems;

the data files are stored in one or more slave logical volumes corresponding to at least one of said plurality of underlying file systems; and

a primary set of system configuration information that maps each of the master logical volume and the slave logical volumes to the file server that hosts that master or slave logical volume is stored on one of said plurality of file servers and a local copy of the system configuration information is maintained on each of the files servers and each of the clients.

6. The virtual file system of claim 5, wherein the primary set of system configuration information is changed in response to a change in a configuration of the plurality file servers and/or underlying file systems, and wherein each client includes an agent that updates the local copy of the system configuration information stored on the client when the system configuration information is changed.