SCALABLE SYSTEM FOR PARTITIONING AND ACCESSING METADATA OVER MULTIPLE SERVERS

• US 20110087696A1
• Filed: 12/20/2010
• Published: 04/14/2011
• Est. Priority Date: 01/20/2005
• Status: Active Grant

• Current Assignee: F5 Networks, Inc. (F5 Incorporated)
• Original Assignee: F5 Networks, Inc. (F5 Incorporated)
• Inventors: LACAPRA, Francesco
• Granted Patent: US 8,433,735 B2
• Time in Patent Office: Days
• US Class Current: 707/769
• CPC Class Codes: H04L 61/4552 Lookup mechanisms between a plurality of directories; Synchronisation of directories, e.g. metadirectories

Abstract

In an aggregated file system, metadata is partitioned into multiple metadata volumes. On receipt of a file processing request, a file switch examines its mount entry cache to identify a target metadata volume that hosts the metadata of the requested file. The identification begins with mount entries at a root volume and continues recursively by examining a portion of the absolute pathname of the file until the target metadata volume is identified. Finally, the file switch forwards the request to a metadata server managing the target metadata volume. Since the identification process is carried out completely within the file switch, there is no need for multiple expensive network accesses to different metadata servers.

Claims

1. A method of handling a file processing request in an aggregated file system, comprising:

receiving a request with respect to a file, the request identifying an absolute pathname of the file in the aggregated file system;

identifying a target metadata volume that hosts metadata of the file among the metadata volumes in accordance with the request and one or more mount entries each associated with a respective one of a plurality of metadata volumes, including recursively identifying a mount entry when associated with a current metadata volume, wherein the mount entry identifies a relative pathname matching a respective portion of the absolute pathname, the identified relative pathname further comprising pathname sub-components of sizes ranging from one pathname component to a size of the identified relative pathname; and

selecting at least one of the pathname sub-components is selected using the identified mount entry based upon a number indicated in a list entry pointed to by the identified mount entry in response to the request to a file switch for assembling the file.

2. The method of claim 1, wherein the recursively identifying includes producing a residual pathname by removing from the absolute pathname the relative pathname associated with each identified mount entry.

3. The method of claim 1, wherein the absolute pathname of the file is unique in the aggregated file system.

4. The method of claim 1, wherein at least a subset of the plurality of metadata volumes includes one or more mount entries, each mount entry identifying a respective distinct child metadata volume and an associated relative pathname.

5. The method of claim 4, wherein at least one metadata volume in the subset of the plurality of metadata volumes includes a reverse mapper, the reverse mapper identifying a parent metadata volume.

6. The method of claim 1, wherein the plurality of metadata volumes include a root metadata volume, zero or more intermediate metadata volumes and zero or more leaf metadata volumes, and wherein the root metadata volume is a metadata volume that has no parent metadata volume, a leaf metadata volume is a metadata volume that has no child metadata volume and an intermediate metadata volume is a metadata volume that has both at least one parent and at least one child metadata volumes.

7. The method of claim 6, wherein the target metadata volume is the root metadata volume when none of the child metadata volumes directly referenced by the root metadata volume has an associated relative pathname matching a portion of the absolute pathname.

8. The method of claim 6, wherein the target metadata volume is an intermediate metadata volume when the intermediate metadata volume has an associated relative pathname matching a portion of the absolute pathname and none of the child metadata volumes referenced by the intermediate metadata volume has an associated relative pathname matching a portion of the absolute pathname.

9. The method of claim 6, wherein the target metadata volume is a leaf metadata volume referenced by the root metadata volume using one or more mount entries when the leaf metadata volume has an associated relative pathname matching a portion of the absolute pathname.

10. The method of claim 1, further comprising:

forming an ordered pair comprising the identified target metadata volume and a residual pathname, if any, based upon said identifying of the target metadata volume.

11. The method of claim 1 further comprising:

accessing a plurality of mount entries, wherein each of the plurality of mount entries is associated with a respective one of the plurality of metadata volumes as a parent metadata volume and identifies a respective distinct child metadata volume and the relative pathname associated with the child metadata volume;

resetting the current metadata volume to the child metadata volume identified by the identified mount entry, until no mount entry associated with the current metadata volume is found that matches a respective portion of the absolute pathname, the target metadata volume comprising the last child metadata volume so identified; and

returning information identifying the target metadata volume, hosted metadata and its associated residual pathname in response to the request to a file switch for assembling the file in response to the request.

12. The method of claim 1, wherein the identified relative pathname has a size based upon a maximum number of pathname components of the relative pathname that can be handled by the identified mount entry.

13. An aggregated file system, comprising:

a plurality of file servers;

a plurality of metadata servers that manage a plurality of metadata volumes; and

a plurality of file switches, each file switch including

a plurality of mount entries, wherein each of the plurality of mount entries is associated with a respective one of the plurality of metadata volumes as a parent metadata volume and identifies a respective distinct child metadata volume and an associated relative pathname;

at least one interface for exchanging information with the file servers, the metadata volumes and client computers; and

at least one processing unit for executing stored instructions comprising;

instructions for receiving a request with respect to a specified file, the request including an absolute pathname of the specified file in the aggregated file system,

instructions for identifying a target metadata volume that hosts metadata of the file among the metadata volumes in accordance with the request and one or more mount entries each associated with a respective one of a plurality of metadata volumes, including instructions for recursively identifying a mount entry when associated with a current metadata volume, wherein the mount entry identifies a relative pathname matching a respective portion of the absolute pathname, the identified relative pathname further comprising pathname sub-components of sizes ranging from one pathname component to a size of the identified relative pathname; and

instructions for selecting at least one of the pathname sub-components is selected using the identified mount entry based upon a number indicated in a list entry pointed to by the identified mount entry in response to the request to a file switch for assembling the file.

14. The file system of claim 13, wherein the respective portion of the absolute pathname that matches the relative pathname of a mount entry comprises a beginning portion of the absolute pathname of the specified file after removing the respective portions, if any, matched during one or more prior iterations of the instructions for recursively identifying a mount entry.

15. The file system of claim 13, wherein each metadata volume in a subset of the metadata volumes includes a reverse mapper, the reverse mapper identifying a parent metadata volume.

16. The file system of claim 13, wherein the plurality of metadata volumes include a root metadata volume, zero or more intermediate metadata volumes and zero or more leaf metadata volumes, and wherein the root metadata volume is a metadata volume that has no parent metadata volume, a leaf metadata volume is a metadata volume that has no child metadata volume and an intermediate metadata volume is a metadata volume that has both at least one parent and at least one child metadata volumes.

17. The file system of claim 16, wherein the target metadata volume is the root metadata volume when none of the child metadata volumes directly referenced by the root metadata volume has an associated relative pathname matching a portion of the absolute pathname.

18. The file system of claim 16, wherein the target metadata volume is an intermediate metadata when the intermediate metadata volume has an associated relative pathname matching a portion of the absolute pathname and none of the child metadata volumes referenced by the intermediate metadata volume has an associated relative pathname matching a portion of the absolute pathname.

19. The file system of claim 16, wherein the target metadata volume is a leaf metadata volume referenced by the root metadata volume using one or more mount entries if the leaf metadata volume has an associated relative pathname matching a portion of the absolute pathname.

20. The file system of claim 13, wherein said instructions for identifying the target metadata volume further comprise instructions for forming an ordered pair comprising the identified target metadata volume and a residual pathname, if any, based upon said identifying of the target metadata volume.

21. The file system of claim 13, wherein the at least one processing unit for executing stored instructions further comprises:

instructions for accessing a plurality of mount entries, wherein each of the plurality of mount entries is associated with a respective one of the plurality of metadata volumes as a parent metadata volume and identifies a respective distinct child metadata volume and the relative pathname associated with the child metadata volume;

instructions for resetting the current metadata volume to the child metadata volume identified by the identified mount entry, until no mount entry associated with the current metadata volume is found that matches a respective portion of the absolute pathname, the target metadata volume comprising the last child metadata volume so identified; and

instructions for returning information identifying the target metadata volume, hosted metadata and its associated residual pathname in response to the request to a file switch for assembling the file in response to the request.

22. The file system of claim 13, wherein the identified relative pathname has a size based upon a maximum number of pathname components of the relative pathname that can be handled by the identified mount entry.

23. A file switch for use in a computer network having a plurality of file servers, a plurality of metadata volumes and a plurality of client computers, the file switch comprising:

at least one processing unit for executing computer programs;

at least one interface for exchanging information with the file servers, the metadata volumes and the client computers, the information exchanged including information concerning a specified file; and

a plurality of mount entries, wherein each of the plurality of mount entries is associated with a respective one of the plurality of metadata volumes as a parent metadata volume and identifies a respective distinct child metadata volume and an associated relative pathname,

instructions for receiving a request with respect to the specified file, the request including an absolute pathname of the specified file in the aggregated file system,

instructions for identifying a target metadata volume that hosts metadata of the file among the metadata volumes in accordance with the request and one or more mount entries each associated with a respective one of a plurality of metadata volumes, including instructions for recursively identifying a mount entry when associated with a current metadata volume, wherein the mount entry identifies a relative pathname matching a respective portion of the absolute pathname, the identified relative pathname further comprising pathname sub-components of sizes ranging from one pathname component to a size of the identified relative pathname; and

instructions for selecting at least one of the pathname sub-components is selected using the identified mount entry based upon a number indicated in a list entry pointed to by the identified mount entry in response to the request to a file switch for assembling the file.

24. The file switch of claim 23, wherein the respective portion of the absolute pathname that matches the relative pathname of a mount entry comprises a beginning portion of the absolute pathname of the specified file after removing the respective portions, if any, matched during one or more prior iterations of the instructions for recursively identifying a mount entry.

25. The file switch of claim 23, wherein said instructions for identifying the target metadata volume further comprise instructions for forming an ordered pair comprising the identified target metadata volume and a residual pathname, if any, based upon said identifying of the target metadata volume.

26. The file switch of claim 23 further comprising:

instructions for accessing a plurality of mount entries, wherein each of the plurality of mount entries is associated with a respective one of the plurality of metadata volumes as a parent metadata volume and identifies a respective distinct child metadata volume and the relative pathname associated with the child metadata volume;

instructions for resetting the current metadata volume to the child metadata volume identified by the identified mount entry, until no mount entry associated with the current metadata volume is found that matches a respective portion of the absolute pathname, the target metadata volume comprising the last child metadata volume so identified; and

instructions for returning information identifying the target metadata volume, hosted metadata and its associated residual pathname in response to the request to a file switch for assembling the file in response to the request.

27. The file switch of claim 23, wherein the identified relative pathname has a size based upon a maximum number of pathname components of the relative pathname that can be handled by the identified mount entry.

28. An aggregated file system, comprising:

a plurality of file servers, wherein at least one of them is a self-sustained file server configured to be insulated from striping and mirroring strategy implemented in the aggregated file system;

a plurality of file switches, each file switch including a plurality of mount entries, wherein at least one of the mount entries references the self-sustained file server; and

at least one of the plurality of file switches having at least one processing unit for executing stored instructions comprising;

instructions for receiving a request with respect to a specified file, the request including an absolute pathname of the specified file in the aggregated file system,

instructions for identifying the mount entry referencing the self-sustained file server in accordance with the absolute pathname of the specified file, and

instructions for transferring the request to the self-sustained file server through the identified mount entry.

29. The file system of claim 28, wherein a new mount entry referencing the self-sustained file server is inserted into a mount entry list associated with one of the plurality of mount entries when the file server is incorporated into the aggregated file system.

30. The file system of claim 28, wherein an existing mount entry referencing the self-sustained file server is removed from a mount entry list associated with one of the plurality of mount entries when the self-sustained file server is disconnected from the aggregated file system.