File aggregation in a switched file system

• US 8,195,760 B2
• Filed: 04/16/2008
• Issued: 06/05/2012
• Est. Priority Date: 01/11/2001
• Status: Active Grant

• Current Assignee: RPX Corporation
• Original Assignee: F5 Networks, Inc. (F5 Incorporated)
• Inventors: Lacapra, Francesco, Duvvuri, Srinivas P., Miloushev, Vladimir I., Nickolov, Peter A.
• Primary Examiner(s): VU, VIET D
• Application Number:
  US12/104,197
• Publication Number:
  US 20090106255A1
• Time in Patent Office: 1,511 Days
• Field of Search: 709/213, 709/214, 709/215, 709/217, 709/219, 707/600
• US Class Current: 709/214
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1446 Point-in-time backing up or restoration of persistent data
• CPC Class Codes: G06F 11/2056 by mirroring
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 2201/86 Event-based monitoring
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/567 Integrating service provisioning from a plurality of service providers
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

In a switched file system, a file switching device is logically positioned between clients and file servers and communicates with the clients and the file servers using standard network file protocols. The file switching device appears as a server to the client devices and as a client to the file servers. The file switching device aggregates storage from multiple file servers into a global filesystem and presents a global namespace to the client devices. The file switching device typically supports a “native” mode for integrating legacy files into the global namespace and an “extended” mode for actively managing files across one or more file servers. Typically, native-mode files may be accessed directly or indirectly via the file switching device, while extended-mode files may be accessed only through the file switching device. The file switching device may manage file storage using various types of rules, e.g., for managing multiple storage tiers or for applying different types of encoding schemes to files. Rules may be applied to pre-existing files.

Claims

1. A method for managing files by a file switch in a file storage system, each file associated with a file type, the method comprising:

aggregating, by the file switch, a plurality of storage volumes including at least one native mode volume and at least one extended mode volume into a global namespace;

allowing, by the file switch, client access to files in the at least one native mode volume indirectly via the aggregated global namespace;

maintaining, by the file switch, a set of rules for storing files in a plurality of file servers including at least a metadata server and a storage server, the rules defining at least one file type to be stored in the metadata server; and

storing, by the file switch, a file based at least in part on the file type associated with the file according to the set of rules, whereby the file is stored in the metadata server only if the file type associated with the file is a file type to be stored in the metadata server.

2. A method according to claim 1, further comprising:

allowing, by the file switch, client access to files in the at least one native mode volume directly.

3. method according to claim 1, further comprising:

selectively migrating, by the file switch, files from at least one of the at least one native mode volume into at least one of the at least one extended mode volume.

4. A method according to claim 3, wherein such selective migrating comprises:

converting, by the file switch, a native mode file to an extended mode file stored in a fragmented form over a plurality of file servers.

5. A method according to claim 3, wherein selectively migrating comprises:

converting, by the file switch, a native mode file to an extended mode file stored redundantly over a plurality of file servers.

6. A method according to claim 1, wherein aggregating comprises:

creating, by the file switch, a mount point for the native mode volume within the global namespace, the mount point associated with a pathname prefix.

7. A method according to claim 6, wherein allowing client access to files in the at least one native mode volume indirectly via the aggregated global namespace comprises:

receiving, by the file switch, a first request for access to a native mode file, the first request including a pathname for the native mode file in the global namespace including the pathname prefix; and

transmitting, by the file switch, a second request to a file server hosting the native mode file, the second request including a pathname for the native mode file in the native mode volume without the pathname prefix.

8. A method according to claim 7, further comprising:

receiving, by the file switch, a handle from the native mode volume in response to the second request; and

transmitting, by the file switch, the handle to the client as a response to the first request.

9. A method according to claim 8, further comprising:

receiving, by the file switch, from the client a third request including the handle; and

transmitting, by the file switch, the third request to the native mode volume.

10. A method according to claim 9, further comprising:

receiving, by the file switch, a reply from the native mode volume in response to the third request; and

transmitting, by the file switch, the reply to the client.

11. A method according to claim 7, wherein transmitting the second request comprises:

spoofing, by the file switch, between a first network file protocol used by the client and a second network file protocol used by the file server.

12. A method according to claim 7, wherein transmitting the second request comprises:

protocol translation, by the file switch, between a first network file protocol used by the client and a second network file protocol used by the file server.

13. A method according to claim 1, further comprising:

managing, by the file switch, file storage operations according to the set of rules,

wherein the rules further define criteria for storing files using the at least one native mode volume and the at least one extended mode volume.

14. A method according to claim 13, wherein the rules further define types of files that may be created in a native mode volume.

15. A method according to claim 14, wherein the rules further define at least one of:

types of files that are expressly allowed to be created in the native mode volume;

or

types of files that expressly denied from being created in the native mode volume.

16. A method according to claim 14, wherein the rules further define the types of files that may be created in the native mode volume based on at least one of a file suffix or a file size.

17. A method according to claim 13, wherein storing the file according to the set of rules is performed upon receipt of a request to create the file.

18. A method according to claim 13, wherein storing the file according to the set of rules is performed upon receipt of a request to rename the file.

19. A method according to claim 13, wherein storing the file according to the set of rules involves reapplying the set of rules to a pre-existing file.

20. A switched file apparatus for managing files, the apparatus comprising:

one or more processors; and

a memory coupled to the one or more processors, the one or more processors configured to execute programmed instructions stored in the memory, the programmed instructions comprising;

aggregating a plurality of storage volumes into a global namespace, the plurality of storage volumes including at least one native mode volume and at least one extended mode volume;

allowing client access to files in the at least one native mode volume indirectly via the aggregated global namespace;

maintaining a set of rules for storing files in a plurality of file servers including at least a metadata server and a storage server, the rules defining at least one file type to be stored in the metadata server; and

storing a file based at least in part on the file type associated with the file according to the set of rules, whereby the file is stored in the metadata server only if the file type associated with the file is a file type to be stored in the metadata server.

21. A switched file apparatus according to claim 20, further comprising selectively migrating files from at least one of the at least one native mode volume into at least one of the at least one extended mode volume.

22. A switched file apparatus according to claim 21, wherein such selective migration comprises:

converting a native mode file to an extended mode file stored in a fragmented form over a plurality of file servers.

23. A switched file apparatus according to claim 21, wherein such selective migration comprises:

converting a native mode file to an extended mode file stored redundantly over a plurality of file servers.

24. A switched file apparatus according to claim 20, further comprising creating a mount point for the native mode volume within the global namespace, the mount point associated with a pathname prefix.

25. A switched file apparatus according to claim 24, further comprising:

receiving a first request for access to a native mode file, the first request including a pathname for the native mode file in the global namespace including the pathname prefix; and

transmitting a second request to a file server hosting the native mode file, the second request including a pathname for the native mode file in the native mode volume without the pathname prefix.

26. A switched file apparatus according to claim 25, further comprising:

receiving a handle from the native mode volume in response to the second request; and

transmitting the handle to a client as a response to the first request.

27. A switched file apparatus according to claim 26, further comprising:

receiving from the client a third request including the handle; and

transmitting the third request to the native mode volume.

28. A switched file apparatus according to claim 27, further comprising:

receiving a reply from the native mode volume in response to the third request; and

transmitting the reply to the client.

29. A switched file apparatus according to claim 25, wherein transmitting the second request comprises:

spoofing between a first network file protocol used by a client and a second network file protocol used by a file server.

30. A switched file apparatus according to claim 25, wherein transmitting the second request comprises:

protocol translation between a first network file protocol used by a client and a second network file protocol used by a file server.

31. A switched file apparatus according to claim 20, further comprising managing file storage operations according to the set of rules, wherein the rules further define criteria for storing files using the at least one native mode volume and the at least one extended mode volume.

32. A switched file apparatus according to claim 31, wherein the rules define types of files that may be created in a native mode volume.

33. A switched file apparatus according to claim 32, wherein the rules define at least one of:

types of files that are expressly allowed to be created in the native mode volume;

or

types of files that are expressly denied from being created in the native mode volume.

34. A switched file apparatus according to claim 32, wherein the rules specify the types of files that may be created in the native mode volume based on at least one of a file suffix or a file size.

35. A switched file apparatus according to claim 31, further comprising storing the file according to the set of rules upon receipt of a request to create the file.

36. A switched file apparatus according to claim 31, further comprising storing the file according to the set of rules upon receipt of a request to rename the file.

37. A switched file apparatus according to claim 31, further comprising selectively reapplying the set of rules to a pre-existing file.

38. A method of storing files by a file switch in a switched file system, each file associated with a file type, the method comprising:

maintaining, by the file switch, a set of rules for storing files in a plurality of file servers including at least a metadata server and a storage server, the rules defining at least one file type to be stored in the metadata server; and

storing, by the file switch, a file based at least in part on the file type associated with the file according to the set of rules, whereby the file is stored in the metadata server only if the file type associated with the file is a file type to be stored in the metadata server.

39. A method according to claim 38, wherein the rules further define a file size threshold for each file type, and wherein the file is stored in the metadata server only if the file type associated with the file is a file type to be stored in the metadata server and a size of the file is below the file size threshold defined for the file type.

40. A method according to claim 39, wherein the file size threshold is the same for all file types.

41. A method according to claim 39, wherein the file size threshold is different for at least two different file types.

42. A method according to claim 38, wherein storing the file according to the set of rules is performed upon receipt of a request to create the file.

43. A method according to claim 38, wherein storing the file according to the set of rules is performed upon receipt of a request to rename the file.

44. A method according to claim 38, wherein storing the file according to the set of rules involves reapplying the set of rules to a pre-existing file.

45. A switched file apparatus for storing files, each file associated with a file type, the apparatus comprising:

one or more processors; and

a memory coupled to the one or more processors, the one or more processors configured to execute programmed instructions stored in the memory, the programmed instructions comprising;

maintaining a set of rules for storing files in a plurality of file servers comprising a metadata server and a storage server, the rules defining at least one file type to be stored in the metadata server; and

storing a file based at least in part on the file type associated with the file according to the set of rules, whereby the file is stored in the metadata server only if the file type associated with the file is a file type to be stored in the metadata server.

46. A switched file apparatus according to claim 45, wherein the rules further define a file size threshold for each file type, and wherein the file is stored in the metadata server only if the file type associated with the file is a file type to be stored in the metadata server and a size of the file is below the file size threshold defined for the file type.

47. A switched file apparatus according to claim 46, wherein the file size threshold is the same for all file types.

48. A switched file apparatus according to claim 46, wherein the file size threshold is different for at least two different file types.

49. A switched file apparatus according to claim 45, wherein storing the file according to the set of rules is performed upon receipt of a request to create the file.

50. A switched file apparatus according to claim 45, wherein storing the file according to the set of rules is performed upon receipt of a request to rename the file.

51. A switched file apparatus according to claim 45, wherein storing the file according to the set of rules involves reapplying the set of rules to a pre-existing file.