File Aggregation in a Switched File System

• US 20090106255A1
• Filed: 04/16/2008
• Published: 04/23/2009
• Est. Priority Date: 01/11/2001
• Status: Active Grant

• Current Assignee: RPX Corporation
• Original Assignee: Attune Systems, Inc. (F5 Incorporated)
• Inventors: Lacapra, Francesco, Nickolov, Peter, Miloushev, Vladimir, Duvvuri, Srinivas P.
• Granted Patent: US 8,195,760 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1446 Point-in-time backing up or restoration of persistent data
• CPC Class Codes: G06F 11/2056 by mirroring
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 2201/86 Event-based monitoring
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/567 Integrating service provisioning from a plurality of service providers
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

In a switched file system, a file switching device is logically positioned between clients and file servers and communicates with the clients and the file servers using standard network file protocols. The file switching device appears as a server to the client devices and as a client to the file servers. The file switching device aggregates storage from multiple file servers into a global filesystem and presents a global namespace to the client devices. The file switching device typically supports a “native” mode for integrating legacy files into the global namespace and an “extended” mode for actively managing files across one or more file servers. Typically, native-mode files may be accessed directly or indirectly via the file switching device, while extended-mode files may be accessed only through the file switching device. The file switching device may manage file storage using various types of rules, e.g., for managing multiple storage tiers or for applying different types of encoding schemes to files. Rules may be applied to pre-existing files.

Claims

1. A method for managing files by a file switch in a file storage system, the method comprising:

aggregating a plurality of storage volumes including at least one native mode volume and at least one extended mode volume into a global namespace; and

allowing client access to files in the at least one native mode volume indirectly via the aggregated global namespace.

2. A method according to claim 1, further comprising:

allowing client access to files in the at least one native mode volume directly.

3. A method according to claim 1, further comprising:

selectively migrating files from the native mode volume into an extended mode volume.

4. A method according to claim 3, wherein such selective migrating comprises:

converting a native mode file to an extended mode file stored in a fragmented form over a plurality of file servers.

5. A method according to claim 3, wherein selectively migrating comprises:

converting a native mode file to an extended mode file stored redundantly over a plurality of file servers.

6. A method according to claim 1, wherein aggregating comprises:

creating a mount point for the native mode volume within the global namespace, the mount point associated with a pathname prefix.

7. A method according to claim 6, wherein allowing client access to files in the at least one native mode volume indirectly via the aggregated global namespace comprises:

receiving a first request for access to a native mode file, the first request including a pathname for the file in the global namespace including the pathname prefix; and

transmitting a second request to a file server hosting the native mode file, the second request including a pathname for the file in the native mode volume without the pathname prefix.

8. A method according to claim 7, further comprising:

receiving a handle from the native mode volume in response to the second request; and

transmitting the handle to the client as a response to the first request.

9. A method according to claim 8, further comprising:

receiving from the client a third request including the handle; and

transmitting the third request to the native mode volume.

10. A method according to claim 9, further comprising:

receiving a reply from the native mode volume in response to the third request; and

transmitting the reply to the client.

11. A method according to claim 7, wherein transmitting the second request comprises:

spoofing between a first network file protocol used by the client and a second network file protocol used by the file server.

12. A method according to claim 7, wherein transmitting the second request comprises:

protocol translation between a first network file protocol used by the client and a second network file protocol used by the file server.

13. A method according to claim 1, further comprising:

maintaining a set of rules for storing files in a plurality of file servers, the rules specifying criteria for storing files using the at least one native mode volume and at least one extended mode volume; and

managing file storage operations according to the set of rules.

14. A method according to claim 13, wherein the rules specify the types of files that may be created in a native mode volume.

15. A method according to claim 14, wherein the rules specify at least one of:

the types of files that are expressly allowed to be created in the native mode volume; and

the types of files that expressly denied from being created in the native mode volume.

16. A method according to claim 14, wherein the rules specify the types of files that may be created in the native mode volume based on at least one of (1) a file suffix and (2) a file size.

17. A method according to claim 13, wherein storing the file according to the set of rules is performed upon receipt of a request to create the file.

18. A method according to claim 13, wherein storing the file according to the set of rules is performed upon receipt of a request to rename the file.

19. A method according to claim 13, wherein storing the file according to the set of rules involves reapplying the set of rules to a pre-existing file.

20. A system for storing files, the system comprising:

a network file manager; and

a plurality of file server nodes supporting a plurality of storage volumes including at least one native mode volume accessible by client nodes directly and indirectly through the network file manager and at least one extended mode volume accessible by client nodes only through the network file manager, wherein the network file manager aggregates a plurality of storage volumes including at least one native mode volume and at least one extended mode volume into a global namespace.

21. A system according to claim 20, wherein the network file manager is configured to selectively migrate files from the native mode volume into an extended mode volume.

22. A system according to claim 21, wherein such selective migration comprises:

converting a native mode file to an extended mode file stored in a fragmented form over a plurality of file servers.

23. A system according to claim 21, wherein such selective migration comprises:

converting a native mode file to an extended mode file stored redundantly over a plurality of file servers.

24. A system according to claim 20, wherein the network file manager is configured to create a mount point for the native mode volume within the global namespace, the mount point associated with a pathname prefix.

25. A system according to claim 24, wherein the network file manager is configured to receive a first request for access to a native mode file, the first request including a pathname for the file in the global namespace including the pathname prefix and transmit a second request to a file server hosting the native mode file, the second request including a pathname for the file in the native mode volume without the pathname prefix.

26. A system according to claim 25, wherein the network file manager is further configured to receive a handle from the native mode volume in response to the second request and transmit the handle to the client as a response to the first request.

27. A system according to claim 26, wherein the network file manager is further configured to receive from the client a third request including the handle and transmit the third request to the native mode volume.

28. A system according to claim 27, wherein the network file manager is further configured to receive a reply from the native mode volume in response to the third request and transmit the reply to the client.

29. A system according to claim 25, wherein transmitting the second request comprises:

spoofing between a first network file protocol used by the client and a second network file protocol used by the file server.

30. A system according to claim 25, wherein transmitting the second request comprises:

protocol translation between a first network file protocol used by the client and a second network file protocol used by the file server.

31. A system according to claim 20, wherein the network file manager is configured to maintain a set of rules for storing files in a plurality of file servers, the rules specifying criteria for storing files using the at least one native mode volume and at least one extended mode volume, and manage file storage operations according to the set of rules.

32. A system according to claim 31, wherein the rules specify the types of files that may be created in a native mode volume.

33. A system according to claim 32, wherein the rules specify at least one of:

the types of files that are expressly allowed to be created in the native mode volume; and

the types of files that expressly denied from being created in the native mode volume.

34. A system according to claim 32, wherein the rules specify the types of files that may be created in the native mode volume based on at least one of (1) a file suffix and (2) a file size.

35. A system according to claim 31, wherein the network file manager is configured to store the file according to the set of rules upon receipt of a request to create the file.

36. A system according to claim 31, wherein the network file manager is configured to store the file according to the set of rules upon receipt of a request to rename the file.

37. A system according to claim 31, wherein the network file manager is configured to selectively reapply the set of rules to a pre-existing file.

38. A method of storing files by a file switch in a switched file system, each file associated with a file type, the method comprising:

maintaining a set of rules for storing files in a plurality of file servers including at least a metadata server and a storage server, the rules defining at least one file type to be stored in the metadata server; and

storing a file based at least in part on the file type associated with the file according to the set of rules, whereby the file is stored in the metadata server only if the file type associated with the file is a file type to be stored in the metadata server.

39. A method according to claim 38, wherein the rules further define a file size threshold for each file type, and wherein the file is stored in the metadata server only if the file type associated with the file is a file type to be stored in the metadata server and the size of the file is below the file size threshold defined for the file type.

40. A method according to claim 39, wherein the file size threshold is the same for all file types.

41. A method according to claim 39, wherein the file size threshold is different for at least two different file types.

42. A method according to claim 38, wherein storing the file according to the set of rules is performed upon receipt of a request to create the file.

43. A method according to claim 38, wherein storing the file according to the set of rules is performed upon receipt of a request to rename the file.

44. A method according to claim 38, wherein storing the file according to the set of rules involves reapplying the set of rules to a pre-existing file.

45. A method of achieving file virtualization in a network including client nodes and file server nodes, the client nodes and the file server nodes using standard network file protocols, the method comprising:

providing a network file manager in communication with the network, the network file manager in a logical data path between the client nodes and the file server nodes, appearing as a client to the file server nodes and as a server to the client nodes, and interacting with both the client nodes and the file server nodes using the standard network file protocols.

46. A method according to claim 45, further comprising:

for each file, establishing and maintaining a virtual path name independent of physical storage location of such file.

47. A method according to claim 45, further comprising:

providing in a single global name space an aggregation of all name spaces residing in the file server nodes.

48. A method according to claim 47, wherein the name spaces relate to at least one native mode volume and at least one extended mode volume.

49. A method according to claim 47, further comprising:

automatically discovering storage resources and integrating the storage resources into the global namespace.

50. A method according to claim 45, further comprising:

aggregating throughput and storage capacity of the file server nodes so as to create a unitary virtual storage system.

51. A method according to claim 45, further comprising:

aggregating different file systems into a single global name space.

52. A method according to claim 51, wherein the different file systems include a CIFS file system and a NFS file system.

53. A method according to claim 45, further comprising:

storing files in a plurality of file servers in a first layout, wherein each file is associated with a global path used by clients and a physical path used for storage of the file; and

changing the layout of the of the files in the plurality of file servers without changing the global paths.