Large file support for a network file server

• US 7,831,641 B2
• Filed: 04/26/2004
• Issued: 11/09/2010
• Est. Priority Date: 04/24/2003
• Status: Expired due to Fees

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: NeoPath Networks, Inc. (Cisco Systems, Inc.)
• Inventors: Iyengar, Anand, Wong, Thomas K., Chawla, Rajeev, Tsirigotis, Panagiotis
• Primary Examiner(s): Wong; Don
• Assistant Examiner(s): Dang; Thanh-Ha
• Application Number:
  US10/832,785
• Publication Number:
  US 20040267831A1
• Time in Patent Office: 2,388 Days
• Field of Search: 707/205, 707/10, 707/100, 707/822, 707/827, 707/828, 709/201, 709/226, 709/203, 709/219, 709/221
• US Class Current: 707/822
• CPC Class Codes: G06F 16/1824 implemented using Network-attached Storage [NAS] architecture distributed or networked storage systems G06F3/067; protocols for distributed storage of data in a network H04L67/1097
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04L 9/40 Network security protocols

Abstract

A NAS switch provides large file support to a file server in a decentralized storage network such as a NAS (Network Attached Storage) storage network. For example, files greater than 2-GB can be stored on a 32-bit commodity file server. The NAS switch sits in the data path of a client on the front end and a commodity NAS file server on the back end. A segmentation module in the NAS switch stores large files as separate data chunks in the file server. To do so, the segmentation module stores a directory file handle, which points to a directory containing the data chunks, in place of the large file. The segmentation module can also store a large file/chunk directory association in a migration cache. A reconstruction module processes client requests concerning large files by issuing requests to specific data chunks. For example, in a read operation, the reconstruction module calculates chunk numbers to determine which file to read and offsets to determine which byte to read within a chunk.

Claims

1. A method comprising:

receiving at a network device a request from a client to store a large file;

determining the large file has a file size larger than a native processing capability of a network attached storage (NAS) file server;

after determining the large file has the file size larger than the native processing capability of the NAS file server, segmenting the large file into a plurality of data chunks, each of the plurality of data chunks having a chunk size that is smaller than or equal to the native processing capability of the NAS file server;

storing the plurality of data chunks in a directory on the NAS file server;

storing a directory file handle, the directory file handle referring to a location of the directory on the NAS file server, on the network device;

providing a switch file handle to the client, which is mapped to the directory file handle;

receiving at the network device a client request that includes the switch file handle;

at the network device, accessing the plurality of data chunks of the large file from the directory on the NAS file server based on the mapping between the received switch file handle and the directory file handle;

determining whether the received client request is requesting access to the large file; and

adding an entry to a file handle migration table that correlates the switch file handle that is provided to the client with the directory file handle, wherein the directory file handle is further associated with a predetermined size,

wherein the determining whether the received client request is requesting access to the large file comprises;

identifying the switch file handle of the received client request in the file handle migration table to determine whether the switch file handle correlates with the directory file handle and the predetermined size, and

based on identifying the switch file handle of the received client request in the file handle migration table, determining the received client request is requesting access to the large file.

2. The method of claim 1, wherein the storing the plurality of data chunks further comprises:

naming the directory in which the plurality of data chunks are stored on the NAS file server based on a name of the large file; and

naming the plurality of data chunks based on the name of the large file.

3. The method of claim 2, wherein the naming the plurality of data chunks comprises:

determining a number of the plurality of data chunks by dividing the large file size by a preset data chunk size; and

for each of the plurality of data chunks, appending a data chunk number to a directory name, the directory name corresponding to the directory in which the plurality of data chunks are stored on the NAS file server.

4. The method of claim 1, wherein the accessing the plurality of data chunks further comprises:

determining a name of the directory from a name of the large file; and

determining a name for each of the plurality of data chunks from the name of the large file.

5. The method of claim 4, wherein determining the name for each of the plurality of data chunks comprises:

for each of the plurality of data chunks, determining a data chunk number by dividing a file offset for the data chunk by a preset data chunk size; and

appending the data chunk number for the data chunk to the directory name for the data chunk.

6. The method of claim 1, further comprising:

adding an entry to a large file table, wherein the determining whether the received client request is requesting access to the large file includes identifying a file handle corresponding to the received client request in the large file table.

7. The method of claim 1, wherein the received client request specifies to read the large file, write to the large file, or truncate the large file.

8. The method of claim 1, wherein the NAS file server'"'"'s native capability comprises a 32-bit addressing capability, and wherein the large file size is greater than approximately 2-GB.

9. The method of claim 1, wherein the NAS file server'"'"'s native capability comprises a NAS protocol which supports a file size of up to approximately 2-GB, and wherein the large file size is greater than approximately 2-GB.

10. A Network Attached Storage (NAS) switch system comprising:

one or more processors; and

one or more memory, wherein at least one of the one or more processors and memory are configured to perform the following operations;

receiving at a network device a request from a client to store a large file;

determining that the large file has a file size larger than a native processing capability of a NAS file server;

after determining the large file has the file size larger than the native processing capability of the NAS file server, segmenting the large file into a plurality of data chunks, each of the plurality of data chunks having a chunk size that is smaller than or equal to the native processing capability of the NAS file server;

storing the plurality of data chunks in a directory on the NAS file server;

storing a directory file handle, the directory file handle referring to a location of the directory on the NAS file server, on the network device;

providing a switch file handle to the client, which is mapped to the directory file handle;

receiving at the network device a client request that includes the switch file handle;

at the network device, accessing the plurality of data chunks of the large file from the directory on the NAS file server based on the mapping between the received switch file handle and the directory file handle;

determining whether the received client request is requesting access to the large file; and

adding an entry to a file handle migration table that correlates the switch file handle that is provided to the client with the directory file handle, wherein the directory file handle is further associated with predetermined size, and wherein determining whether the received client request is requesting access to the large file is accomplished by;

identifying the switch file handle of the received client request in the file handle migration table to determine whether the switch file handle correlates with the directory file handle and the predetermined size, and

based on identifying the switch file handle of the received client request in the file handle migration table, determining the received client request is requesting access to the large file.

11. The NAS switch system of claim 10, wherein the storing the large file is accomplished by:

naming the directory in which the plurality of data chunks are stored on the NAS file server with a directory name based on a name of the large file, and

naming the plurality of data chunks based on the name of the large file.

12. The NAS switch system of claim 11, wherein the naming the plurality of data chunks is accomplished by:

determining a number of data chunks by dividing the large file size by a data chunk size, and

for each of the plurality of data chunks, appending a data chunk number to the directory name for the directory in which the plurality of data chunks are stored.

13. The NAS switch system of claim 10, wherein accessing the plurality of data chunks is accomplished by determining a name of the directory based on a name of the large file, and determining a name for each of the plurality of data chunks based on the name of the large file.

14. The NAS switch system of claim 13, wherein the at least one of the one or more processors and memory are further configured for:

for each data chunk, determining a data chunk number for the data chunk, by dividing a file offset by a preset data chunk size, and

for each data chunk, appending the data chunk number for the data chunk to the directory name such for each data chunk.

15. The NAS switch system of claim 10, wherein the at least one of the one or more processors and memory are further configured for determining whether the received client request is requesting access to the large file.

16. The NAS switch system of claim 10, wherein the received client request specifies to read the large file, write to the large file, or truncate the large file.

17. The NAS switch system of claim 10, wherein the NAS file server comprises a 32-bit capable file server, and wherein the large file size is greater than approximately 2-GB.

18. The NAS switch system of claim 10, wherein the file server has a NAS protocol which supports a file size of up to approximately 2-GB, and wherein the large file size is greater than approximately 2-GB.

20. The computer program product of claim 10, wherein the storing the plurality of data chunks further comprises:

naming the directory in which the plurality of data chunks are stored on the NAS file server based on a name of the large file; and

naming the plurality of data chunks based on the name of the large file.

19. A computer program product, comprising:

a computer-readable storage medium having computer program instructions and data embodied thereon comprising;

receiving at a network device a request from a client to store a large file;

determining that the large file has a file size larger than a native processing capability of a network attached storage (NAS) file server;

after determining that the large file has the file size larger than the native processing capability of the NAS file server, segmenting the large file into a plurality of data chunks, each of the plurality of data chunks having a chunk size that is smaller than or equal to the native processing capability of the NAS file server;

storing the plurality of data chunks in a directory on the NAS file server;

storing a directory file handle, the directory file handle referring to a location of the directory on the NAS file server, on the network device;

providing a switch file handle to the client, the switch file handle being mapped to the directory file handle;

receiving at the network device a client request that includes the switch file handle;

at the network device, accessing the plurality of data chunks of the large file from the directory on the NAS file server based on the mapping between the received switch file handle and the directory file handle;

determining whether the received client request is requesting access to the large file; and

adding an entry to a file handle migration table that correlates the switch file handle that is provided to the client with the directory file handle, wherein the directory file handle is further associated with predetermined size, and wherein determining whether the received client request is requesting access to the large file is accomplished by;

identifying the switch file handle of the received client request in the file handle migration table to determine whether the switch file handle correlates with the directory file handle and the predetermined size, and

based on identifying the switch file handle of the received client request in the file handle migration table, determining the received client request is requesting access to the large file.

21. The computer program product of claim 19, wherein the accessing the plurality of data chunks further comprises:

determining a name of the directory from a name of the large file; and

determining a name of the plurality of data chunks from the name of the large file.

22. The computer program product of claim 19, further comprising program code for determining whether the received client request is requesting access to the large file.

23. The computer program product of claim 19, wherein the NAS file server'"'"'s native capability comprises a 32-bit addressing capability, and wherein the large file size is greater than approximately 2-GB.

24. The computer program product of claim 19, wherein the NAS file server'"'"'s native capability comprises a NAS protocol which supports a file size of up to approximately 2-GB, and wherein the large file size is greater than approximately 2-GB.

25. A Network Attached Storage (NAS) system comprising:

means for receiving at a network device a client request to store a large file;

means for determining the large file has a file size larger than a native processing capability of a NAS file server;

means for segmenting the large file into a plurality of data chunks, each of the plurality of data chunks having a chunk size that is smaller than or equal to the native processing capability of the NAS file server, after determining the large file has the file size larger than the native processing capability of the NAS file server;

means for storing the plurality of data chunks in a directory on the NAS file server;

means for storing a directory file handle, the directory file handle referring to a location of the directory on the NAS file server, on the network device;

means for providing a switch file handle to the client, the switch file handle being mapped to the directory file handle;

means for receiving at the network device a client request that includes the switch file handle; and

means for accessing the plurality of data chunks of the large file from the directory on the NAS file server based on the mapping between the received switch file handle and the directory file handle at the network device;

means for determining whether the received client request is requesting access to the large file; and

means for adding an entry to a file handle migration table that correlates the switch file handle that is provided to the client with the directory file handle, wherein the directory file handle is further associated with predetermined size, and wherein determining whether the received client request is requesting access to the large file is accomplished by;

identifying the switch file handle of the received client request in the file handle migration table to determine whether the switch file handle correlates with the directory file handle and the predetermined size, and

based on identifying the switch file handle of the received client request in the file handle migration table, determining the received client request is requesting access to the large file.