Large file support for a network file server

• US 20040267831A1
• Filed: 04/26/2004
• Published: 12/30/2004
• Est. Priority Date: 04/24/2003
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Systems, Inc.
• Inventors: Wong, Thomas K., Chawla, Rajeev, Tsirigotis, Panagiotis, Iyengar, Anand
• Granted Patent: US 7,831,641 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/1824 implemented using Network-attached Storage [NAS] architecture distributed or networked storage systems G06F3/067; protocols for distributed storage of data in a network H04L67/1097
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04L 9/40 Network security protocols

Abstract

A NAS switch provides large file support to a file server in a decentralized storage network such as a NAS (Network Attached Storage) storage network. For example, files greater than 2-GB can be stored on a 32-bit commodity file server. The NAS switch sits in the data path of a client on the front end and a commodity NAS file server on the back end. A segmentation module in the NAS switch stores large files as separate data chunks in the file server. To do so, the segmentation module stores a directory file handle, which points to a directory containing the data chunks, in place of the large file. The segmentation module can also store a large file/chunk directory association in a migration cache. A reconstruction module processes client requests concerning large files by issuing requests to specific data chunks. For example, in a read operation, the reconstruction module calculates chunk numbers to determine which file to read and offsets to determine which byte to read within a chunk.

Claims

1. A method for storing files on a file server in a decentralized storage network, comprising:

storing a large file as a plurality of data chunks in a directory on the file server, the large file having a size beyond a native processing capability of the file server;

storing a directory file handle on the file server that refers to a location of the directory;

mapping a client request to access the large file to the directory file handle; and

sending a data chunk request to the file server.

2. The method of claim 1, wherein the storing the large file further comprises:

naming the directory based on a name of the large file; and

naming the plurality of data chunks based on the name of the large file.

3. The method of claim 2, wherein the naming the plurality of data chunks comprises:

determining a number of data chunks by dividing the large file size by a preset data chunk size; and

appending a data chunk number to the directory name for each of the data chunks.

4. The method of claim 1, wherein the sending the data chunk request further comprises:

determining a name of the directory from a name of the large file; and

determining a name of data chunks from the name of the large file.

5. The method of claim 4, wherein determining the name of data chunks comprises:

determining a data chunk number by dividing a file offset by a preset data chunk size; and

appending the data chunk number to the directory name to the data chunk number.

6. The method of claim 1, further comprising determining whether a client request is the client request to access the large file.

7. The method of claim 6, further comprising:

adding an entry to a large file table,

wherein the determining whether the client request comprises the client request to access the large file comprises looking-up a file handle corresponding to the client request in the large file table.

8. The method of claim 6, further comprising:

adding an entry to a file handle migration table that correlates a switch file handle for issue to a client with a file handle of a stored file handle that refers to a directory, wherein stored file handles have a predetermined size;

wherein the determining whether the client request comprises the client request to access the large file comprises;

looking-up a file handle corresponding to the client request in the file handle migration table; and

determining whether the file handle refers to a stored file handle based on a size of the file handle.

9. The method of claim 1, wherein the request comprises one from the group consisting of:

read the large file, write to the large file, and truncate the large file.

10. The method of claim 1, wherein the file server native capability comprises a 32-bit addressing capability, and wherein the large file size is greater than approximately 2-GB.

11. The method of claim 1, wherein the file server native capability comprises a NAS file server having a NAS protocol capable of supporting a file size of up to approximately 2-GB, and wherein the large file size is greater than approximately 2-GB.

12. A NAS switch to store files on a file server in a decentralized storage network, comprising:

a segmentation module to store a large file as a plurality of data chunks in a directory on the file server, the large file having a size beyond a native processing capability of the file server, and the segmentation module storing a directory file handle on the file server that refers to a location of the directory;

a reconstruction module to map a client request to access the large file to the directory file handle; and

a file server interface to send a data chunk request to the file server.

13. The NAS switch of claim 12, wherein the segmentation module names the directory based on a name of the large file, and names the plurality of data chunks based on the name of the large file.

14. The NAS switch of claim 13, wherein the segmentation module determines a number of data chunks by dividing the large file size by a data chunk size, and appends a data chunk number to the directory name for each of the data chunks.

15. The NAS switch of claim 12, wherein the reconstruction module determines a name of the directory based on a name of the large file, and determines a name of data chunks based on the name of the large file.

16. The NAS switch of claim 15, wherein the reconstruction module determines a data chunk number by dividing a file offset by a preset data chunk size, and appends the data chunk number to the directory name to the data chunk number.

17. The NAS switch of claim 12, wherein the reconstruction module determines whether a client request comprises the client request to access the large file.

18. The NAS switch of claim 17, wherein the segmentation module adds an entry to a large file table, wherein the reconstruction module determines whether the client request comprises the client request to access the large file comprises looking-up a file handle corresponding to the client request in the large file table.

19. The NAS switch of claim 18, wherein the segmentation module adds an entry to a file handle migration table that correlates a switch file handle for issue to a client with a file handle of a stored file handle that refers to a directory, wherein stored file handles have a predetermined size, and wherein the reconstruction module determines whether the client request is the client request to access the large file by looking-up a file handle corresponding to the client request in the file handle migration table and determining whether the file handle refers to a stored file handle based on a size of the file handle.

20. The NAS switch of claim 12, wherein the request comprises one from the group consisting of:

read the large file, write to the large file, and truncate the large file.

21. The NAS switch of claim 12, wherein the file server comprises a 32-bit capable file server, and wherein the large file size is greater than approximately 2-GB.

22. The NAS switch of claim 12, wherein the file server comprises a NAS file server having a NAS protocol capable of supporting a file size of up to approximately 2-GB, and wherein the large file size is greater than approximately 2-GB.

23. A computer program product, comprising:

a computer-readable medium having computer program instructions and data embodied thereon for a method of storing files on a file server in a decentralized storage network, comprising;

storing a large file as a plurality of data chunks in a directory on the file server, the large file having a size beyond a native processing capability of the file server;

storing a directory file handle on the file server that refers to a location of the directory;

mapping a client request to access the large file to the directory file handle; and

sending a data chunk request to the file server.

24. The computer program product of claim 23, wherein the storing the large file further comprises:

naming the directory based on a name of the large file; and

naming the plurality of data chunks based on the name of the large file.

25. The computer program product of claim 23, wherein the sending the data chunk request further comprises:

determining a name of the directory from a name of the large file; and

determining a name of data chunks from the name of the large file.

26. The computer program product of claim 23, further comprising determining whether a client request comprises the client request to access the large file.

27. The computer program product of claim 23, wherein the file server native capability comprises a 32-bit addressing capability, and wherein the large file size is greater than approximately 2-GB.

28. The computer program product of claim 23, wherein the file server native capability comprises a NAS file server having a NAS protocol capable of supporting a file size of up to approximately 2-GB, and wherein the large file size is greater than approximately 2-GB.

29. A NAS switch to store files on a file server in a decentralized storage network, comprising:

a means for segmenting storing a large file as a plurality of data chunks in a directory on the file server, the large file having a size beyond a processing native capability of the file server, and the segmentation module storing a directory file handle on the file server that refers to a location of the directory;

a means for reconstructing storing a client request to access the large file to the directory file handle; and

a means for file server interfacing sending a data chunk request to the file server.