Shared file system

• US 6,697,846 B1
• Filed: 03/20/1998
• Issued: 02/24/2004
• Est. Priority Date: 03/20/1998
• Status: Expired due to Term

• Current Assignee: Dataplow Incorporated
• Original Assignee: Dataplow Incorporated
• Inventors: Soltis, Steven R.
• Primary Examiner(s): Coulter, Kenneth R.
• Assistant Examiner(s): KANG, PAUL H
• Application Number:
  US09/045,340
• Time in Patent Office: 2,167 Days
• Field of Search: 709/232, 709/230, 709/249, 709/201, 709/219, 709/214, 709/245, 709/217, 709/213, 709/215, 709/203, 707/1, 707/10, 711/147
• US Class Current: 709/217
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: Y10S 707/99931 Database or file accessing

Abstract

A shared storage distributed file system is presented that provides users and applications with transparent access to shared data stored on network attached storage devices by utilizing layering techniques to inherit file management functionality from existing file systems. The present invention stores meta-data for the shared data as real-data in a standard, non-modified, client-server distributed file system, such as NFS. In effect, the standard client-server file system acts as a meta-data server. The name space consisting of inode files stored as real-data on the meta-data server acts as the name space for the shared data. Similarly, file attributes of the inode files are utilized as the file attributes of the shared data. By utilizing an existing client-server system as the meta-data server, development time and complexity are greatly reduced, while speed advances in the underlying client-server system may be incorporated without alteration of the present invention. A method for communicating with network attached storage devices over layered file systems is also presented.

Claims

1. An improved file system of the type in which an application program on a client computer requests regular files, and in which a file server having a file server storage device responds to requests for file server regular files from the client computer by communicating file server regular files stored on the file server storage device to the client computer, the file server being capable of maintaining file attributes, consistency, and lock status of the file server regular files, wherein the improvement comprises:

a) a network attached storage device in communication with the client computer and storing shared storage real-data;

b) shared storage meta-data files stored on the file server storage device as file server regular files; and

c) a software program operating on the client computer for responding to requests concerning shared storage regular files from the application program, the software program determining the physical location of the shared storage real-data on the network attached storage device by analyzing the shared storage meta-data files stored and communicated by the file server.

2. The file system of claim 1, wherein some of the shared storage meta-data files are inode files.

3. The file system of claim 2, wherein there is a one-to-one correspondence between inode files and shared storage regular files, and further wherein the inode files are named identically to names used by the application program for the shared storage regular files.

4. The file system of claim 3, wherein

i) the shared storage regular files are presented to the application program in a shared storage directory hierarchy;

ii) the inode files are stored on the file server storage device in an inode directory hierarchy directly corresponding to the shared storage directory hierarchy; and

iii) wherein the software program allows the application program to access, alter, and traverse the inode directory hierarchy whenever the application program requests access to and alteration or traversal of the shared storage directory hierarchy.

5. The file system of claim 3, wherein the software program determines file attributes of the shared storage regular files by determining the file attributes of the associated inode file stored on the file server.

6. The file system of claim 5, wherein the inode files contain a list of extent addresses identifying the physical locations of the associated shared storage real-data on the network attached storage devices.

7. The file system of claim 6, where the network attached storage device is divided into segments, and each extent address identifies data blocks containing shared storage real-data by indicating the segment, block offset of the extent into the segment, and number of blocks in the extent containing shared storage real-data.

8. The file system of claim 6, wherein the inode files contain holes that make the shared storage regular file size mathematically determinable from the inode file size.

9. The file system of claim 6, wherein the inode files further contain a header having file attribute information not maintained by the file server.

10. The file system of claim 9, wherein extent lists for certain large shared storage regular files are stored on the network attached storage device, and wherein the inode file extent lists contain extent addresses identifying the physical locations of the extent lists stored on the network attached storage devices.

11. The file system of claim 8, wherein real-data from certain small shared storage regular files is inserted into the inode file stored on the file server storage device in place of the extent list and hole in order to increase small file efficiency, and wherein the software program further contains a means for determining whether the inode file contains the shared storage real-data.

12. The file system of claim 9, wherein the header is of a fixed size.

13. The file system of claim 1, wherein

i) the network attached storage device is logically divided into segments, and a free-block allocation table associated with each segment is stored on the file server storage device as a file server regular file;

and

ii) the software program accesses the free-block allocation tables for a segment to allocate and de-allocate shared storage real-data on the network attached storage device.

14. The file system of claim 1, further comprising an installable file system interface to handle communications between the application program and the software program, and handling communications between the software program and the file server.

15. The file system of claim 1, wherein

i) the network attached storage device is logically divided into segments, and a free-block allocation table associated with each segment is divided into a header and an extent list of addresses stored on the file server as a file server regular file, the extent list of addresses identifying the physical locations of bitmap tables stored on the network attached storage device;

and

ii) the software program accesses both the allocation table header and bitmap tables for a segment to allocate and de-allocate file data on the network attached storage device.

16. The file system of claim 1, wherein the file server utilizes an NFS file system.

17. The file system of claim 1, wherein the file server utilizes an SMB based distributed file system.

18. The file system of claim 1, wherein the file server utilizes a DFS file system.

19. The file system of claim 1, further comprising at least one additional network attached storage device containing shared storage real-data.

20. The file system of claim 19, wherein the application program is presented with a common name space for shared storage real-data on multiple devices.

21. The file system of claim 20, wherein a table of network attached storage devices is stored on the file server.

22. The file system of claim 1, wherein the network attached storage device communicates with the client computer through a storage area network.

23. The file system of claim 22, wherein the storage area network utilizes a Fibre Channel interface.

24. The file system of claim 22, wherein the storage area network utilizes a SCSI interface.

25. The file system of claim 23, wherein the network attached storage device utilizes SCSI protocols.

26. The file system of claim 22, wherein the storage area network utilizes a HiPPI interface.

27. The file system of claim 26, wherein the network attached storage device utilizes IPI protocols.

28. The file system of claim 1, wherein the network attached storage device and the file server communicate to the client computer over the same physical network.

29. A shared storage distributed file system comprising:

a) a network attached storage device containing real-data;

b) a meta-data file server;

c) a local storage device directly accessible by the meta-data file server, the local storage device having an inode file associated with the real-data on the network attached storage device, the inode file being treated by the meta-data file server as a regular file and containing extent addresses pointing to physical locations of the associated real-data on the network attached storage device;

d) a client computer in data communication with the network attached storage device and the meta-data file server;

e) an application program operating on the client computer; and

f) a software program operating on the client computer for responding to requests for the real-data from the application program by requesting the inode file associated with the real-data from the meta-data file server, analyzing the received inode file to determine the physical locations of the real-data on the network attached storage device, and requesting from the network attached storage device the data stored at least one of the physical locations indicated by the inode file.

30. The shared storage distributed file system of claim 29, wherein the software program further responds to requests to alter the real-data from the application program by locking the inode file associated with the real-data, requesting the inode file from the meta-data server, analyzing the received inode file to determine the physical locations of the real-data on the network attached storage device, altering the data stored on at least one of the physical locations indicated by the inode file, altering the inode file to reflect changes made to the real-data, storing the altered inode file on the meta-data file server, and unlocking the inode file.

31. A shared storage distributed file system comprising:

a) a storage area network;

b) a network attached storage device on the storage area network, the network attached storage device containing real-data of shared storage regular files and having

i) means for storing particular data received over the storage area network at a specified physical location; and

ii) means for sending the particular data over the storage area network in response to a request for data stored at the specified physical location;

c) a local area network;

d) a file server on the local area network having

i) a storage device for storing meta-data files associated with the shared storage regular files;

ii) means for retrieving and transmitting over the local area network requested meta-data file data stored on the storage device when a request is received;

e) a client computer on the local area network and the storage area network;

the client computer having

i) an application program with a means for requesting data contained in shared storage regular files and a means for requesting changes to the shared storage regular files;

ii) a shared storage distributed file system program having

(1) means for receiving requests concerning shared storage regular files;

(2) means for requesting meta-data files stored on the file server storage device over the local area network;

(3) means for analyzing the file received from the file server as meta-data to determine the physical location of the requested shared storage real-data;

(4) means for requesting data stored at the determined physical location from the network attached storage device over the storage area network;

(5) means for presenting to the application program the data received from the network attached storage device; and

(6) means for altering the data stored at the determined physical locations.

32. The file system of claim 31, wherein the file server further comprises a file locking mechanism capable of maintaining file consistency among clients using the mechanism.

33. A method for providing access to file data stored on a network attached storage device across a storage area network, the method comprising:

a) layering a shared storage file system on top of an existing, complete client-server file system by utilizing the client-server file server as a meta-data file server;

b) communicating with an application program running on a client computer through a shared storage file system client program also running on the client computer;

c) obtaining meta-data concerning the file data that is stored on a separate server computer by making real-data requests from the client program to the meta-data file server;

d) analyzing at the client program the received meta data to determine physical data blocks on the network attached storage device that contain the file data;

e) accessing the data stored at the determined physical blocks by making requests to the network attached storage device from the client program across the storage area network.

34. The method of claim 33, further comprising the step of utilizing the name space and file attributes of the meta-data files stored on the meta-data file server as the name space and file attribute of data files stored on the network attached storage device.

35. A shared storage distributed file system for storing files over a network, each file having real-data and meta-data, with the meta-data containing attributes for the file, the file system comprising:

a) a meta-data file server that stores the meta-data;

b) a network attached storage device that stores the real-data, the network attached storage device being separate from the meta-data file server; and

c) a client computer in network communication with the meta-data file server and the network attached storage device, the client computer having block-level access to the real-data stored on the network attached storage device, and the client computer directly generating and altering the meta-data stored by the meta-data file server.

36. The shared storage distributed file system of claim 35, wherein the network communication with the meta-data file server takes place over a local area network, while the network communication with the network attached storage device takes place over a storage area network.

37. A shared storage distributed file system for storing files over a network, each file having real-data and meta-data, with the meta-data containing attributes for the file, the file system comprising:

a) a meta-data file server that stores the meta-data;

b) a network attached storage device that stores the real-data, the network attached storage device being separate from the meta-data file server; and

c) a client computer in network communication with the meta-data file server and the network attached storage device, the client computer having block-level access to the real-data stored on the network attached storage device, and the client computer handling file allocation by retrieving file allocation information from the meta-data file server, allocating and de-allocating space on the network attached storage device, and sending file allocation information back to the meta-data file server.

38. The shared storage distributed file system of claim 37, wherein the network communication with the meta-data file server takes place over a local area network, while the network communication with the network attached storage device takes place over a storage area network.

39. An improved file system comprising:

a) a file server that stores data as file server regular files and that maintains a namespace for the file server regular files;

b) a network attached storage device;

c) a software program operating on a client computer of the file server,

i) the software program responding to requests for client regular files,

ii) the software program storing real-data on the network attached storage device,

iii) the software program maintaining one inode for each client file, each inode being stored on the file server as a separate file server regular file and having a name in the file server namespace.

40. The file system of claim 39, wherein the name of the client regular file is identical to the name of the associated inode file stored by the file server according to the file server namespace.

41. The file system of claim 40, wherein

i) the inode files are stored on the file server in an inode directory hierarchy;

ii) the client regular files are presented to an application program in a client directory hierarchy directly corresponding to the inode directory hierarchy; and

iii) wherein the software program allows the application program to access, alter, and traverse the inode directory hierarchy whenever the application program requests access, alteration, and traversal of the client directory hierarchy.

42. The file system of claim 39, wherein the file server maintains file attributes for the file server regular files, and the software program determines file attributes of the client regular files by determining the file attributes of the associated inode file stored on the file server.

43. The file system of claim 39, wherein the file server maintains lock status information for the file server regular files, and the software program determines lock status of the client regular files by determining the lock status of the associated inode file stored on the file server.