Shared file system

• US 7,743,111 B2
• Filed: 12/16/2003
• Issued: 06/22/2010
• Est. Priority Date: 03/20/1998
• Status: Active Grant

• Current Assignee: Dataplow Incorporated
• Original Assignee: Data PLOW Incorporated
• Inventors: Soltis, Steven
• Primary Examiner(s): Kang; Paul H
• Application Number:
  US10/738,371
• Publication Number:
  US 20040133570A1
• Time in Patent Office: 2,380 Days
• Field of Search: 709/203, 709/213, 709/214, 709/215, 709/217, 709/219, 709/245, 711/147, 707/1, 707/10, 707/203
• US Class Current: 709/217
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: Y10S 707/99931 Database or file accessing

Abstract

A shared storage distributed file system is presented that provides users and applications with transparent access to shared data stored on network attached storage devices by utilizing layering techniques to inherit file management functionality from existing file systems. The present invention stores meta-data for the shared data as real-data in a standard, non-modified, client-server distributed file system, such as NFS. In effect, the standard client-server file system acts as a meta-data server. The name space consisting of inode files stored as real-data on the meta-data server acts as the name space for the shared data. Similarly, file attributes of the inode files are utilized as the file attributes of the shared data. By utilizing an existing client-server system as the meta-data server, development time and complexity are greatly reduced, while speed advances in the underlying client-server system may be incorporated without alteration of the present invention. A method for communicating with network attached storage devices over layered file systems is also presented.

Claims

1. A distributed file system residing on a plurality of computers, the distributed file system having a namespace defining a directory structure of files and metadata that includes pointers to real-data, the file system comprising:

a) at least one server computer running server software for managing the namespace;

b) a network; and

c) at least one client computer located separately from the server computer and in communication with the server computer over the network, each running client software, the client software

i) issuing namespace requests to the server computer over the network, each namespace request consisting of a filename and an operation encoded into a protocol interpretable by the server computer, the namespace requests selected from the group consisting of requests to add new filenames to the namespace, requests to remove existing filenames from the namespace, and requests to search the namespace for filenames, and

ii) directly retrieving, analyzing, and altering the metadata.

2. The file system of claim 1, wherein metadata includes allocation tables that store information identifying data as allocated and not allocated.

3. The file system of claim 1, wherein the client software directly generates metadata pointers to real-data.

4. The file system of claim 1, wherein the server software enforces file access permissions.

5. The file system of claim 1, wherein the server software manages the namespace in response to namespace requests from the client computer, including requests to read a directory from the namespace.

6. The file system of claim 5, wherein the namespace search for filenames returns information necessary to retrieve the metadata.

7. The file system of claim 6, wherein the server software enforces file permissions during the namespace search.

8. The file system of claim 1, wherein the client computer directly responds to file system requests concerning a file from an application program.

9. The file system of claim 1, wherein the file system is a shared storage distributed file system in which the client computer directly accesses the shared storage device.

10. A network of connected computing devices for implementing a shared storage distributed file system, the file system having a namespace, real-data, and metadata, the network comprising:

a) a network storage device connected to a the network;

b) a server computer that manages the namespace by retrieving and modifying directory files in response to namespace requests, including requests to add new filenames to directory files and to remove existing filenames from directory files; and

c) a client computer in network communication with the server computer and the network storage device, wherein the client computer

i) issues namespace requests to the server computer over the network communication,

ii) reads and writes the real-data directly from the network storage device, and

iii) creates, retrieves, and modifies the metadata.

11. The network of claim 10, wherein the client computer communicates with the server computer via a local area network, and the client computer communicates with the network storage device via a storage area network.

12. The network of claim 11, wherein namespace requests are communicated via the local area network.

13. The network of claim 11, wherein the client reads and writes the real-data via the storage area network.

14. The network of claim 10, wherein the client computer requests file attributes from the server computer.

15. The network of claim 14, wherein file attributes axe communicated via the local area network.

16. The network of claim 10, wherein the client computer acquires a lock prior to modifying the metadata.

17. The network of claim 10, wherein the namespace requests include requests to search the namespace for filenames.

18. The network of claim 17, wherein the server computer enforces file access permissions during the namespace search.

19. The network of claim 10, wherein the client computer stores the metadata by requesting that the metadata be stored by the server computer.

20. The network of claim 19, wherein the server computer stores the metadata within a real-data file.

21. The network of claim 19, wherein the server computer is operating as the server of a client-server file system to store the metadata.

22. The network of claim 19, wherein the server computer stores the metadata on a server storage device locally attached to the server computer.

23. The network of claim 19, wherein the server computer stores the metadata on a server storage device different from the network storage device.

24. An improved file system operating as part of a client computer comprising:

a) a digital communication path to a storage device;

b) a network communication path to a server computer operating a server software program that maintains a namespace by receiving a namespace request from the client computer, reading the directory file corresponding to the request from the storage device, modifying the directory file, and writing the directory file to the storage device; and

c) a client software program that runs on the client computer that responds to file system requests from an application program concerning a file, wherein the client software

i) obtains addressing metadata containing at least one pointer addressing real-data for the file,

ii) uses the addressing metadata to locate real-data associated with the file on the storage device,

iii) alters the addressing metadata for the file; and

iv) submits namespace requests across the network communication path to the server computer.

25. The improved file system of claim 24, wherein the server software adds new filenames to the namespace, removes existing filenames from the namespace, and searches the namespace for filenames.

26. The improved file system of claim 25, wherein the client software sends a namespace search request to the server software in order to obtain addressing metadata.

27. The improved file system of claim 26, wherein the server software enforces file access permissions during the namespace search.

28. The improved file system of claim 26, wherein the addressing metadata is found within an in ode obtained by the client software.

29. The improved file system of claim 24, wherein the client software further obtains allocation table metadata concerning allocation of storage on the storage device and modifies the allocation table metadata when performing file allocation and de-allocation.

30. The improved file system of claim 29, wherein the allocation table metadata is a bitmap table.

31. The improved file system of claim 24, wherein the addressing metadata is found within an mode obtained by the client software.

32. An improved file system operating on a client computer and a server computer to access a storage device containing real-data and one or more direct pointers addressing the real-data, the improved file system comprising:

a) a server software program residing on the server computer, the server software program

i) maintains a namespace defining a directory structure of files on the storage device,

ii) services namespace requests from the client computer, and

iii) stores an indirect pointer within the namespace related to a file, the indirect pointer addressing at least one file related direct pointer on the storage device; and

b) a client software program residing on the client computer, the client computer being located separately from the server computer and issuing namespace requests to the server computer, the client software program responds to file system requests from an application program concerning the file;

the client software program further

i) obtains the indirect pointer for the file from the server software program,

ii) uses the indirect pointer to obtain the file related direct pointer directly from the storage device, and

iii) uses the file related direct pointer to read and write real-data associated with the file directly from the storage device.

33. The improved file system of claim 32, wherein the client software program modifies the file related direct pointer during file write operations.

34. The improved file system of claim 33, wherein the client software acquires a lock prior to modifying the file related direct pointer.

35. The improved file system of claim 32, wherein the namespace requests from the client computer include requests to add new filenames to the namespace, to remove existing filenames from the namespace, and to search the namespace for filenames.

36. The improved file system of claim 32, wherein the client software further obtains allocation table metadata concerning allocation of storage on the storage device and updates the allocation metadata when performing file allocation and de-allocation.

37. The improved file system of claim 36, wherein the client software acquires a lock prior to updating the allocation table metadata.

38. A client computer having an improved file system, the file system providing access to data stored on a storage device, comprising:

a) a network connection to a server computer that is located separately from the client computer and that maintains a namespace defining a directory structure of files on the storage device, and

b) client software residing and operating on the client computer that

i) obtains allocation information concerning the allocation of storage on the storage device,

ii) alters the allocation information for regular files during file allocation and de-allocation, and

iii) submits namespace requests to the server computer over the network connection;

wherein the alteration of the allocation information is performed in response to a request by an application program running on the client computer.

39. The improved file system on the client computer of claim 38, wherein the server software accesses and modifies the directory structure in response to namespace requests from the client computer, including requests to add new filenames to the namespace, to remove existing filenames from the namespace, and to search the namespace for filenames.

40. The improved file system on the client computer of claim 38, wherein the client software acquires a lock prior to obtaining the allocation information.

41. The improved file system on the client computer of claim 38, wherein the client software further:

iii) obtains addressing metadata locating real-data for a particular file,

iv) uses the addressing metadata to access real-data associated with the particular file on the storage device, and

v) alters the addressing metadata for the file.

42. The improved file system on the client computer of claim 41, wherein the client software acquires a lock prior to altering the addressing metadata.

43. The improved file system on the client computer of claim 38, wherein the allocation information is obtained from the storage device and the altered allocation information is stored on the storage device.

44. A network of connected computer devices comprising:

a) a first computer running software for

i) managing a namespace of files, and

ii) servicing namespace requests, the namespace requests including requests to add filenames to the namespace, remove filenames from the namespace, and search the namespace; and

b) a second computer located separately from the first computer and running software for

i) submitting to the first computer namespace requests relating to a file request and

ii) analyzing and altering metadata relating to the file request, the metadata including pointers to real-data.

45. The network of claim 44, wherein the metadata includes data allocation information.

46. The network of claim 44, wherein the second computer directly responds to file requests from an application program.

47. The network of claim 44, wherein the first computer enforces file access permissions for requests received from the second computer.

48. The network of claim 47, wherein the first computer enforces file access permissions while adding a filename to a directory.

49. The network of claim 47, wherein the first computer enforces file access permissions while removing a filename from a directory.

50. The network of claim 47, wherein the first computer enforces file access permissions while searching for a filename within a directory.

51. A method for handling file requests by a file system comprising:

a) receiving the file request at a client computer;

b) requesting at the client computer that a server computer which manages a namespace for the file system perform a namespace search for the file of the request, the server computer being located separately from the client computer;

c) analyzing and altering metadata relating to the location of real-data of the file request at the client computer; and

d) saving on a storage device the metadata altered by the client computer.

52. The method of claim 51, wherein a lock is acquired prior to altering the metadata relating to the location of real-data.

53. A method for handling file requests by a file system comprising

a) receiving the file request at a client computer;

b) requesting at the client computer namespace data relating to the file request from a server computer, the server computer managing a namespace for the file system and the server computer also being located separately from the client computer;

c) analyzing and altering metadata at the client computer, the metadata relating to the allocation information of storage on a storage device; and

d) saving on the storage device the allocation information metadata altered by the client computer.

54. The method of claim 53, wherein the allocation information consists of bitmap tables.

55. The method of claim 53, wherein a lock is acquired prior to analyzing and altering allocation information metadata, and the lock is released after saving the allocation information.

56. A computer product, comprising:

a non-transitory computer readable medium having a computer program code embodied therein, said computer program code adapted to be executed to implement a method for handling file requests by a file system, the method comprising;

a) receiving the file request at a client computer;

b) requesting at the client computer that a server computer which manages a namespace for a file system perform a namespace search for the file of the request, the server computer being located separately from the client computer;

c) analyzing and altering metadata relating to the location of real-data of the file request at the client computer; and

d) saving the metadata altered by the client computer.