Method and system for effectively communicating file properties and directory structures in a distributed file system

• US 20030177178A1
• Filed: 12/11/2002
• Published: 09/18/2003
• Est. Priority Date: 12/12/2001
• Status: Active Grant

• Current Assignee: Valve Corporation
• Original Assignee: Valve LLC (Valve Corporation)
• Inventors: Jones, Paul David, Newcombe, Christopher Richard, Birum, Derrick Jason, Thompson, Mikel Howard, Ellis, Richard Donald
• Granted Patent: US 7,373,406 B2
• Time in Patent Office: Days
• US Class Current: 709/203
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: H04L 2463/101 applying security measures for digital rights management data processing systems or methods, specially adapted for commerce, e.g. marketing, shopping, billing, auctions or e-commerce G06Q30/00
• CPC Class Codes: H04L 41/0896 Bandwidth or capacity management, i.e. automatically increasing or decreasing capacities flow or congestion control using dynamic resource allocation, e.g. in-call renegotiation, H04L47/76
• CPC Class Codes: H04L 61/4552 Lookup mechanisms between a plurality of directories; Synchronisation of directories, e.g. metadirectories
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 63/123 received data contents, e.g. message integrity
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/306 User profiles
• CPC Class Codes: H04L 67/34 involving the movement of software or configuration parameters network booting or remote initial program loading [RIPL] G06F9/4416
• CPC Class Codes: H04L 67/51 Discovery or management thereof, e.g. service location protocol [SLP] or web services
• CPC Class Codes: H04L 67/53 using third party service providers
• CPC Class Codes: H04L 67/55 Push-based network services
• CPC Class Codes: H04L 67/62 Establishing a time schedule for servicing the requests

Abstract

A method and system for effectively communicating file properties and directory structures within a distributed file system is provided. A manifest that includes properties relating to all of the files used in an application is created. The manifest is distributed to a client before the client has received all of the files or properties associated with an application. The manifest is stored on the client and is used to determine properties and locations of files used within the application. A distributed file system layer is used to receive file and property requests from the application; determine the location of the requested file, or the property using the manifest, and retrieve the property or file using the manifest. A tool may be used to generate the manifest manually or automatically. File system calls within an application may be replaced with calls that use the distributed file system layer. An interpreter may also be used to intercept the file system calls and replace them with calls that use the distributed file system layer.

Claims

1. A method for effectively communicating file properties and directory structures within a distributed file system, comprising:

creating a manifest that includes properties relating to the files used in an application;

initially storing the manifest on a server;

distributing the manifest to a client before the client has received all of the files associated with the application;

storing the manifest on the client;

determining when a request for a property relating to a file in the application is made by the client, and then retrieving the property from the manifest stored on the client.

2. The method of claim 1, wherein the properties in the manifest comprise file properties and directory properties.

3. The method of claim 1, wherein creating the manifest further comprises gathering the properties from the files used in the application.

4. The method of claim 2, wherein the properties relating to the file properties comprise at least one of the following properties:

a file attribute, a file name, a file size, a file permission;

a reference to a parent directory node, a reference to another file node, and an indication on how the file should be treated on the client.

5. The method of claim 2, wherein the properties relating to the directory properties comprise at least one of the following properties:

a directory attribute, a directory name, a reference to a parent directory node, a reference to a child directory node, and a reference to a first file node for the directory.

6. The method of claim 4, wherein the indication on how the file should be treated on the client, further comprises an indication of how the file should be stored.

7. The method of claim 2, further comprising creating a linked structure of information about files and directories of an application.

8. The method of claim 7, further comprising using the linked structure to request files associated with the application.

9. The method of claim 2, further comprising logging each request made by the client.

10. The method of claim 9, further comprising determining when logging is complete.

11. The method of claim 10, wherein determining when logging is complete further comprises determining when at least one of the following occurs:

a set number of resource requests has been logged, a certain amount of time has elapsed, a number of bytes corresponding to the resources requested has reached a certain predetermined level.

12. A system for effectively communicating file properties and directory structures in a distributed file system, comprising:

a server, comprising;

a network connection configured to communicate with the client;

a memory configured to store files associated with an application;

a process arranged to provide files to the client; and

a process configured to send a manifest to the client before the client receives one of the files associated with the application, wherein the manifest includes properties relating to all of the files associated with the application; and

a client, comprising;

a network connection configured to communicate with the server;

a memory configured to store files associated with the application;

a memory configured to store the manifest;

an application having associated files; and

a distributed file system layer configured to perform the following actions;

receive a file request from the application;

determine a location of where the requested file is stored using the manifest, wherein the location comprises a local location and a non-local location;

retrieve the file from the determined location; and

update the manifest when a file changes locations.

13. The system of claim 12, further comprising a developer package configured to generate the manifest file.

14. The system of claim 13, wherein the developer package configured to generate the manifest file further comprises a process configured to parse the application and replace file system calls with calls that use a distributed file system API when it a file system call is encountered within the application.

15. The system of claim 12, wherein file calls within the application are replaced with calls that use the distributed file system layer.

16. The system of claim 12, further comprising an interpreter configured to recognize file system calls within the application and replace the calls with calls to the distributed file system layer.

17. The system of claim 12, further comprising an authenticator configured to authenticate the file request.

18. The system of claim 12, wherein the properties in the manifest comprise at least one of the following properties:

a file attribute, a file name, a file size, a file permission;

a reference to a parent directory node, a reference to another file node, and an indication on how the file should be treated on the client.

19. The system of claim 12, wherein the properties in the manifest comprise at least one of the following properties:

a directory attribute, a directory name, a reference to a parent directory node, a reference to a child directory node, and a reference to a first file node for the directory.

20. A system for effectively communicating file properties and directory structures, comprising:

means for creating a manifest that includes properties relating to files used in an application;

means for distributing the manifest to a client;

downloading the manifest from the server to a client;

means for storing the manifest locally on the client; and

means for determining when a request for a property relating to a file in the application is made by the client, and when;

retrieving the property from the manifest stored on the client.

21. A method for effectively communicating file properties and directory structures, comprising:

receiving a request for a file from an application;

determining when the file is local;

retrieving the file from a local storage device, when local, otherwise retrieving the file from a server and storing the retrieved file locally;

updating a data structure indicating that the file is store locally;

providing the file to the application

22. The method of claim 21, wherein receiving the request for the file from the application, further comprises accessing a distributed file system I/O API.

23. The method of claim 22, wherein accessing the distributed file system I/O API, further comprises replacing file system calls within an application with call that use the distributed file system layer I/O API.

24. The method of claim 22, wherein accessing the distributed file system I/O API, further comprises replacing a standard file system I/O API by linking a distributed file system I/O API.

25. The method of claim 22, wherein accessing the distributed file system I/O API, further comprises intercepting standard file system I/O calls and directing the calls to distributed file system I/O API calls.