File system performance enhancement

• US 6,389,427 B1
• Filed: 05/28/1999
• Issued: 05/14/2002
• Est. Priority Date: 02/20/1998
• Status: Expired due to Fees

• Current Assignee: Redleaf Group, Inc.
• Original Assignee: Redleaf Group, Inc.
• Inventors: Faulkner, Michael R.
• Primary Examiner(s): Breene, John
• Assistant Examiner(s): BLACK, LINH
• Application Number:
  US09/322,194
• Time in Patent Office: 1,082 Days
• Field of Search: 707/1, 707/2, 707/102, 707/200, 707/201, 707/100-103, 707/104.1, 707/203, 707/205, 707/530
• US Class Current: 707/741
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/16 File or folder operations, e.g. details of user interfaces specifically adapted to file systems
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

Methods and apparatus that enhance the performance of computer file systems, and in particular the performance of read-only operations in such file systems. The invention can be implemented in a suite of computer program modules that together make up a performance enhancement product. The invention can transparently exist in an operating system after an initial setup is completed. The initial setup involves identifying what directories or files are to be monitored in order to intercept access requests for those files and to respond to those requests with enhanced performance. A system administrator can specify what directories or files are to be monitored. A high-performance index of monitored directories or files is maintained. When a monitored file is opened, a file identifier used, thereby bypassing the access of any directory meta data information. In one embodiment, access to monitored files is enhanced by pinning files in the data cache maintained by the file system cache manager.

Claims

1. A file system performance enhancement product operable to be executed on a computer running a regular file system, the product comprising:

administration means for a user to select some files of the file system as monitored files, the remaining files of the file system being non-monitored files; and

means for bypassing a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the product;

wherein operation of the regular file system is changed only with respect to monitored files;

wherein a request to access a file is passively forwarded to the regular file system if the request is directed to a non-monitored file and the request is controlled by the product if the request is directed to a monitored file; and

wherein the request to access a file is handled entirely by the regular file system if the request is directed to a monitored file and the product is not running on the computer.

2. A file system performance enhancement product operable to be executed on a computer running a regular file system, the product comprising:

administration means for a user to select some files of the file system as monitored files, the remaining files of the file system being non-monitored files; and

means for bypassing a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the product;

wherein operation of the regular file system is changed only with respect to monitored files;

wherein a request to access a file is passively forwarded to the regular file system if the request is directed to a non-monitored file and the request is controlled by the product if the request is directed to a monitored file; and

wherein the administration means is operable to allow the user to cause all file operation requests to be directed to the regular file system driver.

3. A file system performance enhancement product operable to be executed on a computer running a regular file system, the product comprising:

administration means for a user to select some files of the file system as monitored files, the remaining files of the file system being non-monitored files; and

means for bypassing a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the product;

wherein operation of the regular file system is changed only with respect to monitored files;

wherein a request to access a file is passively forwarded to the regular file system if the request is directed to a non-monitored file and the request is controlled by the product if the request is directed to a monitored file;

wherein the administration means is operable to create and maintain registry entries as necessary as the result of user interactions with the administration means; and

wherein the administration means is operable to allow the user to set a toggle that returns a “

file does not exist”

response to requests for monitored files that cannot be found using the lookup data.

4. A file system performance enhancement product operable to be executed on a computer running a regular file system, the product comprising:

administration means for a user to select some files of the file system as monitored files, the remaining files of the file system being non-monitored files; and

means for bypassing a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the product;

wherein operation of the regular file system is changed only with respect to monitored files;

wherein a request to access a file is passively forwarded to the regular file system if the request is directed to a non-monitored file and the request is controlled by the product if the request is directed to a monitored file; and

wherein all requests to monitored files that cannot be found using the lookup data are forwarded to be performed as regular file system operations.

5. A file system performance enhancement product operable to be executed on a computer running a regular file system, the product comprising:

administration means for a user to select some files of the file system as monitored files, the remaining files of the file system being non-monitored files;

means for bypassing a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the product; and

means for regenerating the lookup data whenever the computer is rebooted;

wherein operation of the regular file system is changed only with respect to monitored files; and

wherein a request to access a file is passively forwarded to the regular file system if the request is directed to a non-monitored file and the request is controlled by the product if the request is directed to a monitored file.

6. A file system performance enhancement product operable to be executed on a computer running a regular file system, the product comprising:

administration means for a user to select some files of the file system as monitored files, the remaining files of the file system being non-monitored files;

means for bypassing a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the product; and

means for preventing the regular file system from updating an access time when monitored files are read through the product, the access time being an item of meta data otherwise maintained by the regular file system;

wherein operation of the regular file system is changed only with respect to monitored files; and

wherein a request to access a file is passively forwarded to the regular file system if the request is directed to a non-monitored file and the request is controlled by the product if the request is directed to a monitored file.

7. A file system performance enhancement product operable to be executed on a computer running a regular file system, the product comprising:

administration means for a user to select some files of the file system as monitored files, the remaining files of the file system being non-monitored files; and

means for bypassing a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the product;

wherein operation of the regular file system is changed only with respect to monitored files; and

wherein the means for bypassing a lookup operation uses an open-by-file-ID operation of the regular file system in place of an open-by-file-name operation of the regular file system.

8. A file system performance enhancement product operable to be executed on a computer running a regular file system, the product comprising:

administration means for a user to select some files of the file system as monitored files, the remaining files of the file system being non-monitored files; and

means for bypassing a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the product;

wherein operation of the regular file system is changed only with respect to monitored files; and

wherein the means for bypassing a lookup operation uses a file-ID based vnode access operation of a UNIX virtual file system in place of a corresponding name based access operation.

9. A file system performance enhancement product operable to be executed on a computer running a regular file system, the product comprising:

administration means for a user to select some files of the file system as monitored files, the remaining files of the file system being non-monitored files; and

means for bypassing a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the product;

wherein operation of the regular file system is changed only with respect to monitored files; and

wherein the lookup data comprises a high-performance index of monitored directories and files.

10. A file system performance enhancement product operable to be executed on a computer running a regular file system, the product comprising:

administration means for a user to select some files of the file system as monitored files, the remaining files of the file system being non-monitored files; and

means for bypassing a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the product; and

means for obtaining meta data for monitored files and recording the meta data information for use by the product in bypassing operations of the regular file system;

wherein operation of the regular file system is changed only with respect to monitored files;

wherein the administration means includes means for the user to select a directory as a monitored directory, any files in the monitored directory, or any subdirectory contained within the monitored directory; and

wherein the product operates transparently in conjunction with the regular file system after an initial setup is completed.

11. The product of claim 10, further comprising means for maintaining in random access memory a distinct cache used only for meta data.

12. The product of claim 11, wherein the distinct cache used only for meta data is maintained in a compressed form using run-length encoding compression.

13. The product of claim 10, further comprising means for intercepting access requests made to the regular file system for monitored directories.

14. The product of claim 10, wherein the means for bypassing a lookup operation is operable to bypass directory-level security controls of the regular file system.

15. The product of claim 10, wherein

the regular file system maintains a cache of disk data in random access memory of the computer, the product further comprising means for controlling what data is maintained in the cache to discard data from a least recently used file; and

the administration means includes a computer program application running in user mode and interacting with the user through a graphical user interface.

16. The product of claim 15, wherein

the regular file system is a Windows NT version 4.0 or later file system (NTFS);

and

the means for controlling what data is maintained in the cache comprises means for maintaining a memory-descriptor-list (MDL) read operation on a first monitored file until the first monitored file becomes the least-recently-used file, thereby affecting an NTFS primary file cache and pinning recently-used monitored files in the NTFS primary file cache.

17. The product of claim 16, wherein a read operation is handled without the intervention of the underlying file system by using the memory descriptor list information obtained from pinned data in cache memory as a result of a previously satisfied read operation.

18. A method for enhancing file system performance executed on a computer running a regular file system, the product comprising:

selecting one or more files of the file system as monitored files, the remaining files of the file system being non-monitored files;

maintaining lookup data for all monitored files in a separate enhanced file system;

bypassing a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the separate enhanced file system;

passively forwarding a request to access a file to the regular file system if the request is directed to a non-monitored file and intercepting the request by the separate enhanced file system if the request is directed to a monitored file; and

handling the request to access a file entirely by the regular file system if the request is directed to a monitored file and the separate enhanced file system is not running on the computer;

wherein operation of the regular file system is changed only with respect to monitored files.

19. A method for enhancing file system performance executed on a computer running a regular file system, the product comprising:

selecting one or more files of the file system as monitored files, the remaining files of the file system being non-monitored files;

maintaining lookup data for all monitored files in a separate enhanced file system;

bypassing a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the separate enhanced file system;

passively forwarding a request to access a file to the regular file system if the request is directed to a non-monitored file and intercepting the request by the separate enhanced file system if the request is directed to a monitored file; and

selectively configuring, based user input, the routing of all file operation requests to the regular file system driver;

wherein operation of the regular file system is changed only with respect to monitored files.

20. A method for enhancing file system performance executed on a computer running a regular file system, the product comprising:

selecting one or more files of the file system as monitored files, the remaining files of the file system being non-monitored files;

maintaining lookup data for all monitored files in a separate enhanced file system;

bypassing a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the separate enhanced file system;

passively forwarding a request to access a file to the regular file system if the request is directed to a non-monitored file and intercepting the request by the separate enhanced file system if the request is directed to a monitored file;

creating and maintaining registry entries as necessary as the result of user interactions with the separate enhanced file system; and

selectively configuring the separate enhanced file system to allow the user to set a toggle that returns a “

file does not exist”

response to requests for monitored files that cannot be found using the lookup data;

wherein operation of the regular file system is changed only with respect to monitored files.

21. A method for enhancing file system performance executed on a computer running a regular file system, the product comprising:

selecting one or more files of the file system as monitored files, the remaining files of the file system being non-monitored files;

maintaining lookup data for all monitored files in a separate enhanced file system;

bypassing a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the separate enhanced file system;

passively forwarding a request to access a file to the regular file system if the request is directed to a non-monitored file and intercepting the request by the separate enhanced file system if the request is directed to a monitored file; and

forwarding all requests to monitored files that cannot be found using the lookup data to the regular file system for processing;

wherein operation of the regular file system is changed only with respect to monitored files.

22. A method for enhancing file system performance executed on a computer running a regular file system, the product comprising:

selecting one or more files of the file system as monitored files, the remaining files of the file system being non-monitored files;

maintaining lookup data for all monitored files in a separate enhanced file system;

bypassing a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the separate enhanced file system;

passively forwarding a request to access a file to the regular file system if the request is directed to a non-monitored file and intercepting the request by the separate enhanced file system if the request is directed to a monitored file; and

regenerating the lookup data whenever the computer is rebooted;

wherein operation of the regular file system is changed only with respect to monitored files.

23. A method for enhancing file system performance executed on a computer running a regular file system, the product comprising:

selecting one or more files of the file system as monitored files, the remaining files of the file system being non-monitored files;

maintaining lookup data for all monitored files in a separate enhanced file system;

bypassing a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the separate enhanced file system;

passively forwarding a request to access a file to the regular file system if the request is directed to a non-monitored file and intercepting the request by the separate enhanced file system if the request is directed to a monitored file; and

preventing the regular file system from updating an access time when monitored files are read through the separate enhanced file system, the access time being an item of meta data otherwise maintained by the regular file system;

wherein operation of the regular file system is changed only with respect to monitored files.

24. A method for enhancing file system performance executed on a computer running a regular file system, the product comprising:

selecting one or more files of the file system as monitored files, the remaining files of the file system being non-monitored files;

maintaining lookup data for all monitored files in a separate enhanced file system;

bypassing a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the separate enhanced file system; and

using an open-by-file-ID operation of the regular file system in place of an open-by-file-name operation of the regular file system when bypassing a lookup operation;

wherein operation of the regular file system is changed only with respect to monitored files.

25. A method for enhancing file system performance executed on a computer running a regular file system, the product comprising:

selecting one or more files of the file system as monitored files, the remaining files of the file system being non-monitored files;

maintaining lookup data for all monitored files in a separate enhanced file system;

bypassing a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the separate enhanced file system; and

using a file-ID based vnode access operation of a UNIX virtual file system in place of a corresponding name based access operation when bypassing a lookup operation;

wherein operation of the regular file system is changed only with respect to monitored files.

26. A method for enhancing file system performance executed on a computer running a regular file system, the product comprising:

selecting one or more files of the file system as monitored files, the remaining files of the file system being non-monitored files;

maintaining lookup data for all monitored files in a separate enhanced file system; and

bypassing a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the separate enhanced file system;

wherein operation of the regular file system is changed only with respect to monitored files; and

wherein the lookup data comprises a high-performance index of monitored directories and files.

27. A method for enhancing file system performance executed on a computer running a regular file system, the product comprising:

selecting one or more files of the file system as monitored files, the remaining files of the file system being non-monitored files;

maintaining lookup data for all monitored files in a separate enhanced file system;

bypassing a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the separate enhanced file system;

obtaining meta data for monitored files and recording the meta data information for use by the separate enhanced file system in bypassing operations of the regular file system,

selecting by the user a directory as a monitored directory, any files in the monitored directory, or any subdirectory contained within the monitored directory; and

operating transparently in conjunction with the regular file system after an initial setup is completed;

wherein operation of the regular file system is changed only with respect to monitored files.

28. The method of claim 27, further comprising maintaining in random access memory a distinct cache used only for meta data.

29. The method of claim 28, further comprising maintaining the distinct cache used only for meta data in a compressed form using run-length encoding compression.

30. The method of claim 27, further comprising intercepting access requests made to the regular file system for monitored directories.

31. The method of claim 27, further comprising bypassing directory-level security controls of the regular file system when bypassing a lookup operation.

32. The method of claim 27, further comprising

maintaining by the regular file system a cache of disk data in random access memory of the computer,

controlling by the separate enhanced file system what data is maintained in the cache to discard data from a least recently used file; and

interacting with the user through a graphical user interface using a computer program application running in user mode on the computer.

33. The method of claim 32, wherein the regular file system is a Windows NT version 4.0 or later file system (NTFS), the method further comprising

controlling what data is maintained in the cache including maintaining a memory-descriptor-list (MDL) read operation on a first monitored file until the first monitored file becomes the least-recently-used file, thereby affecting an NTFS primary file cache and pinning recently-used monitored files in the NTFS primary file cache.

34. The method of claim 33, further comprising handling a read operation without the intervention of the underlying file system by using the memory descriptor list information obtained from pinned data in cache memory as a result of a previously satisfied read operation.

46. The method of claim 32, wherein the regular file system is a Windows NT version 4.0 or later file system (NTFS), the method further comprising instructions to

control what data is maintained in the cache including maintaining a memory-descriptor-list (MDL) read operation on a first monitored file until the first monitored file becomes the least-recently-used file, thereby affecting an NTFS primary file cache and pinning recently-used monitored files in the NTFS primary file cache.

47. The method of claim 46, further comprising instructions to handle a read operation without the intervention of the underlying file system by using the memory descriptor list information obtained from pinned data in cache memory as a result of a previously satisfied read operation.

35. A computer implemented method for enhancing file system performance executed on a computer running a regular file system, the method including instructions for causing a computer to:

select one or more files of the file system as monitored files, the remaining files of the file system being non-monitored files;

maintain lookup data for all monitored files in a separate enhanced file system; and

bypass a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the separate enhanced file system;

passively forward a request to access a file to the regular file system if the request is directed to a non-monitored file and intercept the request by the separate enhanced file system if the request is directed to a monitored file; and

selectively configure, based user input, the routing of all file operation requests to the regular file system driver;

wherein operation of the regular file system is changed only with respect to monitored files.

36. A computer implemented method for enhancing file system performance executed on a computer running a regular file system, the method including instructions for causing a computer to:

select one or more files of the file system as monitored files, the remaining files of the file system being non-monitored files;

maintain lookup data for all monitored files in a separate enhanced file system;

bypass a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the separate enhanced file system;

passively forward a request to access a file to the regular file system if the request is directed to a non-monitored file and intercept the request by the separate enhanced file system if the request is directed to a monitored file;

create and maintain registry entries as necessary as the result of user interactions with the separate enhanced file system; and

selectively configure the separate enhanced file system to allow the user to set a toggle that returns a “

file does not exist”

response to requests for monitored files that cannot be found using the lookup data;

wherein operation of the regular file system is changed only with respect to monitored files.

37. A computer implemented method for enhancing file system performance executed on a computer running a regular file system, the method including instructions for causing a computer to:

select one or more files of the file system as monitored files, the remaining files of the file system being non-monitored files;

maintain lookup data for all monitored files in a separate enhanced file system;

bypass a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the separate enhanced file system;

passively forward a request to access a file to the regular file system if the request is directed to a non-monitored file and intercept the request by the separate enhanced file system if the request is directed to a monitored file; and

forward all requests to monitored files that cannot be found using the lookup data to the regular file system for processing;

wherein operation of the regular file system is changed only with respect to monitored files.

38. A computer implemented method for enhancing file system performance executed on a computer running a regular file system, the method including instructions for causing a computer to:

select one or more files of the file system as monitored files, the remaining files of the file system being non-monitored files;

maintain lookup data for all monitored files in a separate enhanced file system;

bypass a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the separate enhanced file system;

passively forward a request to access a file to the regular file system if the request is directed to a non-monitored file and intercept the request by the separate enhanced file system if the request is directed to a monitored file; and

regenerate the lookup data whenever the computer is rebooted;

wherein operation of the regular file system is changed only with respect to monitored files.

39. A computer implemented method for enhancing file system performance executed on a computer running a regular file system, the method including instructions for causing a computer to:

select one or more files of the file system as monitored files, the remaining files of the file system being non-monitored files;

maintain lookup data for all monitored files in a separate enhanced file system;

bypass a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the separate enhanced file system;

passively forward a request to access a file to the regular file system if the request is directed to a non-monitored file and intercept the request by the separate enhanced file system if the request is directed to a monitored file; and

prevent the regular file system from updating an access time when monitored files are read through the separate enhanced file system, the access time being an item of meta data otherwise maintained by the regular file system;

wherein operation of the regular file system is changed only with respect to monitored files.

40. A computer implemented method for enhancing file system performance executed on a computer running a regular file system, the method including instructions for causing a computer to:

select one or more files of the file system as monitored files, the remaining files of the file system being non-monitored files;

maintain lookup data for all monitored files in a separate enhanced file system;

bypass a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the separate enhanced file system; and

use an open-by-file-ID operation of the regular file system in place of an open-by-file-name operation of the regular file system when bypassing a lookup operation;

wherein operation of the regular file system is changed only with respect to monitored files.

41. A computer implemented method for enhancing file system performance executed on a computer running a regular file system, the method including instructions for causing a computer to:

select one or more files of the file system as monitored files, the remaining files of the file system being non-monitored files;

maintain lookup data for all monitored files in a separate enhanced file system;

bypass a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the separate enhanced file system; and

use a file-ID based vnode access operation of a UNIX virtual file system in place of a corresponding name based access operation when bypassing a lookup operation;

wherein operation of the regular file system is changed only with respect to monitored files.

42. A computer implemented method for enhancing file system performance executed on a computer running a regular file system, the method including instructions for causing a computer to:

select one or more files of the file system as monitored files, the remaining files of the file system being non-monitored files;

maintain lookup data for all monitored files in a separate enhanced file system;

bypass a lookup operation of the regular file system in favor of a lookup operation using lookup data maintained by the separate enhanced file system;

obtain meta data for monitored files and recording the meta data information for use by the separate enhanced file system in bypassing operations of the regular file system;

select by the user a directory as a monitored directory, any files in the monitored directory, or any subdirectory contained within the monitored directory; and

operate transparently in conjunction with the regular file system after an initial setup is completed;

wherein operation of the regular file system is changed only with respect to monitored files.

43. The method of claim 42, further comprising instructions to maintain in random access memory a distinct cache used only for meta data.

44. The method of claim 42, further comprising instructions to bypass directory-level security controls of the regular file system when bypassing a lookup operation.

45. The method of claim 42, further comprising instructions to

maintain by the regular file system a cache of disk data in random access memory of the computer,

control by the separate enhanced file system what data is maintained in the cache to discard data from a least recently used file; and

interact with the user through a graphical user interface using a computer program application running in user mode on the computer.