Method and apparatus for a caching file server

• US 5,452,447 A
• Filed: 12/21/1992
• Issued: 09/19/1995
• Est. Priority Date: 12/21/1992
• Status: Expired due to Term

• Current Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Khalidi, Yousef A., Nelson, Michael N.
• Primary Examiner(s): Black, Thomas G.
• Assistant Examiner(s): ELLCESSOR, LARR
• Application Number:
  US07/995,423
• Time in Patent Office: 1,002 Days
• Field of Search: 395/600, 395/650, 395/700, 395/425
• US Class Current: 1/1
• CPC Class Codes: G06F 12/08 in hierarchically structured memory systems, e.g. virtual memory systems
• CPC Class Codes: G06F 12/0806 Multiuser, multiprocessor or multiprocessing cache systems
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: Y10S 707/99956 File allocation

Abstract

A method and apparatus are described for a caching file server ("CFS") system which permits end user client programs on a local node of a distributed computer system, to issue requests to read/write data to a remote file and to query/set attributes of the remote file, and to have these requests serviced by the CFS in a way which minimizes the caching resources used as well as minimizes the related network communications. The CFS establishes CFS file programs to interface with the client programs, and sets up a common CFS cache for the file attributes, which cache is kept current via a communications link to a file program in a file server at the remote node containing the file, wherein the file program automatically updates the common CFS cache whenever any attributes change. The CFS also caches the results of bind operations in order to further minimize network calls and redundant cache usage. In addition the CFS coordinates all client program requests for read/write data requests with a virtual memory manager ("VMM") on the local node, servicing all client programs from a single pager program in the VMM which is coupled to a cache program in the file server of the remote node. VMM page-in/page-out operations go directly to the remote file server without CFS intervention. In this manner, network communications and related processing overhead as well as memory resources may be minimized by use of the VMM for caching file data and the Common CFS cache for caching file attributes.

Claims

1. A computer system having a plurality of computers, each computer having a processor, a memory, and a microkernel operating system, said computer system comprising:

a. an extensible file system wherein a new file system can be added to the system without modification to the microkernel operating system;

b. a virtual memory manager (VMM) in a computer which can cache data from said new file system; and

c. a caching file server (CFS) resident on the computer containing the VMM, said CFS configured to cache file system attributes from said new file system which attributes said VMM cannot cache, said CFS coordinating the caching operations for clients between said CFS and said VMM so that data caching is not duplicated by said CFS and said VMM.

2. The computer system of claim 1 wherein said CFS can also cache data related to bind operations which said VMM cannot cache.

3. A data processing system having one or more file systems and connected at a local computer to at least one remote computer by a communications link, and said local computer having a memory and a virtual memory manager ("VMM"), said data processing system comprising:

a. at least one file in the one or more file systems;

b. a microkernel operating system in said local computer thereby permitting a user to add a new file system without modification to the microkernel operating system;

c. said VMM in said local computer comprising mechanisms for caching data from said at least one file system required by clients in said local computer, said mechanisms comprising memory in said local computer and devices for managing said caches and said memory and supplying said clients with data from said at least one file system; and

d. a caching file server ("CFS") in said local computer outside of said microkernel operating system which provides caching services for the at least one file, said caching services comprising mechanisms for caching specific data within said CFS if said specific data is not cached by said VMM and mechanisms for using existing caches in said VMM for caching other data where said VMM is able to cache said other data thereby preventing any duplication of data caching in said CFS and said VMM, said CFS comprising a first file program in the CFS which recognizes requests from a first client program, to read first designated data which originates in the at least one file and which is passed to said first client through said cache in said VMM and to write second designated data which originates in said first client program and is ultimately written to said at least one file having passed through said cache in said VMM, said CFS coordinating the caching of said first and second designated data to the at least one file with the VMM in the local computer when said VMM can cache said first and second designated data, and wherein said CFS recognizes requests to query/set attributes of the at least one file and caches said file attributes in the CFS when said attributes are not cached by said VMM, said attributes of said at least one file comprising data which resides in said at least one file and which is accessible by said first client program.

whereby the caching services provided by the CFS and the VMM for the first client program need not be duplicated and can minimize data transfer overhead by virtue of being cached at the local computer.

4. The data processing system as recited in claim 3 wherein said first file program in the CFS recognizes requests from a second client program, to read/write data to the at least one file and recognizes requests from the second client program to query/set attributes of the at least one file, and wherein said CFS coordinates the caching of read/write data to the at least one file with the VMM in the local computer to insure that only one cache of the read/write data is maintained for both of the first and second client programs when addressing a file, and wherein said CFS caches said file attributes in the CFS, whereby the caching services provided by the CFS and the VMM for said first and second client programs need not be duplicated and can minimize data transfer overhead by virtue of being cached at the local computer.

5. The data processing system as recited in claim 4 wherein the CFS services both the first client program and the second client program by use of a common CFS cache.

6. The data processing system as recited in claim 5 wherein bind requests are sent by a client program to the CFS, and wherein the CFS uses the common CFS cache to cache results of bind operations so that each new bind request is checked to determine if a binding already exists before transmitting the bind request to a remote computer.

7. The data processing system as recited in claim 6 wherein the CFS uses the common CFS cache to cache attributes of the at least one file.

8. The data processing system as recited in claim 7 further comprising a file program in a file server located on the at least one remote computer and a communications link between said file program and the CFS for supplying attribute data of the at least one file to the CFS.

9. The data processing system as recited in claim 8 wherein the file program in the file server located on the at least one remote computer supplies new attribute data from the at least one file to the common CFS cache to maintain coherent file attribute data, thereby permitting the CFS to control "set length" and "write" operations to said at least one file.

10. The data processing system as recited in claim 8 further comprising a cache program in the file server and a communications link between said cache program and a pager program in the VMM for supplying file data to and from the at least one file to the VMM, in response to page-in/page-out from the pager program in said VMM, whereby said VMM services requests from CFS programs for data from the at least one file.

11. The data processing system as recited in claim 10 wherein said programs are object oriented programs.

12. A data processing system having one or more file systems, one or more computers, each computer having a processor, a memory, program instructions in said memory, and a virtual memory manager ("VMM"), said data processing system comprising:

a. at least one file in the one or more file systems;

b. a microkernel operating system in a local one of said one or more computers, said microkernel operating system using an extensible file system wherein new file systems can be added by a user;

c. said VMM comprising mechanisms for caching data from said at least one file required by a client in a local computer, said mechanisms comprising memory in said local computer and devices for managing caches and said memory and supplying said client with data from said at least one file; and

d. a caching file server ("CFS") in said local computer which provides caching services for the at least one file, said caching services comprising mechanisms for caching specific data within said CFS if said specific data is not cached by said VMM and mechanisms for using existing cache in said VMM for caching other data where said VMM is able to cache said other data thereby preventing any duplication of data caching in said CFS and said VMM, said CFS comprising a first file program in the CFS which recognizes requests from a first client program, to read first designated data which originates in the at least one file and which is passed to said first client through said cache in said VMM and to write second designated data which originates in said first client program and is ultimately written to said at least one file having passed through said cache in said VMM, said CFS coordinating the caching of said first and second designated data to the at least one file with the VMM when said VMM can cache said first and second designated data, and wherein said CFS recognizes requests to query/set attributes of the at least one file and caches said file attributes in the CFS when said attributes are not cached by said VMM, said attributes of said at least one file comprising data which resides in said at least one file and which is accessible by said first client program,

whereby the caching services provided by the CFS and the VMM for the first client program need not be duplicated and disk input/output operations can be reduced.

13. The data processing system as recited in claim 12 wherein the file program in the CFS recognizes requests from a second client program to read/write data to the at least one file and coordinates the caching of read/ write data to the at least one file with the VMM, and recognizes requests from the second client program to query/set attributes of the at least one file and caches said file attributes in the CFS, whereby the caching services provided by the CFS and the VMM for said first and second client programs need not be duplicated, and related disk input/ output operations can be reduced.

14. The data processing system as recited in claim 13 wherein the CFS services both the first client program and the second client program by use of a common CFS cache.

15. The data processing system as recited in claim 14 wherein the CFS uses the common CFS cache to cache results of bind operations.

16. The data processing system as recited in claim 15 wherein the CFS uses the common CFS cache to cache attributes of the at least one file.

17. The data processing system as recited in claim 16 further comprising a file program in a file server containing the at least one file and a communications link between said file program and the common CFS cache for supplying attribute data of the at least one file to the CFS.

18. The data processing system as recited in claim 17 wherein the file program in the file server supplies new attribute data from the at least one file to the common CFS cache to maintain coherent file attribute data.

19. The data processing system as recited in claim 18 further comprising a cache program in the file server and a communications link between said cache program and a pager program in the VMM, for supplying file data to and from the at least one file, in response to page-in/page-out from the VMM, whereby said VMM services requests for data from the at least one file.

20. The data processing system as recited in claim 19 wherein said programs are object oriented programs.

21. A method, performed by a computer having a processor, a memory, a computer program residing in said memory, of accessing a remote file from a local computer in a data processing system having one or more file systems and connected at said local computer to at least one remote computer by a communications link, said method comprising:

a. using a microkernel operating system in said local computer;

b. using a virtual memory manager ("VMM") in said local computer outside of said microkernel operating system, comprising mechanisms for caching data from said one or more file systems required by a client in said local computer, said mechanisms comprising memory in said local computer and devices for managing said cache and said memory and supplying said client with data from said one or more file systems;

c. establishing a caching file server ("CFS") in said local computer which provides caching services for at least one file, said caching services comprising mechanisms for caching specific data within said CFS if said specific data is not cached by said VMM, and mechanisms for using existing caches in said VMM for caching other data where said VMM is able to cache said other data thereby preventing any duplication of data caching in said CFS and said VMM,; and

d. establishing a file program in the CFS which recognizes requests from a first client program to read first designated data which originates in the at least one file and which is passed to said first client through said cache in said VMM and to write second designated data which originates in said first client program and is ultimately written to said at least one file having passed through said cache in said VMM, said CFS coordinating the caching of said first and second designated data to the at least one file with the VMM when said VMM can cache said first and second designated data, and wherein said CFS recognizes requests to query/set attributes of the at least one file and caches said file attributes in the CFS when said attributes are not cached by said VMM, said attributes of said at least one file comprising data which resides in said at least one file and which is accessible by said first client program,

whereby the caching services provided by the CFS and the VMM for the first client program need not be duplicated and related network communications traffic between the local and remote computers can be minimized.

22. The method as recited in claim 21 further comprising the additional steps of recognizing requests from a second client program to read/ write data to the at least one file and coordinating the caching of read/write data to the at least one file with the VMM in the local computer, and recognizing requests from the second client program to query/set attributes of the at least one file and caching said file attributes in the CFS, whereby the caching services provided by the CFS and the VMM for said first and second client programs need not be duplicated and related network communications traffic between the local and remote computers can be minimized.

23. The method as recited in claim 22 wherein the CFS services both the first client program and the second client program by use of a common CFS cache.

24. The method as recited in claim 23 wherein the CFS uses the common CFS cache to cache results of bind operations.

25. The method as recited in claim 24 wherein the CFS uses the common CFS cache to cache attributes of the at least one file.

26. The method as recited in claim 25 further comprising the steps of establishing a communications link between a file program in a file server and the common CFS cache for supplying attribute data of the at least one file to the common CFS cache.

27. The method as recited in claim 26 wherein the file program in the file server supplies new attribute data from the at least one file to the common CFS cache to maintain coherent file attribute data.

28. The method as recited in claim 27 further comprising a step of establishing a cache program in the file server and a communications link between the cache program and a pager program in the VMM, said VMM being located on the local computer, for supplying file data to and from the at least one file, in response to page-in/page-out operations from the pager program in said VMM, whereby said VMM services requests from CFS programs for data from the at least one file.