Method of dynamically allocating network node memory's partitions for caching distributed files

• US 6,047,356 A
• Filed: 04/18/1994
• Issued: 04/04/2000
• Est. Priority Date: 04/18/1994
• Status: Expired due to Term

• Current Assignee: Hanger Solutions, LLC (IP Investments Group LLC)
• Original Assignee: Sonic Solutions (Adeia Inc.)
• Inventors: Anderson, David P., Moorer, James A.
• Primary Examiner(s): Nguyen, Hiep T.
• Application Number:
  US08/229,041
• Time in Patent Office: 2,178 Days
• Field of Search: 711/113, 711/117, 711/118, 711/129, 711/137, 711/173, 711/170, 711/171, 711/172, 711/153, 711/168, 707/104, 707/205, 709/203, 709/214, 709/215, 709/216, 709/217, 709/218, 709/219, 709/234, 710/52, 710/56, 710/60
• US Class Current: 711/129
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: G06F 2212/311 In host system
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

A distributed file system with dedicated nodes capable of being connected to workstations at their bus. The system uses a complementary client-side and server-side file caching method that increases parallelism by issuing multiple server requests to keep the hardware devices busy simultaneously. Most of the node memory is used for file caching and input/output (I/O) device buffering using dynamic memory organization, reservation and allocation methods for competing memory-intensive activities.

Claims

1. In a distributed file system including high speed random access general purpose memory within a network node coupled to a host computer and a plurality of mass storage devices interconnected via a network for storing data files in disparate locations, a method for caching data files from said mass storage devices using a limited amount of said general purpose memory, said method comprising the steps of:

providing at least one cache area in said general purpose memory for each accessed file;

evaluating a data flow rate over network data paths and direct data paths associated with said each accessed file through said at least one cache area ("_-- file data flow rate_-- "); and

allocating, by means of a processor within said network node, a portion of said general purpose memory to said at least one cache area in an amount proportional to said associated file data flow rate.

2. The method according to claim 1 for caching data in response to instructions for performing read operations by a client device from a network file server, said network file server defining file structures on selected ones of said at least one mass storage device, said method comprising the steps of:

providing, on said network file server, a first cache area of said at least one cache area for caching first data from said at least one mass storage device;

providing, on said network file server, a second cache area of said at least one cache area for caching second data which is to be read by an application program from said at least one mass storage device, said second data being stored on said network file server; and

reading ahead into the second server cache area enough data to keep operations involving the first server cache area at least one mass storage device access interval ahead of operations involving data currently being read by said application.

3. The method according to claim 1 further including the step for organizing said general purpose memory, said step of organizing comprising the substeps of:

establishing a plurality of units of said general purpose memory of equal size;

assigning each of said units to one of a plurality of sets of the units, said sets being defined for mass storage device cache areas and for I/O device buffers, the order of the units of each set being independent of the order and distribution of the units in said general purpose memory;

maintaining a data structure of remaining unassigned units, the data structure being independent of the order of the remaining unused units in said general purpose memory;

using said sets of units as said cache areas for said mass storage devices and for said I/O device buffers; and

changing assignment of at least one of said units from a first one of said sets to a second one of said sets.

4. The method according to claim 1 further including the step for organizing said memory, said step for organizing said memory comprising the substeps of:

specifying a first minimum amount and a first maximum amount of said general purpose memory to be reserved for a first one of a plurality of activities;

if a sum of minimum amounts of said general purpose memory to be reserved by all said activities is less than a total amount of said general purpose memory available for said activities, reserving for a first one of said activities an actual amount of said general purpose memory between said first minimum amount and said first maximum amount; and

if said general purpose memory has been reserved for said first one of said activities, and thereafter if a second one of said activities requests a change in a second minimum amount and a second maximum amount of said general purpose memory to be reserved for said second one of said activities, adjusting said actual amount of said general purpose memory reserved for said first one of said activities and for said second one of said activities between said first and second minimum amounts and said first and second maximum amounts such that a sum of all actual reservations is less than or equal to total available amount of said general purpose memory.

5. The method according to claim 1 further including the step of:

caching, responsive to a first read request, a first block of file data from a local mass storage device which is one of said at least one mass storage devices into a first cache area associated with said accessed file, said caching comprising the sub-steps of;

determining whether said first read request is part of a sequential pattern of read requests;

if said first read request is part of a sequential pattern of read requests and said first block of file data is not in said first cache area of said general purpose cache memory, reading a range of blocks beginning with said first block of file data into said first cache area from said local mass storage device;

if said first read request is part of a sequential pattern of read requests and said first block of file data is in said first cache area, verifying that said range of blocks beginning with said first block of file data is in said first cache area; and

if said first read request is part of a sequential pattern of read requests, said first block of file data is in said first cache area and said range of blocks beginning with said first block of file data is not in said first cache area, reading those blocks of said range which were not in said first cache area into said first cache area from said local mass storage device.