Apparatus and method for hardware-based file system

• US 7,457,822 B1
• Filed: 11/01/2002
• Issued: 11/25/2008
• Est. Priority Date: 11/01/2002
• Status: Active Grant

• Current Assignee: Hitachi Vantara, LLC (Hitachi, Ltd.)
• Original Assignee: Bluearc UK Ltd. (Hitachi, Ltd.)
• Inventors: Barrall, Geoffrey S., Willis, Trevor E., Benham, Simon L., Aston, Christopher J.
• Primary Examiner(s): Wu; Yicun
• Application Number:
  US10/286,015
• Time in Patent Office: 2,216 Days
• Field of Search: 707/1, 707/2, 707/3, 707/4, 707/5, 707/6, 707/7, 707/8, 707/9, 707/10, 707/100, 707/101, 707/102, 707/204, 707/104, 707/203, 707/200, 707/205, 710/128, 710/39, 710/5, 710/104, 710/48, 710/107, 710/6, 710/10, 710/305, 710/8, 710/129, 714/1, 714/4, 714/6, 709/220, 709/227, 709/300, 709/200, 709/230, 709/238, 709/243, 709/221, 709/2, 709/214, 709/226, 709/316, 709/250, 709/224, 709/229, 395/200, 395/325, 395/182, 395/842, 395/683, 395/275, 395/700, 395/725, 395/800, 395/650, 395/148, 395/400, 395/145, 395/425, 395/101, 395/575, 395/183, 395/185, 395/550, 711/111, 711/104, 711/118, 711/114, 711/170, 711/162, 711/141, 711/1, 379/96, 379/28, 364/200, 364/900, 370/60, 370/124, 370/85, 370/58.2, 370/464, 371/9, 179/15, 179/18, 340/825, 340/172, 719/313, 307/602, 360/77, 712/12, 713/375, 713/201, 703/25, 455/552, 273/143.R
• US Class Current: 1/1
• CPC Class Codes: G06F 12/0246 in block erasable memory, e.g. flash memory
• CPC Class Codes: G06F 16/128 Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion use of snapshots for error detection or correction G06F11/14, G06F11/16
• CPC Class Codes: G06F 16/183 Provision of network file services by network file servers, e.g. by using NFS, CIFS network file access protocols H04L67/1097
• CPC Class Codes: G06F 16/1873 Versioning file systems, temporal file systems, e.g. file system supporting different historic versions of files
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image

Abstract

A hardware-based file system includes multiple linked sub-modules that perform functions ancillary to client data handling. Each sub-module is associated with a metadata cache. A doubly-rooted structure is used to store each file system object at successive checkpoints. Metadata is stored within an object and/or as a separate object. Provisions are made for allocating sparse objects. A delayed write feature is used for writing certain objects into non-volatile storage. Checkpoints can be retained as read-only versions of the file system. Modifications to the file system are accomplished without affecting the contents of retained checkpoints. The file system can be reverted to a retained checkpoint. Multiple file servers can be interconnected as a cluster, and each file server stores requests from another file server. Interconnections between file servers can be dynamically modified. A distributed locking mechanism is used to control access to file system objects stored by the file servers.

Claims

1. A file server system for accessing and utilizing a data storage system, the file server system including a network subsystem for receiving and responding to file service requests over a network and further including a file subsystem, in communication with the network subsystem, for satisfying the file service requests, the file subsystem comprising:

a. a data bus arrangement, in communication with the network subsystem and the data storage system, for providing data to be stored in the data storage system and for retrieving data from the data storage system;

b. a plurality of linked sub-modules, wherein the linked sub-modules as a group are in communication with a control input for receiving file service requests and a control output for responding to file service requests and process such service requests and generate responses thereto over the control output, each sub-module configured to perform a distinct set of operations pertinent to processing of such file service requests;

wherein the control input and the control output are distinct from the data bus arrangement; and

c. a plurality of metadata memory caches, each metadata memory cache associated with a corresponding sub-module for storing metadata pertinent to operations of such sub-module.

3. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for causing storage and retrieval of file system objects in the storage system and a file sub-module for managing data structure associated with file attributes.

4. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for causing storage and retrieval of file system objects in the storage system, a file sub-module for managing data structure associated with file attributes, and a directory sub-module for handling directory management for the file sub-module.

5. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules include an object store sub-module for causing storage and retrieval of file system objects in the storage system and a free space allocation sub-module for retrieving and updating data pertinent to allocation of space in the data storage system.

6. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for causing storage and retrieval of file system objects in the storage system, a file sub-module for managing data structure associated with file attributes, a directory sub-module for handling directory management for the file sub-module, and a tree sub-module for handling directory lookups for the directory sub-module.

7. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for causing storage and retrieval of file system objects in the storage system, a file sub-module for managing data structure associated with file attributes, a directory sub-module for handling directory management for the file sub-module, and a tree sub-module for handling directory lookups for the directory sub-module, wherein the directory sub-module associates each file with a randomized value, and wherein the tree sub-module manages a logical tree structure based upon the randomized values from the directory sub-module.

8. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for causing storage and retrieval of file system objects in the storage system, a file sub-module for managing data structure associated with file attributes, a directory sub-module for handling directory management for the file sub-module, and a tree sub-module for handling directory lookups for the directory sub-module, wherein the directory sub-module associates each file with a randomized value comprising a cyclic redundancy checksum (CRC) of a file name associated with the file, and wherein the tree sub-module manages a logical tree structure based upon the randomized values from the directory sub-module.

9. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for causing storage and retrieval of file system objects in the storage system, a file sub-module for managing data structure associated with file attributes, a directory sub-module for handling directory management for the file sub-module, and a tree sub-module for handling directory lookups for the directory sub-module, wherein the directory sub-module associates each file with a randomized value, and wherein the tree sub-module associates each randomized value with an index into the logical tree structure and uses the randomized values to access the logical tree structure.

10. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for causing storage and retrieval of file system objects in the storage system, a file sub-module for managing data structure associated with file attributes, a directory sub-module for handling directory management for the file sub-module, and a tree sub-module for handling directory lookups for the directory sub-module, wherein the directory sub-module associates each file with a randomized value, and wherein the directory sub-module maintains a list of files that are associated with the same randomized value and uses the list of files to access files associated with the same randomized value.

11. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include a non-volatile storage processing sub-module with associated non-volatile storage for storing file system request data for subsequent storage in the storage system.

12. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include a non-volatile storage processing sub-module with associated non-volatile storage for storing file system request data for subsequent storage in the storage system, and wherein the non-volatile storage processing sub-module is operably coupled to store the file system request data in the non-volatile storage at the request of a processor for recovery from a failure.

13. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include a non-volatile storage processing sub-module with associated non-volatile storage for storing file system request data for subsequent storage in the storage system, and wherein the non-volatile storage processing sub-module is operably coupled to store the file system request data in the non-volatile storage at the request of a processor for recovery from a failure and to send an acknowledgment to the processor confirming storage of the file system request data in the non-volatile storage.

14. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include a non-volatile storage processing sub-module with associated non-volatile storage for storing file system request data for subsequent storage in the storage system, and wherein the non-volatile storage processing sub-module is operably coupled to receive file system request data from another file server via an interface and to store the file system request data from the other file server in the non-volatile storage.

15. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include a non-volatile storage processing sub-module with associated non-volatile storage for storing file system request data for subsequent storage in the storage system, and wherein the non-volatile storage processing sub-module is operably coupled to send the file system request data to another file server via an interface for non-volatile storage of the file system request data by the other file server.

16. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for causing storage and retrieval of file system objects in the storage system, and wherein the object store sub-module maintains a file structure for each file system object to be stored in the storage system.

17. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for causing storage and retrieval of file system objects in the storage system, and wherein the object store sub-module maintains a file structure in its associated metadata cache for each file system object to be stored in the storage system.

18. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for causing storage and retrieval of file system objects in the storage system, and wherein the object store sub-module maintains a file structure for each file system object to be stored in the storage system, the file system objects comprising files, directories, and file attributes.

19. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for causing storage and retrieval of file system objects in the storage system, and wherein the object store sub-module maintains a file structure for each file system object to be stored in the storage system and effectuates storage of the file structures into the storage system at various checkpoints.

20. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for causing storage and retrieval of file system objects in the storage system, and wherein the object store sub-module maintains a file structure for each file system object to be stored in the storage system and effectuates storage of the file structures into the storage system when initiated by an external processor.

21. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for causing storage and retrieval of file system objects in the storage system, and wherein the object store sub-module maintains a file structure for each file system object to be stored in the storage system and effectuates storage of the file structures into the storage system when a predetermined amount of time has elapsed since a last storage of the file structures into the storage system.

22. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include a non-volatile storage processing sub-module with associated non-volatile storage for storing file system request data for subsequent storage in the storage system and an object store sub-module for causing storage and retrieval of file system objects in the storage system, and wherein the object store sub-module maintains a file structure for each file system object to be stored in the storage system and effectuates storage of the file structures into the storage system when a portion of the non-volatile storage used for storage of the file system request data is becoming full.

23. A file server according to any of claims 1 or 2, further comprising a sector cache for maintaining sector-level information about the storage system, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for causing storage and retrieval of file system objects in the storage system, and wherein the object store sub-module maintains a file structure for each file system object to be stored in the storage system and effectuates storage of the file structures into the storage system when the sector cache is becoming full.

24. A file server according to any of claims 1 or 2, further comprising a controller external to the plurality of linked sub-modules, wherein the plurality of linked sub-modules are hierarchically arranged and include a non-volatile storage processing sub-module with associated non-volatile storage for storing file system request data for subsequent storage in the storage system, a file sub-module for managing data structure associated with file attributes, a directory sub-module for handling directory management for the file sub-module, a tree sub-module for handling directory lookups for the directory sub-module, an object store sub-module for maintaining a file structure for each file system object to be stored in the storage system and for effectuating storage of the file structures into the storage system at various checkpoints, and a free space allocation sub-module for retrieving and updating data pertinent to allocation of space in the data storage system, and wherein:

the controller sends a number of file system requests to the non-volatile storage processing sub-module and subsequently sends a checkpoint inquiry command to the non-volatile storage processing sub-module to initiate storage of the file structures into the storage system for a checkpoint, the checkpoint inquiry command including a checkpoint number for the checkpoint;

the non-volatile storage processing sub-module stores the number of file system requests in the non-volatile storage, optionally sends the number of file system requests to another file server via an interface for non-volatile storage of the number of file system requests by the other file server, sends the number of file system requests to the file sub-module, and subsequently sends a checkpoint command to the file sub-module;

the file sub-module processes the number of file system requests, and, upon receiving the checkpoint command from the non-volatile storage processing sub-module, waits for certain operations to complete through the remaining sub-modules and then sends a checkpoint command to the directory sub-module;

the directory sub-module receives the checkpoint command from the file-sub-module and sends a checkpoint command to the tree sub-module;

the tree sub-module receives the checkpoint command from the directory sub-module and sends a checkpoint command to the object store sub-module;

the object store sub-module receives the checkpoint command from the tree sub-module and sends a checkpoint inquiry to the free space allocation sub-module;

the free space allocation sub-module receives the checkpoint inquiry from the object store sub-module, completes any operations necessary for the checkpoint including operations initiated subsequent to receiving the checkpoint inquiry, and then sends a response to the object store sub-module; and

the object store sub-module causes the file system objects to be written to the storage system, including an updated objects lists indicating any and all objects that have been modified since a last checkpoint.

25. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for maintaining a file structure comprising a plurality of nodes and at least one data block for each file system object to be stored in the storage system.

26. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for maintaining a file structure comprising a plurality of nodes and at least one data block for each file system object to be stored in the storage system, each node comprising at least one of:

a pointer to another node; and

a data block descriptor.

27. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for maintaining a file structure comprising a plurality of nodes and at least one data block for each file system object to be stored in the storage system, each node comprising a checkpoint number indicating a checkpoint during which the node was created.

28. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for maintaining a file structure comprising a plurality of nodes and at least one data block for each file system object to be stored in the storage system, the plurality of nodes comprising at least two root nodes for storing information for a first and a second checkpoint.

29. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for maintaining a file structure comprising a plurality of nodes and at least one data block for each file system object to be stored in the storage system, the plurality of nodes comprising at least two root nodes for storing information for a first and a second checkpoint, the two root nodes stored in adjacent sectors in the storage system.

30. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for maintaining a file structure comprising a plurality of nodes and at least one data block for each file system object to be stored in the storage system, the plurality of nodes comprising at least two root nodes for storing information for a first and a second checkpoint, each root node comprising at least one of:

an object type for indicating whether the file system object is a file or a directory;

an object length for indicating the number of data blocks associated with the file system object;

a reuse count indicating the number of time the root node has been used;

a pointer to a previous instantiation of the root node;

a pointer to a subsequent instantiation of the root node;

at least one a data block descriptor including a pointer to a data block, a checkpoint number indicating a relative time the data block was created, and an indicator to indicate whether the data block is zero or non-zero; and

file attributes (enode).

31. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for maintaining a file structure comprising a plurality of nodes and at least one data block for each file system object to be stored in the storage system, the plurality of nodes comprising at least two root nodes for storing information for a first and a second checkpoint and at least one direct node including at least one a data block descriptor.

32. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for maintaining a file structure comprising a plurality of nodes and at least one data block for each file system object to be stored in the storage system, the plurality of nodes comprising at least two root nodes for storing information for a first and a second checkpoint and at least one direct node including at least one a data block descriptor, at least one of the root nodes comprising a pointer to the direct node.

33. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for maintaining a file structure comprising a plurality of nodes and at least one data block for each file system object to be stored in the storage system, the plurality of nodes comprising at least two root nodes for storing information for a first and a second checkpoint and at least two indirect nodes, each indirect node including at least one pointer to one of:

another indirect node; and

a direct node.

34. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for maintaining a file structure comprising a plurality of nodes and at least one data block for each file system object to be stored in the storage system, the plurality of nodes comprising at least two root nodes for storing information for a first and a second checkpoint and at least two indirect nodes, at least one of the root nodes comprising a pointer to an indirect node, the indirect node comprising pointers to at least two direct nodes.

35. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for maintaining a file structure comprising a plurality of nodes and at least one data block for each file system object to be stored in the storage system, the file structure comprising an indicator for each data block associated with the file system object to indicate whether the data block is zero or non-zero.

36. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for maintaining a file structure comprising a plurality of nodes and at least one data block for each file system object to be stored in the storage system, the file structure comprising an indicator for node and data block to indicate whether each node and data block has been created.

37. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for maintaining a file structure comprising a plurality of nodes and at least one data block for each file system object to be stored in the storage system, and wherein the object store sub-module is operably coupled to create nodes and data blocks as necessary to accommodate file system write requests and to set an indicator for each node and data block to indicate that the node or data block has been created.

38. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for maintaining a file structure comprising a plurality of nodes and at least one data block for each file system object to be stored in the storage system and a free space allocation sub-module for retrieving and updating data pertinent to allocation of space in the storage system, and wherein the object store sub-module creates a data block by allocating space for the data block from the free space allocation sub-module.

39. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for maintaining a file structure for each file system object to be stored in the storage system, and wherein the object store sub-module is operably coupled to maintain a transaction log.

40. A file server according to any of claims 1 or 2, wherein the plurality of linked sub-modules are hierarchically arranged and include an object store sub-module for maintaining a file structure for each file system object to be stored in the storage system and a transaction log, and wherein the object store sub-module is operably coupled to store the transaction log and the file structure in the storage system from time to time.

41. A file server according to any of claims 1 or 2, wherein at least one of the plurality of sub-modules comprises dedicated hardware.

42. A file server according to any of claims 1 or 2, wherein at least one of the plurality of sub-modules comprises a dedicated processor.

43. A file server according to any of claims 1 or 2, wherein the storage system comprises at least one of:

magnetic storage;

magneto-optical storage; and

optical storage.

2. A file server system for accessing and utilizing a data storage system, the file server system including a network subsystem for receiving and responding to file service requests over a network and further including a file subsystem, in communication with the network subsystem, for satisfying the file service requests, the file subsystem comprising:

a. a data bus arrangement, in communication with the network subsystem and the data storage system, for providing data to be stored in the data storage system and for retrieving data from the data storage system;

b. a plurality of linked sub-modules, wherein the linked sub-modules as a group are in communication with a control input for receiving file service requests and a control output for responding to file service requests and process such service requests and generate responses thereto over the control output, each sub-module configured to perform a distinct set of operations pertinent to processing of such file service requests; and

c. a plurality of metadata memory caches, each metadata memory cache associated with a corresponding sub-module for storing metadata pertinent to operations of such sub-module without storage of file content data.