Preallocation of file system cache blocks in a data storage system

• US 6,571,259 B1
• Filed: 09/26/2000
• Issued: 05/27/2003
• Est. Priority Date: 09/26/2000
• Status: Expired due to Term

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Zheng, Jiannan, Jiang, Xiaoye, Vahalia, Uresh
• Primary Examiner(s): Metjahic, Safet
• Assistant Examiner(s): Nguyen, Cindy
• Application Number:
  US09/669,347
• Time in Patent Office: 973 Days
• Field of Search: 707/202, 707/205, 707/206, 710/74
• US Class Current: 1/1
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: Y10S 707/99953 Recoverability
• CPC Class Codes: Y10S 707/99956 File allocation

Abstract

A file server provides transaction processing capabilities previously supplied by the operating system of a host computer. On-disk file system metadata is changed only at commit time, and a transaction log protects the transition. The disk state can only be a consistent state, resulting from a commit operation. All disk-block reservation and pre-allocation mapping are in the memory, and after a crash, they are automatically discarded. The file server therefore relieves the client of processing burden and also reduces network traffic. In addition, the file server can more efficiently perform the transaction processing capabilities and reduce the frequency of access to storage by judicious allocation of file system blocks and transfer of file system blocks between file system objects, cache memory, and the transaction log. The differentiation between preallocation states and allocation states of in-memory file system blocks also permits application programs to more efficiently transfer data between files.

Claims

1. A method of operating a file server having a file system cache memory and storage, a file system being stored in the storage, said method comprising:

(a) the file server receiving at least one write request from at least one client, and in response, writing new file system attributes and new file system data to the file system cache memory, the new file system attributes including new links between file system objects and file system blocks,

(b) the-file server receiving a commit request from the client, the new file system attributes and the new file system data not being written into the file system in storage until receipt of the commit request, and in response to the commit request, writing the new file system attributes and the new file system data to the file system in storage;

wherein the file server maintains in memory a directory and file mapping data structure for the file system, the directory and file mapping data structure permitting file system block allocations and linkages between file system objects and the file system blocks to change during read/write access to the file system by the client prior to receiving the commit request, the file system block allocations including allocated blocks having block allocations that are the same as block allocations in the file system as stored in the storage, and preallocated blocks having block allocations that are different from block allocations in the file system as stored in the storage, the preallocated blocks storing new file system data for changes to be made to the on-disk file system and not yet committed to the on-disk file system, and the directory and file mapping data structure identifying the allocated blocks and the preallocated blocks and distinguishing the preallocated blocks from the allocated blocks.

2. The method as claimed in claim 1, wherein the file server changes the preallocated blocks to allocated blocks in response to the commit request.

3. The method as claimed in claim 2, wherein the file server maintains a bit map of preallocated blocks and a bit map of allocated blocks, and in response to the commit request, writes the bit map of preallocated blocks to the bit map of allocated blocks.

4. The method as claimed in claim 1, wherein the file server changes the preallocated blocks to free blocks in response to an abort request from the client.

5. The method as claimed in claim 4, wherein the file server has write pending entries in the cache memory that are linked to write pending entries in the directory and file mapping data structure in memory, and the file server invalidates the write pending entries in the cache memory and in the directory and file mapping data structure in response to the abort request.

6. The method as claimed in claim 1, wherein the file server changes allocated blocks to free blocks during read/write access to the file system by the client prior to receiving the commit request.

7. The method as claimed in claim 1, wherein the file server changes allocated blocks to preallocated blocks during read/write access to the file system by the client prior to receiving the commit request.

8. The method as claimed in claim 7, wherein in response to an abort request, the file server finds the preallocated blocks that were changed from allocated blocks and restores them to allocated blocks.

9. The method as claimed in claim 1, wherein the file server unlinks at least one block from a first file in the file system and links said at least one block to a second file in the file system in response to read/write access to the file system by the client prior to receiving the commit request.

10. The method as claimed in claim 9, wherein said at least one block is an allocated block when it is in the first file and said at least one block is a preallocated block when it is in the second file.

11. The method as claimed in claim 1, wherein the file server inserts a block into a hole in a file that is later accessed during read/write access to the file system by the client prior to receiving the commit request.

12. The method as claimed in claim 1, wherein the file server maintains a transaction log in the storage, and after a system crash, performs a recovery operation by accessing the transaction log to return the file system in the storage to a consistent state.

13. The method as claimed in claim 12, wherein the transaction log includes block before images that are written to the file system in storage to return the file system in storage to a consistent state, and the transaction log also includes new metadata including block allocations that the file server uses to delete revisions from the file system in the storage during the recovery operation.

14. A file server comprising, in combination:

a file system cache memory;

storage;

means responsive to a write request from a client for writing new file system attributes and new file system data to the file system cache memory, the new file system attributes including linkages between file system objects and file system blocks;

means responsive to a commit request from the client for writing the new file system data and new file system attributes to a file system in the storage;

means for maintaining in memory a directory and file mapping data structure for the file system, the directory and file mapping data structure permitting file system block allocations and linkages between file system objects and file system blocks to change during read/write access to the file system by the client prior to receiving the commit request, the file system block allocations including allocated blocks having block allocations that are the same as block allocations in the file system as stored in the storage, and preallocated blocks having block allocations that are different from block allocations in the file system as stored in the storage, the preallocated blocks storing new file system data for changes to be made to the on-disk file system and not yet committed to the on-disk file system, and the directory and file mapping data structure identifying the allocated blocks and the preallocated blocks and distinguishing the preallocated blocks from the allocated blocks.

15. The file server as claimed in claim 14, including means responsive to a commit request for changing the preallocated blocks to allocated blocks, and means responsive to an abort request for changing the preallocated blocks to free blocks.

16. The file server as claimed in claim 15, wherein the file server includes means for maintaining a bit map of preallocated blocks and a bit map of allocated blocks, including means responsive to a commit request for writing the bit map of preallocated blocks to the bit map of allocated blocks.

17. The file server as claimed in claim 14, wherein the file server has write pending entries in the cache memory that are linked to write pending entries in the directory and file mapping data structure in memory, and the file server includes means responsive to an abort request for invalidating the write pending entries in the cache memory and in the directory and file mapping data structure.

18. The file server as claimed in claim 14, further including means for changing allocated blocks to free blocks during read/write access to the file system by the client prior to receiving the commit request.

19. The file server as claimed in claim 14, further including means for changing allocated blocks to preallocated blocks during read/write access to the file system by the client prior to receiving the commit request.

20. The file server as claimed in claim 19, further including means for responding to an abort request by finding the preallocated blocks that were changed from allocated blocks and restoring them to allocated blocks.

21. The file server as claimed in claim 14, further including means responsive to read/write access to the file system by the client for unlinking at least one allocated block from a first file in the file system and linking said at least one allocated block to a second file in the file system prior to receiving the commit request.

22. The file server as claimed in claim 14, including means for maintaining a transaction log in the storage, and means for recovering from a system crash by accessing the transaction log to return the file system in storage to a consistent state.

23. The file server as claimed in claim 22, wherein the transaction log includes block before images that are written to the file system in storage to return the file system storage to a consistent state, and the transaction log also includes new metadata including block allocations, and the means for recovering from a system crash includes means for accessing the log to obtain the new metadata including block allocations and using the new metadata for deleting revisions from the file system in storage during the recovery operation.

24. A file server comprising, in combination:

a file system layer for mapping file names to data storage locations in response to a write request from a client;

a file system cache connected to the file system layer for storing new file system attributes and new file system data in response to the write request from the client; and

nonvolatile storage connected to the file system layer for storing the new file system attributes and the new file system data in response to a commit request from the client;

wherein the file system layer is programmed for responding to the write request from the client by writing the new file system attributes and the new file system data to the file system cache and not writing the new file system attributes and the new file system data to the file system in storage until receipt of the commit request from the client; and

wherein the file system layer is programmed for responding to the commit request from the client by writing the new file system data and the new file system attributes from the file system cache to the nonvolatile storage; and

wherein the file system layer is programmed to maintain in memory a directory and file mapping data structure for the file system, the directory and file mapping data structure permitting file system block allocations and linkages between file system objects and the file system blocks to change during read/write access to the file system by the client prior to receiving the commit request, the file system block allocations including allocated blocks having block allocations that are the same as block allocations in the file system as stored in the storage, and preallocated blocks having block allocations that are different from block allocations in the file system as stored in the storage, the preallocated blocks storing new file system data for changes to be made to the on-disk file system and not yet committed to the on-disk file system, and the directory and file mapping data structure identifying the allocated blocks and the preallocated blocks and distinguishing the preallocated blocks from the allocated blocks.

25. The file server as claimed in claim 24, wherein the file system layer is programmed to respond to a commit request for changing the preallocated blocks to allocated blocks, and for responding to an abort request for changing the preallocated blocks to free blocks.

26. The file server as claimed in claim 25, wherein the file system layer is programmed for maintaining a bit map of preallocated blocks and a bit map of allocated blocks, and for responding to a commit request by writing the bit map of preallocated blocks to the bit map of allocated blocks.

27. The file server as claimed in claim 24, wherein the file server has write pending entries in the cache memory that are linked to write pending entries in the directory and file mapping data structure in memory, and the file system layer is programmed to respond to an abort request for invalidating the write pending entries in the cache memory and in the directory and file mapping data structure.

28. The file server as claimed in claim 24, wherein the file system layer is programmed for changing allocated blocks to free blocks during read/write access to the file system by the client prior to receiving the commit request.

29. The file server as claimed in claim 24, wherein the file system layer is programmed for changing allocated blocks to preallocated blocks during read/write access to the file system by the client prior to receiving the commit request.

30. The file server as claimed in claim 29, wherein the file system layer is programmed for responding to an abort request by finding the preallocated blocks that were changed from allocated blocks and restoring them to allocated blocks.

31. The file server as claimed in claim 24, wherein the file system layer is programmed for responding to read/write access to the file system by the client for unlinking at least one allocated block from a first file in the file system and linking said at least one allocated block to a second file in the file system prior to receiving the commit request.

32. The file server as claimed in claim 24, wherein the file system layer is programmed for maintaining a transaction log in the storage, and the file server is programmed for recovering from a system crash by accessing the transaction log to return the file system in storage to a consistent state.

33. The file server as claimed in claim 32, wherein the transaction log includes block before images that are written to the file system in storage to return the file system storage to a consistent state, and the transaction log also includes new metadata including block allocations, and the file server is programmed for accessing the log to obtain the new metadata including block allocations and using the new metadata for deleting revisions from the file system in storage during the recovery operation.