PRE-ALLOCATION AND HIERARCHICAL MAPPING OF DATA BLOCKS DISTRIBUTED FROM A FIRST PROCESSOR TO A SECOND PROCESSOR FOR USE IN A FILE SYSTEM

• US 20070260842A1
• Filed: 05/08/2006
• Published: 11/08/2007
• Est. Priority Date: 05/08/2006
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Faibish, Sorin, Jiang, Xiaoye, Fridella, Stephen, Gupta, Uday
• Granted Patent: US 7,945,726 B2
• Time in Patent Office: Days
• US Class Current: 711/170
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

In a data processing system, a first processor pre-allocates data blocks for use in a file system at a later time when a second processor needs data blocks for extending the file system. The second processor selectively maps the logical addresses of the pre-allocated blocks so that when the pre-allocated blocks are used in the file system, the layout of the file system on disk is improved to avoid block scatter and enhance I/O performance. The selected mapping can be done at a program layer between a conventional file system manager and a conventional logical volume layer so that there is no need to modify the data block mapping mechanism of the file system manager or the logical volume layer. The data blocks can be pre-allocated adaptively in accordance with the allocation history of the file system.

Claims

1. A method of operating a primary data processor and a secondary data processor for access to a file system in data storage, said method comprising:

the primary data processor pre-allocating data blocks of the data storage to the file system before the pre-allocated data blocks are needed for extending the file system, and the secondary data processor using at least some of the pre-allocated data blocks for extending the file system,

wherein said method further includes selecting an address mapping for said at least some of the pre-allocated blocks in order to reduce block scatter on disk and improve I/O performance.

2. The method as claimed in claim 1, wherein the secondary data processor is programmed with programming layers including a file system manager layer mapping file block addresses to logical block addresses, a logical address translation layer for translating the logical block addresses from the file system manager layer in accordance with the selected address mapping for said at least some of the pre-allocated data blocks, and a logical volume layer mapping the translated logical block addresses from the logical address translation layer to storage addresses.

3. The method as claimed in claim 1, wherein respective groups of said at least some of the data blocks are further pre-allocated to different processes writing to a sparse file of the file system.

4. The method as claimed in claim 1, wherein the primary data processor pre-allocates the data blocks to the file system by linking the pre-allocated data blocks to a sparse file.

5. The method as claimed in claim 1, wherein the primary data processor pre-allocates the data blocks to the file system adaptively in accordance with an allocation history of the file system.

6. The method as claimed in claim 1, wherein the primary data processor pre-allocates an initial chunk size of data blocks when the file system is created, and upon expiration of a release timeout duration from the pre-allocation of the initial chunk size, the secondary data processor releases to the primary data processor an excess of the pre-allocated data blocks over a release threshold.

7. The method as claimed in claim 5, wherein the primary data processor pre-allocates a subsequent chunk size of data blocks to the file system in response to the number of pre-allocated data blocks falling below a reserve threshold.

8. The method as claimed in claim 1, wherein the primary data processor pre-allocates a respective disjoint set of data blocks to each of a plurality of secondary data processors, and maintains a map of free data blocks and a respective map of the data blocks that have been pre-allocated to each of the secondary data processors and not yet committed to a file system in storage.

9. A method of operating a primary data processor and a secondary data processor for access to a file system in data storage, said method comprising:

the primary data processor pre-allocating data blocks of the data storage to the file system before the pre-allocated data blocks are needed by the secondary data processor for extending the file system, and the secondary data processor using at least some of the pre-allocated data blocks for extending the file system,

wherein the primary data processor pre-allocates an initial chunk size of data blocks when the file system is created, and upon expiration of a release timeout duration from the pre-allocation of the initial chunk size, the secondary data processor releases to the primary data processor an excess of the pre-allocated data blocks over a release threshold.

10. The method as claimed in claim 9, wherein respective groups of the data blocks pre-allocated to the file system are further pre-allocated to different processes writing to a sparse file of the file system.

11. The method as claimed in claim 9, wherein the primary data processor pre-allocates the data blocks to the file system by linking the pre-allocated data blocks to a sparse file.

12. The method as claimed in claim 9, wherein the primary data processor pre-allocates a subsequent chunk size of data blocks in response to the number of pre-allocated data blocks falling below a reserve threshold.

13. The method as claimed in claim 9, wherein the primary data processor pre-allocates a respective disjoint set of data blocks of the data storage to each of a plurality of secondary data processors, and maintains a map of free data blocks and a respective map of the data blocks that have been pre-allocated to each of the secondary data processors and not yet committed to file systems in the data storage.

14. A data processing system comprising:

data storage;

a primary data processor programmed for pre-allocating data blocks of the data storage to a file system in the data storage before the pre-allocated data blocks are needed for extending the file system; and

a secondary data processor coupled to the data storage for access to the file system in the data storage and coupled to the primary data processor for obtaining a map of the data blocks pre-allocated to the file system;

wherein the data processing system is further programmed for selecting an address mapping of at least some of the pre-allocated data blocks in order to reduce block scatter on disk and improve I/O performance.

15. The data processing system as claimed in claim 14, wherein the secondary data processor is programmed with a stack of programming layers including a file system manager layer mapping file block addresses to logical block addresses, a logical address translation layer for translating the logical block addresses from the file system manager layer in accordance with the selected address mapping for said at least some of the pre-allocated data blocks, and a logical volume layer mapping the translated logical block addresses from the logical address translation layer to storage addresses.

16. The data processing system as claimed in claim 14, wherein the secondary data processor is programmed for pre-allocating respective groups of the data blocks to different processes writing to a sparse file of the file system.

17. The data processing system as claimed in claim 14, wherein the primary data processor is programmed for pre-allocating data blocks of the data storage to the file system by linking the pre-allocated data blocks to a sparse file.

18. The data processing system as claimed in claim 14, which is further programmed for pre-allocating an initial chunk size of data blocks when the file system is created, and upon expiration of a release timeout duration from the pre-allocation of the initial chunk size, releasing an excess of the pre-allocated data blocks over a release threshold.

19. The data processing system as claimed in claim 18, which is further programmed for pre-allocating a subsequent chunk size of data blocks in response to the number of pre-allocated data blocks falling below a reserve threshold.

20. The data processing system as claimed in claim 14, which includes a plurality of secondary processors coupled to the data storage for access to respective file systems in the data storage, and wherein the primary data processor is programmed for pre-allocating a respective disjoint set of data blocks of the data storage to each of the secondary data processors, and for maintaining a map of free data blocks and a respective map of the data blocks that have been pre-allocated to each of the secondary data processors and not yet committed to file systems in the data storage.