Pre-allocation and hierarchical mapping of data blocks distributed from a first processor to a second processor for use in a file system

• US 7,945,726 B2
• Filed: 05/08/2006
• Issued: 05/17/2011
• Est. Priority Date: 05/08/2006
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Fridella, Stephen A., Faibish, Sorin, Jiang, Xiaoye, Gupta, Uday K.
• Primary Examiner(s): Bragdon; Reginald G
• Assistant Examiner(s): Ahmed; Hamdy S
• Application Number:
  US11/382,075
• Publication Number:
  US 20070260842A1
• Time in Patent Office: 1,835 Days
• Field of Search: 711/112, 707/693
• US Class Current: 711/112
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

In a data processing system, a first processor pre-allocates data blocks for use in a file system at a later time when a second processor needs data blocks for extending the file system. The second processor selectively maps the logical addresses of the pre-allocated blocks so that when the pre-allocated blocks are used in the file system, the layout of the file system on disk is improved to avoid block scatter and enhance I/O performance. The selected mapping can be done at a program layer between a conventional file system manager and a conventional logical volume layer so that there is no need to modify the data block mapping mechanism of the file system manager or the logical volume layer. The data blocks can be pre-allocated adaptively in accordance with the allocation history of the file system.

Claims

1. A method of operating a primary data processor and a secondary data processor for access to a file system in data storage, said method comprising:

the primary data processor pre-allocating data blocks of the data storage to the file system before the pre-allocated data blocks are needed by the secondary data processor for extending the file system, and the secondary data processor using at least some of the pre-allocated data blocks for extending the file system,

wherein the primary data processor pre-allocates an initial chunk size of data blocks when the file system is created, and upon expiration of a release timeout duration from the pre-allocation of the initial chunk size, the secondary data processor releases to the primary data processor an excess of the pre-allocated data blocks over a release threshold.

2. The method as claimed in claim 1, wherein respective groups of the data blocks pre-allocated to the file system are further pre-allocated to different processes writing to a sparse file of the file system.

3. The method as claimed in claim 1, wherein the primary data processor pre-allocates the data blocks to the file system by linking the pre-allocated data blocks to a sparse file.

4. The method as claimed in claim 1, wherein the primary data processor pre-allocates a subsequent chunk size of data blocks in response to the number of pre-allocated data blocks falling below a reserve threshold.

5. The method as claimed in claim 1, wherein the primary data processor pre-allocates a respective disjoint set of data blocks of the data storage to each of a plurality of secondary data processors, and maintains a map of free data blocks and a respective map of the data blocks that have been pre-allocated to each of the secondary data processors and not yet committed to file systems in the data storage.

6. A data processing system comprising:

data storage;

a primary data processor programmed to pre-allocate data blocks of the data storage to a file system in the data storage before the pre-allocated data blocks are needed for extending the file system; and

a secondary data processor coupled to the data storage for using at least some of the pre-allocated data blocks for extending the file system,

wherein the primary data processor is programmed to pre-allocate an initial chunk size of data blocks when the file system is created, and the secondary data processor is programmed to release to the primary data processor an excess of the pre-allocated data blocks over a release threshold upon expiration of a release timeout duration from the preallocation of the initial chunk size.

7. The data processing system as claimed in claim 6, wherein the secondary data processor is programmed to pre-allocate respective groups of the data blocks to different processes writing to a sparse file of the file system.

8. The data processing system as claimed in claim 6, wherein the primary data processor is programmed to pre-allocate data blocks of the data storage to the file system by linking the pre-allocated data blocks to a sparse file.

9. The data processing system as claimed in claim 6, wherein the primary data processor is programmed to pre-allocate a subsequent chunk size of data blocks in response to the number of pre-allocated data blocks falling below a reserve threshold.

10. The data processing system as claimed in claim 6, wherein the primary data processor is programmed to pre-allocate a respective disjoint set of data blocks of the data storage to each of a plurality of secondary data processors, and to maintain a map of free data blocks and a respective map of the data blocks that have been pre-allocated to each of the secondary data processors and not yet committed to file systems in the data storage.