Reorganization of striped data during file system expansion in a data storage system

• US 6,499,039 B1
• Filed: 09/23/1999
• Issued: 12/24/2002
• Est. Priority Date: 09/23/1999
• Status: Expired due to Term

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Venkatesh, Dinesh, Gupta, Uday, Forecast, John, Duso, Wayne W.
• Primary Examiner(s): Vu, Kim
• Assistant Examiner(s): Truong, Cam Y T
• Application Number:
  US09/404,016
• Time in Patent Office: 1,188 Days
• Field of Search: 707/200-205, 711/114
• US Class Current: 1/1
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/0632 by initialisation or re-initialisation of storage systems
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users
• CPC Class Codes: Y10S 707/99953 Recoverability
• CPC Class Codes: Y10S 707/99955 Archiving or backup

Abstract

To reorganize a striped file system, data blocks are sequentially moved from storage locations in original data storage to storage locations in an array including the original data storage and additional data storage. If the new striping has the same precedence order as the original striping, then temporary storage is not needed for the block movement. Otherwise, stripes of the data blocks are sequentially moved to a stripe of temporary storage locations, and moved from the stripe of the temporary locations to storage locations in the array. Once a pivot point is reached, however, there is no need to use the temporary storage. Moreover, there is an increasing separation between the read and write pointers. Once this separation exceeds the stripe length, the file system metadata can be synchronized at decreasing intervals and still permit concurrent read access. The number of block transfers between the synchronization points can increase exponentially, and the synchronization points can be aligned with extent boundaries.

Claims

1. A method of operating a data storage system for on-line expansion of a file system so that the file system uses additional data storage added to original data storage of the data storage system, data of the file system residing in the original data storage prior to the expansion, said method comprising the steps of reorganizing at least a portion of the data of the file system by moving some of the data of the file system from the original data storage to the additional data storage so tat the data of the file system is distributed over the original data storage and the additional data storage, and a file of the file system is reorganized by distributing data of the file over the original data storage and the additional data storage so that one portion of the data of the file remains in the original data storage and another portion of the data of the file is moved from we original data storage to the additional data storage, and repetitively updating metadata of the file during the movement of data of the file to permit a host processor to access concurrently the data of the file during the reorganization of the file;

wherein data of the file is sequentially striped across the original data storage prior to the expansion, and during the expansion data of the file is sequentially striped across the original data storage and the additional data storage; and

wherein an order of striping of data of the file in the original data storage prior to the expansion is not preserved during the striping of data across the original data storage and the additional data storage during the expansion, and the movement of data of the file from the original data storage to the additional data storage is performed up to a pivot point by using temporary storage for the data that is moved, and after the pivot point the movement of data of the file from the original data storage to the additional data storage is performed without using temporary storage for the data that is moved.

2. The method as claimed in claim 1, which includes the data storage system sending a synchronization command to the host processor after each repetitive updating of the metadata of the file system.

3. The method as claimed in claim 1, wherein an exponentially increasing amount of data of the file system is moved from the original data storage to the additional data storage between the repetitive updating of the metadata of the file system.

4. The method as claimed in claim 1, wherein the data storage system performs the movement of data of the file system in response to data access commands from an increasing number of host processors between the repetitive updates of the metadata of the file system.

5. The method as claimed in claim 1, wherein the striping of data across the original data storage and the additional data storage permits simultaneous access to the original data storage and the additional data storage during access to the file.

6. The method as claimed in claim 5, wherein the original data storage and the additional data storage are provided by disk drives in the data storage system, and the striping of data across the original data storage and the additional data storage permits simultaneous access to a greater number of the disk drives during access to the file after the reorganization of the file system than before the reorganization of the file system.

7. The method as claimed in claim 1, wherein the original data storage and the additional data storage are provided by disk drives in the data storage system, the host processor accesses data in the disk drives by specifying logical track addresses to the data storage system, and the striping of data of the file during the reorganization process preserves a linear relationship between the logical track addresses and physical track addresses of data of the file in the disk drives.

8. The method as claimed in claim 1, wherein the data storage system is a cached disk array in a file server, the file server also includes a plurality of stream server processors, the stream server processors function as host processors by servicing client requests for file data being reorganized, and the stream servers service the client requests concurrently with the reorganization of the file data.

9. The method as claimed in claim 8, wherein the file server further includes a controller server, and the method includes the controller server controlling the expansion and reorganization of the file system.

10. A data storage system comprising:

original data storage containing a file system; and

a data processor for accessing the file system in the data storage, the data processor being programmed for on-line expansion of the file system so that the file system may use additional data storage added to the original data storage, the data processor being programmed to perform the on-line expansion by moving some of the data of the file system from the original data storage to the additional data storage so that the data of the file system is distributed over the original data storage and the additional data storage, and a file of the file system is reorganized by distributing data of the file over the original data storage and the additional data storage so that one portion of the data of the file remains in the original data storage and another portion of the data of the file is moved from the original data storage to the additional data storage, and repetitively updating metadata of the file during the movement of data of the file to permit a host processor to access concurrently the data of the file during the reorganization of the file;

wherein data of the file is sequentially striped across the original data storage and the additional data storage; and

wherein the data processor is programmed to move data of the file from the original data storage to the additional data storage up to a pivot point by using temporary storage for the data that is moved, and after the pivot point to move the data of the file from the original data storage to the additional data storage without using temporary storage for the data that is moved.

11. The data storage system as claimed in claim 10, wherein the data processor is programmed for sending a synchronization command to the host processor after each repetitive updating of the metadata of the file system.

12. The data storage system as claimed in claim 10, wherein the data processor is programmed for moving an exponentially increasing amount of data of the file system from the original data storage to the additional data storage between the repetitive updating of the metadata of the file system.

13. The data storage system as claimed in claim 10, wherein the data processor is programmed to delegate the process of moving data of the file system to an increasing number of host processors between the repetitive updates of the metadata of the file system.

14. The data storage system as claimed in claim 10, wherein the striping of data across the original data storage and the additional data storage permits simultaneous access to the original data storage and the additional data storage during access to the file.

15. The data storage system as claimed in claim 14, wherein the original data storage and the additional data storage are provided by disk drives in the data storage system, and the striping of data across the original data storage and the additional data storage permits simultaneous access to a greater number of the disk drives during access to the file after the reorganization of the file system than before the reorganization of the file system.

16. The data storage system as claimed in claim 10, wherein the original data storage and the additional data storage are provided by disk drives in the data storage system, the host processor accesses data in the disk drives by specifying logical track addresses to the data storage system, and the striping of data of the file system during reorganization preserves a linear relationship between the logical track addresses and physical track addresses of data of the file system in the disk drives.

17. The data storage system as claimed in claim 10, wherein the data storage is a cached disk array in a file server, the file server also includes a plurality of stream server processors, the stream server processors are programmed to function as host processors by servicing client requests for file data being reorganized, and the stream servers are programmed to service the client requests concurrently with the reorganization of the file data.

18. The data storage system as claimed in claim 17, wherein the data processor is a controller server for the file server.

19. A program storage device containing a program for a data processor of a data storage system, the program being executable by the data processor for on-line expansion of a file system so that the file system may use additional data storage added to original data storage, the program being executable by the data processor to perform the on-line expansion by moving some of the data of the file system from the original data storage to the additional data storage so that the data of the file system is distributed over the original data storage and the additional data storage, and a file of the file system is reorganized by distributing data of the file over the original data storage and the additional data storage so that one portion of the data of the file remains in the original data storage and another portion of the data of the file is moved from the original data storage to the additional data storage, and repetitively updating metadata of the file during the movement of data of the file to permit a host processor to access concurrently the data of the file during the reorganization of the file;

wherein data of the file is sequentially striped across the original data storage and the additional data storage, and the data processor is programmed to move data of the file from the original data storage to the additional data storage up to a pivot point by using temporary storage for the data that is moved, and after the pivot point to move the data of the file from the original data storage to the additional data storage without using temporary storage for the data that is moved.

20. The program storage device as claimed in claim 19, wherein the program is executable by the data processor for sending a synchronization command to the host processor after each repetitive updating of the metadata of the file system.

21. The program storage device as claimed in claim 19, wherein the program is executable by the data processor for moving an exponentially increasing amount of data of the file system from the original data storage to the additional data storage between the repetitive updating of the metadata of the file system.

22. The program storage device as claimed in claim 19, wherein the program is executable by the data processor for delegating the process of moving data of the file system to an increasing number of host processors between the repetitive updates of the metadata of the file system.