METHOD AND APPARATUS FOR MIGRATION OF A VIRTUALIZED FILE SYSTEM, DATA STORAGE SYSTEM FOR MIGRATION OF A VIRTUALIZED FILE SYSTEM, AND FILE SERVER FOR USE IN A DATA STORAGE SYSTEM

• US 20150213032A1
• Filed: 07/02/2013
• Published: 07/30/2015
• Est. Priority Date: 07/02/2013
• Status: Active Grant

• Current Assignee: Hitachi Vantara, LLC (Hitachi, Ltd.)
• Original Assignee: Hitachi Data Systems Engineering Uk Limited (Hitachi, Ltd.)
• Inventors: Powell, Richard, Tabaaloute, Zahra, Gibbs, James, Picken, Daniel, Warner, Timothy, Aston, Chris, Smith, Jon, Brown, Ray, Smith, Joseph, Benson, Michael
• Granted Patent: US 9,454,532 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/119 Details of migration of file systems migration mechanisms in storage systems G06F3/0647
• CPC Class Codes: G06F 16/1824 implemented using Network-attached Storage [NAS] architecture distributed or networked storage systems G06F3/067; protocols for distributed storage of data in a network H04L67/1097
• CPC Class Codes: G06F 16/188 Virtual file systems
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

The present invention relates to a method and an apparatus for data migration of a first file system in a data storage system comprising a plurality of client computers 100, a first file system managing unit 310 for managing the first file system and being connected to the plurality of client computers 100, and a second file system managing unit 410 for managing a second file system and being connected to the first file system managing unit 310, the second file system comprising a plurality of data files and the first file system comprising a plurality of external link objects, each external link object of the first file system being associated with a respective data file of the second file system for enabling client access to the respective data file of the second file system.

Claims

1. An apparatus for data migration of a first file system in a data storage system comprising a plurality of client computers, a first file system managing unit for managing the first file system and being connected to the plurality of client computers, and a second file system managing unit for managing a second file system and being connected to the first file system managing unit,

the second file system comprising a plurality of data files and the first file system comprising a plurality of external link objects, each external link object of the first file system being associated with a respective data file of the second file system for enabling client access to the respective data file of the second file system;

the apparatus comprising the first file system managing unit adapted to;

divide file content data of a first file of the second file system into a first group of migration data blocks, and

migrate file content data of a first file of the second file system from the second file system managing unit to the first file system managing unit in units of migration data blocks of the first group of migration data blocks,

wherein client access to data content of a migration data block of the first group of migration data blocks is temporarily held off while the respective migration data block is migrated from the second file system managing unit to the first file system managing unit, and client access requests directed to the respective migration data block of the first group of migration data blocks received during migration of the respective migration data block are queued and executed after the respective migration data block of the first group of migration data blocks has been migrated.

2. The apparatus according to claim 1, wherein,

client access to data content of the first group of migration data blocks is enabled for all migration data blocks, which have been migrated from the second file system managing unit to the first file system managing unit, and for all migration data blocks, which have not yet started migration from the second file system managing unit to the first file system managing unit.

3. The apparatus according to claim 1, wherein

the first file system managing unit is adapted to determine a block size of data migration data blocks of the first group of migration data blocks as the smaller value of a maximal block size as supported by the first file system management unit for receiving data blocks and the maximal block size as supported by the second file system management unit for sending data blocks.

4. The apparatus according to claim 1, wherein

when a write access request directed to one or more target migration data blocks of the first group of migration data blocks is received at the first file system managing unit from a client computer, the first file system managing unit is further adapted to:

determine whether at least one target migration data block has been migrated already,

wherein, if at least one target migration data block has been migrated already, the write access request is executed locally by the first file system management unit, and, if none of the target migration data blocks has started migration yet, the write access request is passed through to the second file system management unit for being executed in the second file system by the second file system management unit.

5. The apparatus according to claim 4, wherein

if at least one target migration data block has been migrated already, the first file system managing unit is further adapted to further pass through the write access request to the second file system management unit for being executed in the second file system by the second file system management unit.

6. The apparatus according to claim 1, wherein

when a write access request directed to one or more target migration data blocks of the first group of migration data blocks is received at the first file system managing unit from a client computer, the first file system managing unit is further adapted to:

determine whether all target migration data blocks have been migrated already,

wherein, if none of the target migration data blocks has been migrated yet, the write access request is executed locally by the first file system management unit, and,

when later migrating the one or more target migration data blocks of the first group of migration data blocks, the first file system managing unit is adapted to refrain from migrating a data portion corresponding to a data write block of the write access request.

7. The apparatus according to claim 1, wherein

when a write access request directed to one or more target migration data blocks of the first group of migration data blocks is received at the first file system managing unit from a client computer, the first file system managing unit is further adapted to:

whether all target migration data blocks have been migrated already,

wherein, if none of the target migration data blocks has been migrated yet, the write access request is executed locally by the first file system management unit, and,

the first file system managing unit is further adapted to assign a checkpoint number, which is smaller than a current checkpoint number that is assigned to the data write block of the write access request, to a data hole between an end position of an already migrated data portion and a start position of a data write block of the write access request.

8. The apparatus according to claim 1, wherein

when a read access request directed to one or more target migration data blocks of the first group of migration data blocks is received at the first file system managing unit from a client computer, the first file system managing unit is further adapted to:

determine whether all target migration data blocks have been migrated already,

wherein, if all target migration data blocks have been migrated already, the read access request is executed locally by the first file system management unit, and, if at least one of the target migration data blocks has not started migration yet, the read access request is passed through to the second file system management unit for being executed in the second file system by the second file system management unit, and, a corresponding read response received, at the first file system management unit from the second file system management unit, is passed through by the first file system management unit to the requesting client computer.

9. The apparatus according to claim 1, wherein

the first file system managing unit is further adapted to:

determine, at the first file system managing unit, an object store length of file content data stored by the first file system managing unit in the first file system being associated with the first file of the second file system,

determine, at the first file system managing unit based on data received from the second file system managing unit, a length of the first file of the second file system, and

determine that all migration data blocks of the first group have been migrated when it is determined that said object store length is equal to said length of the first file.

10. The apparatus according to claim 9, wherein characterized in that

the first file system managing unit is further adapted to:

delete the external link object of the first file system being associated with the first file of the second file system when it is determined that all migration data blocks of the first group have been migrated; and

/or

send, to the second file system management unit, a request for deletion of the first file of the second file system when it is determined that all migration data blocks of the first group have been migrated.

11. The apparatus according to any claim 1, wherein

the first file system managing unit is further adapted to:

receive from a client computer, a file creation request for creating a second file in the second file system in a path location indicated in the file creation request, and

create the second file in the first file system based on path location indicated in the file creation request.

12. The apparatus according to claim 1,

the first file system managing unit is further adapted to:

determine, for a first migration data block of the first group of migration data blocks, whether said first migration data block has at least one non-zero bit; and

refrain from migrating said first migration data block from the second file system managing unit to the first file system managing unit when it is determined that said first migration data block does not have at least one non-zero bit and/or

update metadata of the first file indicating that a storage region associated with the said first migration data block shall contain only zeros.

13. The apparatus according to claim 12, wherein

determining whether said first migration data block has at least one non-zero bit is performed by the first file system management unit based on data indicative of a position of a first non-zero bit in the file content of the first file in the second file system based on a read response directed to the first file being received from the second file system management unit.

14. The apparatus according to claim 12, wherein

determining whether said first migration data block has at least one non-zero bit is performed by the first file system management unit based on a byte-by-byte analysis of the first migration data block.

15. The apparatus according to claim 1, wherein

the first file system managing unit is further adapted to:

determine, for a second migration data block of the first group of migration data blocks, whether said migration data block has at least one non-zero bit, if a ratio of a usage percentage of storage space allocated for the first file in the second file system and a length of the first file indicating the storage space allocated for the first file in the second file system falls below a predetermined threshold, and

refrain from migrating said second migration data block from the second file system managing unit to the first file system managing unit if it is determined that said second migration data block does not have at least one non-zero bit and/or

update metadata of the first file indicating that a storage region associated with the said first migration data block shall contain only zeros.

16. A method for data migration of a first file system in a data storage system comprising a plurality of client computers, a first file system managing unit for managing the first file system and being connected to the plurality of client computers, and a second file system managing unit for managing a second file system and being connected to the first file system managing unit,

the second file system comprising a plurality of data files and the first file system comprising a plurality of external link objects, each external link object of the first file system being associated with a respective data file of the second file system for enabling client access to the respective data file of the second file system;

the method comprising;

dividing file content data of a first file of the second file system into a first group of migration data blocks, and

dividing file content data of a first file of the second file system from the second file system managing unit to the first file system managing unit in units of migration data blocks of the first group of migration data blocks,

wherein client access to data content of a migration data block of the first group of migration data blocks is temporarily held off while the respective migration data block is migrated from the second file system managing unit to the first file system managing unit, and client access requests directed to the respective migration data block of the first group of migration data blocks received during migration of the respective migration data block are queued and executed after the respective migration data block of the first group of migration data blocks has been migrated.

17. A data storage system for data migration of a first file system, the data storage system comprising a plurality of client computers, a first file system managing unit for managing the first file system and being connected to the plurality of client computers, and a second file system managing unit for managing a second file system and being connected to the first file system managing unit,

the second file system comprising a plurality of data files and the first file system comprising a plurality of external link objects, each external link object of the first file system being associated with a respective data file of the second file system for enabling client access to the respective data file of the second file system;

the first file system managing unit being adapted to;

divide file content data of a first file of the second file system into a first group of migration data blocks, and

migrate file content data of a first file of the second file system from the second file system managing unit to the first file system managing unit in units of migration data blocks of the first group of migration data blocks,

wherein client access to data content of a migration data block of the first group of migration data blocks is temporarily held off while the respective migration data block is migrated from the second file system managing unit to the first file system managing unit, and client access requests directed to the respective migration data block of the first group of migration data blocks received during migration of the respective migration data block are queued and executed after the respective migration data block of the first group of migration data blocks has been migrated.

18. A file system server for use in a data storage system for data migration of a first file system, the data storage system comprising a plurality of client computers, a first file system managing unit for managing the first file system and being connected to the plurality of client computers, and a second file system managing unit for managing a second file system and being connected to the first file system managing unit,

the second file system comprising a plurality of data files and the first file system comprising a plurality of external link objects, each external link object of the first file system being associated with a respective data file of the second file system for enabling client access to the respective data file of the second file system;

the first file system managing unit being adapted to;

divide file content data of a first file of the second file system into a first group of migration data blocks, and

migrate file content data of a first file of the second file system from the second file system managing unit to the first file system managing unit in units of migration data blocks of the first group of migration data blocks,

wherein client access to data content of a migration data block of the first group of migration data blocks is temporarily held off while the respective migration data block is migrated from the second file system managing unit to the first file system managing unit, and client access requests directed to the respective migration data block of the first group of migration data blocks received during migration of the respective migration data block are queued and executed after the respective migration data block of the first group of migration data blocks has been migrated.

19. A non-transitory computer readable storage medium comprising computer program code being configured to cause a processing unit of a file system management unit in a data storage system, comprising a plurality of client computers, a first file system managing unit for managing the first file system and being connected to the plurality of client computers, and a second file system managing unit for managing a second file system and being connected to the first file system managing unit, the second file system comprising a plurality of data files and the first file system comprising a plurality of external link objects, each external link object of the first file system being associated with a respective data file of the second file system for enabling client access to the respective data file of the second file system, to execute the steps:

dividing file content data of a first file of the second file system into a first group of migration data blocks, and

migrating file content data of a first file of the second file system from the second file system managing unit to the first file system managing unit in units of migration data blocks of the first group of migration data blocks,

wherein client access to data content of a migration data block of the first group of migration data blocks is temporarily held off while the respective migration data block is migrated from the second file system managing unit to the first file system managing unit, and client access requests directed to the respective migration data block of the first group of migration data blocks received during migration of the respective migration data block are queued and executed after the respective migration data block of the first group of migration data blocks has been migrated.