Transferring data between block and file storage systems

• US 10,063,624 B2
• Filed: 12/15/2017
• Issued: 08/28/2018
• Est. Priority Date: 10/31/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Gao, Xue Dong, Nielsen, Karl A., Peng, Yao, Rinaldi, Brian A., Yao, Tang Ren
• Primary Examiner(s): Lazaro, David
• Assistant Examiner(s): Chang, Julian
• Application Number:
  US15/843,167
• Publication Number:
  US 20180124157A1
• Time in Patent Office: 256 Days
• CPC Class Codes: G06F 11/1446 Point-in-time backing up or restoration of persistent data
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0661 Format or protocol conversion arrangements
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A processor may receive a request to transmit a logical volume to a cloud-based server. The logical volume may be stored in a data storage subsystem that includes one or more ranks. Each rank may include one or more extents, which may include one or more stride. The processor may determine an extent of the one or more extents that includes data of the logical volume. The processor may determine a set of strides associated with the extent. The processor may copy the set of strides into a stride buffer and combine the set of strides into a block extent file. The processor may transmit the block extent file to the cloud server.

Claims

1. A method comprising:

receiving a request to transmit a logical volume stored on a host server to a remote server, the logical volume being stored in a data storage system that includes one or more ranks, each rank having one or more extents, each extent having one or more strides,

wherein the logical volume includes one or more distinct logical volumes,

wherein the data storage system is a Network-attached Storage (NAS) device that includes a storage processor, wherein each rank of the one or more ranks is a redundant array of independent disks (RAID) array;

determining that the logical volume is stored using block-level storage;

determining, in response to determining that the logical volume is stored using block-level storage, an extent of the one or more extents that includes data of the logical volume by;

determining one or more logical block address (LBAs) associated with the logical volume;

determining, using the LBAs and a rank segment table, one or more ranks that are associated with the LBAs; and

mapping the one or more LBAs associated with the logical volume to one or more extent ranges using a logic configuration volume segment table;

determining a set of strides associated with the extent;

copying the set of strides for the extent into a stride buffer;

combining the set of strides for the extent into a block extent file;

transmitting the block extent file to the remote server;

generating a master volume table file (MVTF) that stores metadata information of the one or more logical volumes, the MVTF having an entry for each of the one or more logical volumes transferred;

generating, for each transferred logical volume, a volume extent meta file (VEMF) that stores metadata for the associated logical volume,

wherein the metadata information stored in the MVTF and the one or more VEMFs includes file names, sizes, access control lists, and identifications of the ranks and extents from which the one or more logical volumes were copied;

transmitting the master volume table file and the one or more volume extent meta files to the remote server;

receiving a request to restore the logical volume on the host server;

determining, by the remote server and using the MVTF and one or more VEMFs, the set of strides in the data storage system that correspond to the block extent file;

splitting the block extent file into a set of data pieces using the MVTF and the VEMFs, each data piece corresponding to a stride in the set of strides;

transmitting each data piece from the remote server to the stride buffer; and

copying each data piece in the stride buffer over a corresponding stride in the data storage system.