Transfer control device, non-transitory computer-readable storage medium storing program, and storage apparatus

• US 9,356,992 B2
• Filed: 12/20/2012
• Issued: 05/31/2016
• Est. Priority Date: 03/16/2012
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Ozawa, Toshihiro, Ogihara, Kazutaka, Noguchi, Yasuo, Kumano, Tatsuo, Tamura, Masahisa, Kato, Jun, Iizawa, Ken, Oe, Kazuichi, Maeda, Munenori
• Primary Examiner(s): Etienne, Ario
• Assistant Examiner(s): Waugh, Thorne E
• Application Number:
  US13/722,331
• Publication Number:
  US 20130246580A1
• Time in Patent Office: 1,258 Days
• US Class Current: 1/1
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A transfer data block is selected, and a data file including the transfer data block is selected as a transfer data file. All the data blocks of the transfer data file are added to the transfer data block. When all the data files including each transfer data block is selected as a transfer data file, the transfer data block is selected as a deletion data block. When a total data amount of the deletion data block is larger than or equal to a predetermined target deletion data amount and a ratio of a total data amount of the transfer data block to the total data amount of the deletion data block is smaller than or equal to a predetermined value, the data file is transferred from a transfer source node to a transfer destination node. Accordingly, a storage can be used efficiently.

Claims

1. A transfer control device that is provided in a distributed storage system, the transfer control device comprising:

a memory; and

a processor configured to;

store, for each of a plurality of data blocks generated by dividing a data file among a plurality of data files, data block count information that is used to count a number of occurrence of that particular data block in the plurality of data files in the memory;

select a transfer data block among the plurality of data blocks, when there is an imbalance in respective amounts of data stored in nodes of a plurality of nodes in the distributed storage system, in an ascending order of the number of occurrence of the selected data block in data files among the plurality of data files, based on the data block count information, for relocating the data file in order to correct the imbalance, from a source node to a destination node, among the plurality of nodes;

select the data file and at least one different data file including a same content as the selected transfer data block as transfer data files;

add all data blocks included in the transfer data files to the transfer data block;

select the transfer data block as a deletion data block, when all the data files including the transfer data block are selected as the transfer data files; and

transfer the data files from the source node to the destination node by storing the transfer data block in the destination node and deleting the deletion data block from the source node.

2. The transfer control device according to claim 1, wherein when a total data amount of the deletion data block is larger than or equal to a predetermined target deletion data amount and a ratio of a total data amount of the transfer data block to a total data amount of the deletion data block is smaller than or equal to a predetermined value, the processor transfers the data files from the source node to the destination node.

3. A non-transitory computer-readable storage medium storing a program causing a computer that is provided in a distributed storage system, to execute the processing of:

storing, for each of a plurality of data blocks generated by dividing a data file among a plurality of data files, data block count information that is used to count a number of occurrence of that particular data block in the plurality of data files in the memory;

selecting a transfer data block among the plurality of data blocks, when there is an imbalance in respective amounts of data stored in nodes of a plurality of nodes in the distributed storage system, in an ascending order of the number of occurrence of the selected data block in data files among the plurality of data files, based on the data block count information, for relocating the data file in order to correct the imbalance, from a source node to a destination node, among the plurality of nodes;

selecting the data file and at least one different data file including a same content as the selected transfer data block as transfer data files;

adding all data blocks included in the selected transfer data files to the transfer data block;

selecting the transfer data block as a deletion data block, when all the data files including the transfer data block are selected as the transfer data files; and

transferring the data files from the transfer source node to the destination node by storing the transfer data block in the destination node and deleting the deletion data block from the source node.

4. The non-transitory computer-readable storage medium according to claim 3, wherein when a total data amount of the deletion data block is larger than or equal to a predetermined target deletion data amount and a ratio of a total data amount of the transfer data block to a total data amount of the deletion data block is smaller than or equal to a predetermined value, the data file is transferred from the source node to the destination node.

5. A storage apparatus that is provided in a distributed storage system, the storage apparatus comprising:

a data storage unit capable of storing a plurality of data files;

a memory; and

a processor, the processor being configured to;

generate a plurality of data blocks by dividing a data file among the plurality of data files, perform de-duplication between the generated data blocks, and store the de-duplicated data blocks in the data storage unit;

store, for each of the plurality of data blocks, data block count information that is used to count a number of occurrence of that particular data block in the plurality of data files in the memory;

select a transfer data block among the plurality of data blocks, when there is an imbalance in respective amounts of data stored in nodes of a plurality of nodes in the distributed storage system, in an ascending order of the number of occurrence of the selected data block in data files among the plurality of data files, based on the data block count information, the transfer data block to be transferred from a source node to a destination node in order to correct the imbalance;

select the data file and at least one different data file including a same content as the selected transfer data block as transfer data files;

add all data blocks included in the selected transfer data files to the transfer data block;

select the transfer data block as a deletion data block, when all the data files including the transfer data block are selected as the transfer data files; and

transfer the data files from the source node to the destination node by storing the transfer data block in the node and deleting the deletion data block from the source node.

6. The storage apparatus according to claim 5, wherein when a total data amount of the deletion data block is larger than or equal to a predetermined target deletion data amount and a ratio of a total data amount of the transfer data block to a total data amount of the deletion data block is smaller than or equal to a predetermined value, the processor transfers the data files from the source node to the transfer node.