Distributed storage system, method thereof and computer readable medium storing distributed storage program

• US 8,539,069 B2
• Filed: 02/03/2011
• Issued: 09/17/2013
• Est. Priority Date: 03/31/2010
• Status: Expired due to Fees

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Kubota, Tomonori
• Primary Examiner(s): Ly, Anh-Vu
• Assistant Examiner(s): BHATTI, HASHIM S
• Application Number:
  US13/020,230
• Publication Number:
  US 20110246643A1
• Time in Patent Office: 957 Days
• Field of Search: 709/238, 709/239, 709/240, 709/241, 709/242
• US Class Current: 709/224
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04N 21/2182 involving memory arrays, e.g. RAID disk arrays RAID arrays per se G06F3/0689; use of parity to protect data in RAID systems G06F11/1008
• CPC Class Codes: H04N 21/23103 using load balancing strategies, e.g. by placing or distributing content on different disks, different memories or different servers storage management G06F3/0604; allocation of resources considering the load in multiprogramming arrangements G06F9/505; techniques for rebalancing the load in a distributed system G06F9/5083; access to distributed or replicated servers, e.g. load balancing, in data networks H04L67/1001
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876

Abstract

A distributed storage system includes, a plurality of storages configured to store a plurality of items of block data obtained by dividing an original data in a distributed manner, a route planning unit configured to obtain unused bandwidth information of a plurality of networks that are connected to the plurality of storages, and a balancing control unit configured to obtain block data that is a target of data transfer among the plurality of the storages based on the unused bandwidth information of the plurality of networks, and position information indicating a position of the block data in the original data.

Claims

1. A distributed storage system, comprising:

a plurality of storages configured to store a plurality of items of block data obtained by dividing an original data in a distributed manner;

a memory; and

a processor executing an instruction stored in the memory to implement;

obtaining unused bandwidth information of a plurality of networks that are connected with the plurality of storages; and

selecting block data that is a target of a data transfer from a transfer source storage among the plurality of storages based on the unused bandwidth information, and position information indicating a position of the block data in the original data such that a block number of the position being above the threshold of the transfer source storage and a block number of the position being below the threshold of a transfer destination storage are alternatively arranged as a result of the data transfer.

2. The system according to claim 1, wherein a transfer request is transmitted to a transfer source storage of data transfer among the plurality of storages requesting the transfer source storage to transfer the block data to a transfer destination storage of the data transfer among the plurality of storages.

3. The system according to claim 2, wherein a size and the position information of the block data is transmitted together with the transfer request to the transfer source storage of the data transfer among the plurality of storages.

4. The device according to claim 1, comprising:

monitoring a plurality of networks connected with the plurality of storages and to obtain bandwidth information of the plurality of networks; and

wherein the unused bandwidth information of the plurality of networks is obtained based on the bandwidth information of the plurality of networks.

5. The system according to claim 1, wherein a number of items of the block data is obtained so that a ratio of the number of items of the block data stored in each of the plurality of storages becomes substantially equal to a ratio of the unused bandwidth information for the plurality of networks.

6. The system according to claim 5, comprising:

determining, for each of the plurality of storages, the block data so that a ratio of the number of items of block data for each data local position defined based on the position information for a plurality of items of consecutive block data in the original data becomes substantially equal to a ratio of unused bandwidth information for the plurality of networks.

7. The system according to claim 1, wherein a threshold of a number of items of block data is defined for each of data local positions in each of the plurality of storages according to a ratio of the unused bandwidth information for the plurality of networks, and for each data local position, a storage that stores the number of items of block data that exceeds the threshold is assumed to be a transfer source of data transfer among the plurality of storages and a storage that stores the number of items of block data below the threshold is assumed to be a transfer destination storage of the data transfer among the plurality of storages.

8. The system according to claim 7, comprising:

transmitting the transfer request for one of one block data item and a plurality of items of block data selected from the block data based on unused bandwidth information for the plurality of networks to a transfer source storage of data transfer among the plurality of storages.

9. A computer-readable non transitory storage medium storing a balancing control processing program that causes a computer having a distributed storage system to execute a process, comprising:

obtaining unused bandwidth information for a plurality of networks connected with a plurality of storages of the distributed storage system, said plurality of storages being configured to store a plurality of items of block data obtained by dividing an original data in a distributed manner; and

selecting block data that is a target of data transfer from a transfer source storage among the plurality of storages based on the unused bandwidth information of the plurality of networks and position information that indicates a position of the block data in the original data such that a block number of the position being above the threshold of the transfer source storage and a block number of the position being below the threshold of a transfer destination storage are alternatively arranged as a result of the data transfer.

10. The computer-readable non transitory medium according to claim 9, the program causing the computer to execute the process comprising:

transmitting a transfer request to a transfer source storage of data transfer among the plurality of storages requesting the transfer source storage to transfer the block data to a transfer destination storage of data transfer among the plurality of storages.

11. The computer-readable non transitory medium according to claim 10, the program causing the computer to execute the process comprising:

transmitting a size and the position information of the block data to the transfer source storage of data transfer among the plurality of storages together with the transfer request.

12. The computer-readable non transitory medium according to claim 9, the program causing the computer to execute the process comprising:

monitoring a plurality of networks connected with the plurality of storages and obtaining bandwidth information of the plurality of networks; and

wherein the obtaining of the unused bandwidth information is based on bandwidth information of the plurality of networks.

13. The computer-readable non transitory medium according to claim 9, the program causing the computer to execute the process comprising:

obtaining a number of items of the block data so that a ratio of the number of items of the block data stored in each of the plurality of storages becomes substantially equal to a ratio of the unused bandwidth information for the plurality of networks.

14. The computer-readable non transitory medium according to claim 13, the program causing the computer to execute the process comprising:

obtaining the block data, for each of the plurality of storages, so that a ratio of the number of items of block data for each data local position defined based on the position information for a plurality of items of consecutive block data in the original data becomes substantially equal to a ratio of unused bandwidth information for the plurality of networks.

15. The computer-readable non transitory medium according to claim 9, the program causing the computer to execute the process comprising:

defining a threshold of a number of items of block data for each of the data local positions in each of the plurality of storages according to a ratio of the unused bandwidth of the plurality of networks, and for each data local position, assuming a storage that stores the number of items of block data that exceeds the threshold as a transfer source storage of data transfer among the plurality of storages and assuming a storage that stores the number of items of block data below the threshold as a transfer destination storage of data transfer among the plurality of storages.

16. The computer-readable non transitory medium according to claim 15, the program causing the computer to execute the process comprising:

transmitting the transfer request to the transfer source storage of data transfer among the plurality of storages for one of one block data item or a plurality of items of block data selected from the block data based on unused bandwidth information for the plurality of networks.

17. A method of controlling distributed storage system, comprising:

executing by a processor;

dynamically determining unused bandwidth of across available storages of the distributed storage system; and

adjustably controlling transfer of a target block data of a divided original data based on the unused bandwidth dynamically determined and content of the original data, where the target block data is selected such that a block number of the position being above the threshold of a transfer source storage and a block number of the position being below the threshold of a transfer destination storage are alternatively arranged as a result of the data transfer.