Partition Extension Method and Apparatus

• US 20150293707A1
• Filed: 06/26/2015
• Published: 10/15/2015
• Est. Priority Date: 12/27/2012
• Status: Active Grant

• Current Assignee: Xfusion Digital Technologies Company Limited
• Original Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: Zhang, Feng
• Granted Patent: US 9,665,284 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 12/0223 User address space allocation, e.g. contiguous or non contiguous base addressing
• CPC Class Codes: G06F 2212/1048 Scalability
• CPC Class Codes: G06F 2212/152 Virtualized environment, e.g. logically partitioned system
• CPC Class Codes: G06F 2212/154 Networked environment
• CPC Class Codes: G06F 2212/254 Distributed memory
• CPC Class Codes: G06F 2212/261 Storage comprising a plurality of storage devices
• CPC Class Codes: G06F 2212/263 Network storage, e.g. SAN or NAS
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/0629 Configuration or reconfiguration of storage systems
• CPC Class Codes: G06F 3/0632 by initialisation or re-initialisation of storage systems
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

In a distributed storage system, a method for extending a number N of data node devices comprises a server receives an instruction of adding X new data nodes into the distributed storage system. Then, the server obtains a number M of the all partitions included in a hash ring and determines that M/(N+X) is lower than a preset threshold. The hash ring is organized according to a distributed hash table (DHT), and includes a plurality of partitions. Each partition is mapping to a data node. Based upon the determination, the server generates new partitions by multiplying partitions mapping to each data node, based upon the determination. After storing mapping relationship between the new partitions and the X new data nodes, the server adds the X new data nodes into the distributed storage system.

Claims

1. A method for extending N data node devices included in a distributed storage system, which further includes a server, wherein the server stores a hash ring of data nodes organized according to a distributed hash table (DHT), wherein the hash ring includes a plurality of partitions, and wherein each partition is mapped to a data node, the method performed by the server comprising:

receiving an instruction of adding X new data nodes into the distributed storage system, wherein X is not less than 1;

obtaining M of the partitions included in the hash ring;

determining that M/(N+X) is lower than a preset threshold;

generating new partitions by multiplying and mapping partitions to each data node, based upon the determination;

storing mapping relationship between the new partitions and the X new data nodes; and

adding the X new data nodes into the distributed storage system.

2. The method according to claim 1, wherein obtaining M of the partitions comprises obtaining M of the partitions according to the DHT.

3. A method for migrating data from a data node to a newly added data node in a distributed storage system which includes a plurality of data nodes and a server, wherein the server stores a hash ring of the data nodes organized according to a distributed hash table (DHT), wherein the hash ring includes a plurality of partitions, the method performed by the server comprising:

obtaining data stored in an original data node included in the distributed storage system, wherein the original data node is corresponding to at least one first partition which are a part of the partitions included in the hash ring;

receiving an instruction of adding at least one new data node into the distributed storage system;

generating at least one second partition by multiplying the first partitions;

adding the second partitions into the hash ring;

storing mapping relation between the second partitions and the new data nodes;

adding the new data nodes into the distributes storage system;

generating a key by performing a hashing operation on the data;

applying the key to the hash ring added the second partitions to identify a matching partition;

determining a target data node from the new data nodes, wherein the target data node is corresponding to the matching partition, based upon the matching partition and the mapping relation; and

migrating the data from the original data node to the target data node.

4. A server in a distributed storage system which includes N data nodes, comprising:

a memory configured to store a hash ring of data nodes organized according to a distributed hash table (DHT), wherein the hash ring includes a plurality of partitions, and wherein each partition is mapped to a data node; and

a processor configured to;

receive an instruction of adding X new data nodes into the distributed storage system, wherein X is not less than 1;

obtain M of the partitions included in the hash ring;

determine that M/(N+X) is lower than a preset threshold;

generate new partitions by multiplying and mapping partitions to each data node, based upon the determination;

store mapping relationship between the new partitions and the X new data nodes; and

add the X new data nodes into the distributed storage system.

5. The server according to claim 4, wherein the processor is further configured to obtain M of the partitions comprises obtaining M of the partitions according to the DHT.

6. A server in a distributed storage system which includes a plurality of data nodes, comprising:

a memory configured to store a hash ring of the data nodes organized according to a distributed hash table (DHT), wherein the hash ring includes a plurality of partitions; and

a processor configured to;

obtain data stored in an original data node included in the distributed storage system, wherein the original data node is corresponding to at least one first partition which are a part of the partitions included in the hash ring;

receive an instruction of adding at least one new data node into the distributed storage system;

generate at least one second partition by multiplying the first partitions;

add the second partitions into the hash ring;

store mapping relation between the second partitions and the new data nodes;

add the new data nodes into the distributes storage system;

generate a key by performing a hashing operation on the data;

apply the key to the hash ring added the second partitions to identify a matching partition;

determine a target data node from the new data nodes, wherein the target data node is corresponding to the matching partition, based upon the matching partition and the mapping relation; and

migrate the data from the original data node to the target data node.