Cluster system with calculation and storage converged

• US 10,481,804 B2
• Filed: 09/30/2013
• Issued: 11/19/2019
• Est. Priority Date: 12/31/2012
• Status: Active Grant

• Current Assignee: Huawei Cloud Computing Technologies Company Limited (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: Gu, Jiongjiong, Wang, Daohui, Min, Xiaoyong
• Primary Examiner(s): Keller, Michael A
• Application Number:
  US14/041,986
• Publication Number:
  US 20140189128A1
• Time in Patent Office: 2,241 Days
• CPC Class Codes: G06F 3/06 Digital input from, or digital output to, record carriers , e.g. RAID, emulated record carriers or networked record carriers
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0632 by initialisation or re-initialisation of storage systems
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Embodiments of the present invention provide a cluster system with calculation and storage converged, including a resource server group comprising at least two resource servers. Each resource server in the resource server group has at least one of a calculation resource and a storage resource. At least one resource server in the resource server group has a calculation resource and a storage resource, and the storage resource includes a persistent disk and a cache that corresponds to the persistent disk. A storage resource control module is configured to construct a shared storage resource pool by using storage resources in the resource server group, so as to provide a storage service, where storage resources included in each storage resource pool are from at least two resource servers in the resource server group.

Claims

1. A system comprising:

a first resource server comprising a first calculation resource and a first storage resource, wherein the first storage resource comprises a first persistent disk and a first cache associated with the first persistent disk, wherein the first persistent disk is a hard disk driver (HDD) or a solid state disk (SSD), and wherein the first calculation resource is coupled to the first storage resource through a peripheral component interconnect express (PCIe) interface;

a second resource server comprising a second calculation resource and a second storage resource, wherein the second storage resource comprises a second persistent disk and a second cache associated with the second persistent disk, and the first and second storage resources together form a storage resource pool comprising physical storage resource blocks, and wherein the second persistent disk is a HDD or a SSD and the second calculation resource is coupled to the second storage resource through another PCIe interface, and

a third resource server configured to receive a user request from the first calculation resources or the second calculation resource to create a user volume, and determine physical storage resource blocks to be allocated for the user volume from the physical storage resource blocks of the storage resource pool, wherein the physical storage resource blocks to be allocated for the user volume belong to different persistent disks.

2. The system according to claim 1, wherein one of the first and the second resource servers comprises a cluster view management component, and is configured to logically divide, via the cluster view management component, the first and the second persistent disks of the storage resource pool to obtain logical partitions, wherein each of the logical partitions is associated with a number of the physical storage resource blocks of the storage resource pool.

3. The system according to claim 2, wherein the first resource server further comprises a first data component, the second resource server further comprises a second data component;

the one of the first and the second resource servers comprising the cluster view management component is further configured to;

allocate the logical partitions to the first and the second data components via the cluster view management component, establish a logical partition topology of the first and the second data components, and send allocated logical partition information to the first and the second data components.

4. The system according to claim 2, wherein one of the the third resource server is configured to:

receive a data writing operation request sent by a user, wherein the data writing operation request includes an identifier of the user volume and data to be written;

allocate, according to the identifier of the user volume, physical storage resource blocks to be written from the physical storage resource blocks to be allocated for the user volume;

obtain, logical partitions to which the physical storage resource blocks to be written belong;

determine, according to the logical partitions to which the physical storage resource blocks to be written belong, the first and second resource servers for performing a data writing operation; and

send a first part of the data to the first resource server and a second part of the data to the second resource server.

5. The system according to claim 4, wherein the second resource server is further configured to:

write the second part of the data into the second cache of the second resource server; and

write the second part of the data into the second persistent disk from the second cache of the second resource server.

6. The system according to claim 2, wherein the third resource server is further configured to:

receive a data reading operation request sent by a user, wherein the data reading operation request includes an identifier of the user volume;

obtain, according to the identifier of the user volume, physical storage resource blocks to be read from the physical storage resource blocks to be allocated for the user volume;

obtain, logical partitions to which the physical storage resource blocks to be read belong;

determine, according to the logical partitions to which the physical storage resource blocks to be read belong, resource servers for performing a data reading operation, wherein the resource servers performing the data reading operation comprise the first and second resource servers; and

send a first data reading operation sub-request to the first resource server and a second data reading operation sub-request to the second resource server.

7. The system according to claim 6, wherein the second resource server is further configured to

read data from the second cache of the second resource server in responsive to the second data reading operation sub-request.

8. The system according to claim 6, if data requested by the data reading operation does not exist in the second cache of the second resource server, and the second resource server is further configured to:

read data from the second persistent disk, and write the data into the second cache of the second resource server.

9. The system according to claim 3, wherein if a third resource server that has a third storage resource and a third data component is added in the system or deleted from the system, the one of the first and the second resource servers comprising the cluster view management component is further configured to regenerate the logical partition topology of the first, the second and the third data components according to a preset logical partition allocation algorithm, broadcast the regenerated logical partition topology to the first, the second and the third data components in the system.

10. The system according to claim 1, wherein

each resource server comprises an interface component, and the third resource server is configured to receive the user request to create the user volume via the interface component;

wherein each of the first and second resource servers comprises a data component, configured to perform data writing or reading operation via the data component; and

wherein the third resource server comprises a cluster view management component, configured to manage logical partitions via the cluster view management component wherein the first and second persistent disks of the storage resource pool are logically divided into logical partitions.

11. The system according to claim 1, wherein the first resource server in the system performs communication with the second resource server through a high speed data switch network, wherein the high speed data switch network comprises a high speed peripheral component interconnection express (PCI-E) network or an optical fiber network.

12. The system according to claim 3, wherein the third resource server is configured to:

receive a data writing operation request sent by a user, wherein the data writing operation request includes an identifier of the user and data to be written;

allocate, according to the identifier of the user volume, physical storage resource blocks to be written from the physical storage resource blocks to be allocated for the user volume;

obtain, logical partitions to which the physical storage resource blocks to be written belong;

determine, according to the logical partitions to which the physical storage resource blocks to be written belong, the first and second data components for performing the data writing operation; and

send a first part of the data to the first data component and a second part of the data to the second data component.

13. A system comprising:

a first resource server comprising a first storage resource and a first object storage device (OSD), wherein the first storage resource comprises a first persistent disk and a first cache associated with the first persistent disk, and wherein the first persistent disk is a hard disk driver (HDD) or a solid state disk (SSD);

a second resource server comprising a second storage resource and a second OSD, wherein the second storage resource comprises a second persistent disk and a second cache associated with the second persistent disk, and the first and second storage resources together form a storage resource pool comprising physical storage resource blocks, and wherein the second persistent disk is a HDD or a SSD;

a third resource server comprising a calculation resource; and

one of the plurality of resource servers configured to receive a user request to create a user volume, and determine physical storage resource blocks to be allocated for the user volume from the physical storage resource blocks of the storage resource pool,

wherein the physical storage resource blocks to be allocated for the user volume belong to different persistent disks,

wherein one of the plurality of resource servers comprises a meta data controller (MDC) is configured to logically divide, via the MDC, the first and the second persistent disks of the storage resource pool to obtain logical partitions,

wherein each of the logical partitions is associated with a number of physical storage resource blocks of the storage resource pool, and

wherein the one of the plurality of resource serves comprises the MDC is further configured to allocate the logical partitions to the first and the second OSDs via the MDC, establish a logical partition topology of the first and the second OSDs, and send allocated logical partition information to the first and the second OSDs.

14. A system comprising a plurality of resource servers, the system comprising:

a first resource server comprising a first storage resource that includes a first persistent disk and a first cache associated with the first persistent disk that is hard disk driver (HDD) or a solid state disk (SSD);

a second resource server comprising a second storage resource that includes a second persistent disk and a second cache associated with the second persistent disk, and the first and second storage resources together form a storage resource pool comprising physical storage resource blocks, and wherein the second persistent disk is a HDD or a SSD;

a third resource server comprising a calculation resource;

one of the plurality of resource servers is configured to receive a user request to create a user volume, and determine physical storage resource blocks to be allocated for the user volume from the physical storage resource blocks of the storage resource pool, wherein the physical storage resource blocks to be allocated for the user volume belong to different persistent disks;

one of the plurality of resource servers comprises a meta data controller (MDC) and is configured to logically divide, via the MDC, the first and the second persistent disks of the storage resource pool to obtain logical partitions,

wherein each of the logical partitions is associated with a number of physical storage resource blocks of the storage resource pool; and

one of the plurality of resource servers is configured to;

receive a data writing operation request sent by a user via a virtualization block service (VBS),

wherein the data writing operation request includes an identifier of the user volume and data to be written;

allocate, according to the identifier of the user volume, physical storage resource blocks to be written from the physical storage resource blocks to be allocated for the user volume;

obtain logical partitions to which the physical storage resource blocks to be written belong;

determine, according to the logical partitions to which the physical storage resource blocks to be written belong, the first and second resource servers for performing a data writing operation; and

send a first part of the data to a first object storage device (OSD) of the first resource server, and a second part of the data to a second OSD of the second resource server.

15. The system according to claim 14, wherein the first resource server is further configured to:

write the first part of the data into the first cache of the first resource server; and

write the first part of the data into the first persistent disk from the first cache of the first resource server.

16. A system comprising a plurality of resource servers, the system comprising:

a first resource server comprising a first storage resource that includes a first persistent disk and a first cache associated with the first persistent disk, wherein the first persistent disk is a hard disk driver (HDD) or a solid state disk (SSD);

a second resource server comprising a second storage resource that includes a second persistent disk and a second cache associated with the second persistent disk, and the first and second storage resources together form a storage resource pool comprising physical storage resource blocks, wherein the second persistent disk is a HDD or a SSD;

a third resource server comprising a calculation resource;

one of the plurality of resource servers is configured to receive a user request to create a user volume, and determine physical storage resource blocks to be allocated for the user volume from the physical storage resource blocks of the storage resource pool, wherein the physical storage resource blocks to be allocated for the user volume belong to different persistent disks;

one of the plurality of resource servers comprises a meta data controller (MDC) and is configured to logically divide, via the MDC, the first and the second persistent disks of the storage resource pool to obtain logical partitions, wherein each of the logical partitions is associated with a number of physical storage resource blocks of the storage resource pool; and

one of the plurality of resource servers is configured to;

receive a data reading operation request sent by a user via a virtualization block service (VBS), wherein the data reading operation request includes an identifier of the user volume;

obtain, according to the identifier of the user volume, physical storage resource blocks to be read from the physical storage resource blocks to be allocated for the user volume;

obtain, logical partitions to which the physical storage resource blocks to be read belong;

determine, according to the logical partitions to which the physical storage resource blocks belong, resource servers for performing a data reading operation, wherein the resource servers for performing the data reading operation comprise the first and second resource servers; and

send a first data reading operation sub-request to a first object storage device (OSD) of the first resource server and a second data reading operation sub-request to a second OSD of the second resource server.

17. A system comprising:

a first resource server comprising a first calculation resource and a first storage resource, wherein the first storage resource comprises a first persistent disk and a first cache associated with the first persistent disk, wherein the first persistent disk is a hard disk driver (HDD) or a solid state disk (SSD), and wherein the first calculation resource is coupled to the first storage resource through a peripheral component interconnect express (PCIe) interface;

a second resource server comprising a second calculation resource and a second storage resource, wherein the second storage resource comprises a second persistent disk and a second cache associated with the second persistent disk, and the first and second storage resources together form a storage resource pool comprising physical storage resource blocks, and wherein the second persistent disk is a HDD or a SSD and the second calculation resource is coupled to the second storage resource through another PCIe interface, and

the first resource server is further configured to receive a user request from the first calculation resource or the second calculation resource to create a user volume, and determine physical storage resource blocks to be allocated for the user volume from the physical storage resource blocks of the storage resource pool, wherein the physical storage resource blocks to be allocated for the user volume belong to different persistent disks.