CONSTRUCTING A SCALABLE STORAGE DEVICE, AND SCALED STORAGE DEVICE

• US 20160306550A1
• Filed: 04/12/2016
• Published: 10/20/2016
• Est. Priority Date: 04/15/2015
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Liu, Hui, Gao, Vivian Wenwen, Di, Jieming, Cao, Yu, Guo, Xiaoyan
• Granted Patent: US 11,070,979 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 13/16 for access to memory bus G06F13/28 takes precedence
• CPC Class Codes: G06F 13/4068 Electrical coupling
• CPC Class Codes: G06F 16/2255 Hash tables
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 21/6272 by registering files or documents with a third party
• CPC Class Codes: G06F 2221/2111 Location-sensitive, e.g. geographical location, GPS
• CPC Class Codes: G06F 2221/2143 Clearing memory, e.g. to prevent the data from being stolen
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0623 in relation to content
• CPC Class Codes: G06F 3/0629 Configuration or reconfiguration of storage systems
• CPC Class Codes: G06F 3/0637 Permissions
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: H04W 12/08 Access security

Abstract

Embodiments of the present disclosure relate to a method an apparatus and a computer program product for constructing a scalable storage device by constructing the scalable storage device by combining a plurality of modularized building blocks; wherein each modularized building block in the plurality of modularized building blocks comprises a magnetic disk enclosure; and at least one modularized building block in the plurality of modularized building blocks comprises a storage processor, the storage processor comprising an input output processing unit; forming a cluster using the input output processing unit in the at least one modularized building block; and processing, using the cluster, an input or output (I/O) request from a host and metadata service.

Claims

1. A method of constructing a scalable storage device, the method comprising:

constructing a scalable storage device by combining a plurality of modularized building blocks;

wherein each modularized building block comprises a magnetic disk enclosure; and

at least one modularized building block in the plurality of modularized building blocks comprises a storage processor, the storage processor comprising an input output processing unit;

forming a cluster using the input output processing unit in the at least one modularized building block; and

processing, using the cluster, at least one of an input or an output (I/O) request from a host and a metadata service.

2. The method according to claim 1, wherein only a first modularized building block in the plurality of modularized building blocks comprises the storage processor;

and wherein constructing the scalable storage device by combining a plurality of modularized building blocks comprises performing at least one of;

constructing the scalable storage device by coupling the first modularized building block to each of other modularized building blocks in the plurality of modularized building blocks;

OR

constructing the scalable storage device by interconnecting each modularized building block in the plurality of modularized building blocks.

3. The method according to claim 1, wherein the plurality of modularized building blocks comprise a first group of modularized building blocks and a second group of modularized building blocks, and only the first group of modularized building blocks comprises the storage processor;

and wherein constructing the scalable storage device by combining a plurality of modularized building blocks comprises;

constructing the scalable storage device by coupling each modularized building block in the first group of modularized building blocks and by coupling each modularized building block in the first group of modularized building blocks with one or more modularized building blocks in the second group of modularized building blocks.

4. The method according to claim 1, wherein forming a cluster using the input output processing unit in the at least one modularized building block further comprises:

selecting one input output processing unit in the cluster as a head of the cluster;

wherein the head of the cluster serves the a metadata updating request; and

each input output processing unit in the cluster capable of providing the metadata service and a data service.

5. The method according to claim 4, further comprises:

selecting another input output processing unit in the cluster as a new head of the cluster when the head of the cluster fails.

6. The method according to claim 1, wherein processing, using the cluster, at least one of an input or an output (I/O) request from a host and a metadata service comprises performing at least one of:

on detecting commencing of operations by one input output processing unit, notifying other input output processing units of a local magnetic disk attached to the one input output processing unit through the metadata service;

OR

determining a storage location for data according to a consistent Hash algorithm, such that the data is evenly distributed across the plurality of modularized building blocks.

7. The method according to claim 6, wherein determining a storage location for data according to a consistent Hash algorithm comprises:

computing a Hash value based on a volume identifier and an offset value the I/O request;

determining a list of hard disk drivers corresponding to the Hash value;

querying a metadata service, to determine input output processing units directly attached to the hard disk drivers in the list, and obtain an input output load condition of each input output processing unit in the determined input output processing units; and

selecting, based on a result of the querying, an input output processing unit for processing the I/O request from among the determined input output processing units directly attached to the hard disk drivers in the list.

8. The method according to claim 7, wherein a number of the hard disk drivers included in the list is more than 1, and the number defined by a terminal user.

9. The method according to claim 7, further comprises:

sending the I/O request to the selected input output processing unit for processing the I/O request.

10. An apparatus of constructing a scalable storage device, comprising at least one processor;

and at least one memory comprising computer program codes, wherein the at least one memory and the computer program codes are configured, with the at least one processor, configured to;

construct a scalable storage device by combining a plurality of modularized building blocks;

wherein each modularized building block comprises a magnetic disk enclosure; and

at least one modularized building block in the plurality of modularized building blocks comprises a storage processor, the storage processor comprising an input output processing unit;

form a cluster using the input output processing unit in the at least one modularized building block; and

process, using the cluster, at least one of an input or an output (I/O) request from a host and a metadata service.

11. The apparatus according to claim 10, wherein only a first modularized building block in the plurality of modularized building blocks comprises the storage processor;

and wherein constructing the scalable storage device by combining a plurality of modularized building blocks comprises performing at least one of;

constructing the scalable storage device by coupling the first modularized building block to each of other modularized building blocks in the plurality of modularized building blocks;

OR

constructing the scalable storage device by interconnecting each modularized building block in the plurality of modularized building blocks.

12. The apparatus according to claim 10, wherein the plurality of modularized building blocks comprise a first group of modularized building blocks and a second group of modularized building blocks, and only the first group of modularized building blocks comprises the storage processor;

and wherein constructing the scalable storage device by combining a plurality of modularized building blocks comprises;

constructing the scalable storage device by coupling each modularized building block in the first group of modularized building blocks and by coupling each modularized building block in the first group of modularized building blocks with one or more modularized building blocks in the second group of modularized building blocks.

13. The apparatus according to claim 10, further configured to:

select one input output processing unit in the cluster as a head of the cluster; and

wherein the head of the cluster serves a metadata updating request; and

each input output processing unit in the cluster capable of providing the metadata service and data service.

14. The apparatus according to claim 10, further configured to select another input output processing unit in the cluster as a new head of the cluster when the head of the cluster fails.

15. The apparatus according claim 10, further configured to perform at least one of:

on detecting commencing of operations by when one input output processing unit is started, notify notifying other input output processing units of a local magnetic disk attached to the one input output processing unit through the metadata service;

OR

determine a storage location for data according to a consistent Hash algorithm, such that the data is evenly distributed across all of the storage processors.

16. The apparatus according to claim 15, wherein determining a storage location for data according to a consistent Hash algorithm comprises:

computing a Hash value based on a volume identifier and an offset value in the I/O request;

determining a list of hard disk drivers corresponding to the Hash value;

querying a metadata service to determine input output processing units directly attached to the hard disk drivers in the list, and obtain an input output load condition of each input output processing unit in the determined input output processing units; and

selecting, based on a result of the querying, an input output processing unit for processing the I/O request from among the determined input output processing units directly attached to the hard disk drivers in the list.

17. The apparatus according to claim 16, wherein a number of the hard disk drivers included in the list is more than 1, and the number defined by a terminal user.

18. The apparatus according to claim 16, further configured to:

send the I/O request to the selected input output processing unit for processing the I/O request.

19. A computer program product, the computer program product being embodied in a computer-readable medium and includes a computer readable program instruction, and when being loaded into an apparatus, the computer-readable program instruction performs

constructing a scalable storage device by combining a plurality of modularized building blocks;

wherein each modularized building block comprises a magnetic disk enclosure; and

at least one modularized building block in the plurality of modularized building blocks comprises a storage processor, the storage processor comprising an input output processing unit; and

wherein constructing the scalable storage device by combining a plurality of modularized building blocks comprises performing at least one of;

constructing the scalable storage device by coupling the first modularized building block to each of other modularized building blocks in the plurality of modularized building blocks;

OR

constructing the scalable storage device by interconnecting each modularized building block in the plurality of modularized building blocks;

forming a cluster using the input output processing unit in the at least one modularized building block, and wherein forming a cluster using the input output processing unit in the at least one modularized building block further comprises;

selecting one input output processing unit in the cluster as a head of the cluster;

wherein the head of the cluster serves the a metadata updating request; and

each input output processing unit in the cluster capable of providing the metadata service and a data service, and selecting another input output processing unit in the cluster as a new head of the cluster when the head of the cluster fails; and

processing, using the cluster, at least one of an input or an output (I/O) request from a host and a metadata service, and wherein processing, using the cluster, at least one of an input or an output (I/O) request from a host and a metadata service comprises performing at least one of;

on detecting commencing of operations by one input output processing unit, notifying other input output processing units of a local magnetic disk attached to the one input output processing unit through the metadata service;

OR

determining a storage location for data according to a consistent Hash algorithm, such that the data is evenly distributed across the plurality of modularized building blocks, and wherein determining a storage location for data according to a consistent Hash algorithm comprises;

computing a Hash value based on a volume identifier and an offset value the I/O request;

determining a list of hard disk drivers corresponding to the Hash value;

querying a metadata service, to determine input output processing units directly attached to the hard disk drivers in the list, and obtain an input output load condition of each input output processing unit in the determined input output processing units; and

selecting, based on a result of the querying, an input output processing unit for processing the I/O request from among the determined input output processing units directly attached to the hard disk drivers in the list, and wherein a number of the hard disk drivers included in the list is more than 1, and the number defined by a terminal user; and

sending the I/O request to the selected input output processing unit for processing the I/O request

20. The method according to claim 19, wherein the plurality of modularized building blocks comprise a first group of modularized building blocks and a second group of modularized building blocks, and only the first group of modularized building blocks comprises the storage processor;

and wherein constructing the scalable storage device by combining a plurality of modularized building blocks comprises;

constructing the scalable storage device by coupling each modularized building block in the first group of modularized building blocks and by coupling each modularized building block in the first group of modularized building blocks with one or more modularized building blocks in the second group of modularized building blocks.