Creating global aggregated namespaces for storage management

• US 9,330,161 B2
• Filed: 04/12/2013
• Issued: 05/03/2016
• Est. Priority Date: 04/12/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: D'Amato, Andrea, Rajpal, Shiv, Domzalski, Gregory Robert, Kuznetsov, Vyacheslav, Havewala, Sarosh Cyrus, Kasturiya, Ankur, Mehra, Karan
• Primary Examiner(s): Rostami, Mohammad S
• Application Number:
  US13/861,997
• Publication Number:
  US 20140310278A1
• Time in Patent Office: 1,117 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Embodiments are directed to creating global, aggregated namespaces for storage management and to providing consistent namespaces in a distributed storage system. In one scenario, a computer system defines data storage objects for each data storage node. The data storage objects uniquely identify storage elements of the data storage nodes, where each data storage object includes various associated attributes. The computer system replicates the defined data storage objects and any associated attributes from a first data storage node to a second, different data storage node among the data storage nodes. As such, the defined data storage objects are visible from any node in the data storage nodes. The computer system also aggregates the defined data storage objects for each of the data storage nodes and creates a global, aggregated namespace that includes the aggregated data storage objects for each of the data storage nodes.

Claims

1. A computer system, comprising:

one or more processors; and

one or more computer-readable storage media having stored thereon computer-executable instructions representing one or more modules that are executable by the one or more processors to create a global, aggregated namespace for storage management in a distributed data storage system comprising a plurality of data storage nodes, the one or more modules being configured to perform at least the following;

define a plurality of data storage objects corresponding to and defining a plurality of hardware storage elements of the plurality of data storage nodes, each data storage object defining at least one hardware storage element of the plurality of hardware storage elements that is made available by at least one of the plurality of data storage nodes, and comprising;

a unique identification of the at least one hardware storage element that is being defined by the data storage object; and

one or more associated attributes that identify whether the at least one hardware storage element is local to a single node of the plurality of data storage nodes, or distributed over two or more of the plurality of data storage nodes;

replicate the plurality of data storage objects across the plurality of data storage nodes, such that the plurality data storage objects, and the plurality of hardware storage elements that are defined by the plurality data storage objects, are visible from any node in the plurality of data storage nodes; and

generate an aggregate view of the plurality of data storage objects, including creating a global, aggregated namespace that includes each of the plurality of data storage objects that are replicated across the plurality of data storage nodes, along with the plurality of hardware storage elements that are defined by the plurality data storage objects.

2. The computer system of claim 1, wherein the one or more modules are also configured to perform a management action using the global, aggregated namespace.

3. The computer system of claim 2, wherein when performing the management action, the one or more modules are configured to determine on which one or more data storage nodes among the plurality of data storage nodes the management action is to be performed.

4. The computer system of claim 3, wherein the one or more modules are configured to route function calls corresponding to the management action to the determined one or more data storage nodes.

5. The computer system of claim 4, wherein when routing function calls corresponding to the management action to the determined one or more data storage nodes, the one or more modules are also configured to:

determine that delivery for at least one of the function calls sent to the determined one or more data storage nodes has failed; and

re-route the at least one function call to at least one other data storage object among the aggregated data storage objects.

6. The computer system of claim 2, wherein the management action comprises monitoring the plurality of data storage objects using the global, aggregated namespace.

7. The computer system of claim 6, wherein when monitoring the plurality of data storage objects, the one or more modules are configured to monitor the plurality of data storage objects from any data storage node of the plurality of data storage nodes.

8. The computer system of claim 6, wherein when monitoring the plurality of data storage objects, the one or more modules are configured to diagnose one or more problems detected during the monitoring.

9. The computer system of claim 1, wherein the one or more hardware storage elements comprise at least one of:

one or more hard disks that are part of a data storage array, one or more storage area networks, and one or more optical drives.

10. The computer system of claim 1, wherein the global, aggregated namespace is resilient, such that users see a coherent, global namespace for the plurality of data storage objects.

11. The computer system of claim 10, wherein the global, aggregated namespace maintains the plurality of data storage objects despite physical data storage device failures.

12. The computer system of claim 1, wherein the one or more modules are also configured to provide a user interface that allows users to enter intent-based commands, the intent-based commands being performed using the global, aggregated namespace.

13. A computer system, comprising:

one or more processors; and

one or more computer-readable storage media having stored thereon computer-executable instructions representing one or more modules that are executable by the one or more processors to provide a consistent namespace in a distributed data storage system comprising a plurality of data storage nodes, the one or more modules being configured to perform at least the following;

receive an input from a user, the input requesting one or more portions of information regarding a plurality of data storage objects corresponding to and defining a plurality of hardware storage elements of the plurality of data storage nodes, each data storage object defining at least one hardware storage element of the plurality of hardware storage elements that is made available by at least one of the plurality of data storage nodes, and comprising;

a unique identification of the at least one hardware storage element that is being defined by the data storage object; and

one or more associated attributes that identify whether the at least one hardware storage element is local to a single node of the plurality of data storage nodes, or distributed over two or more of the plurality of data storage nodes;

determine that the user input was received at a particular data storage node of the plurality of data storage nodes that is internal to the distributed data storage system;

aggregate one or more node-specific data storage objects of the plurality of data storage objects that define one or more of the plurality of hardware storage elements that is made available by the particular data storage node;

aggregate one or more global data storage objects of the plurality of data storage objects for the distributed data storage system; and

create a global, unified namespace for the distributed data storage system, the global, unified namespace including the aggregated node-specific data storage objects and the aggregated global data storage objects.

14. The computer system of claim 13, wherein the global, unified namespace for the distributed data storage system is consistent for each node of the distributed data storage system, regardless of which internal data storage node from which the user input was received.

15. The computer system of claim 13, wherein a single application programming interface (API) is used to manage local resources on the internal particular data storage node and one or more global resources on the distributed data storage system.

16. The computer system of claim 13, wherein data storage objects are automatically generated and aggregated into the global, unified namespace for data storage devices that are added to the distributed data storage system.

17. The computer system of claim 13, wherein the global, unified namespace includes the aggregated global data storage objects as well as node-specific data storage objects that are specific to the node at which the input was received.

18. A computer system, comprising:

one or more processors; and

one or more computer-readable storage media having stored thereon computer-executable instructions representing one or more modules that are executable by the one or more processors to provide a consistent namespace in a distributed data storage system comprising a plurality of data storage nodes, the one or more modules being configured to perform at least the following;

receive an input from a user, the input requesting one or more portions of information regarding a plurality of data storage objects corresponding to and defining a plurality of hardware storage elements of the plurality of data storage nodes, each data storage object defining at least one hardware storage element of the plurality of hardware storage elements that is made available by at least one of the plurality of data storage nodes, and comprising;

a unique identification of the at least one hardware storage element that is being defined by the data storage object; and

one or more associated attributes that identify whether the at least one hardware storage element is local to a single node of the plurality of data storage nodes, or distributed over two or more of the plurality of data storage nodes;

determine that the user input was received at a particular data storage node of the plurality of data storage nodes that is external to the distributed data storage system;

aggregate one or more global data storage objects of the plurality of data storage objects for the distributed data storage system; and

create a global, unified namespace for the distributed data storage system, the global, unified namespace including the aggregated global data storage objects.

19. The computer system of claim 18, wherein the global, unified namespace for the distributed data storage system is consistent for each node of the distributed data storage system, regardless of which external data storage node from which the user input was received.

20. The computer system of claim 18, wherein the one or more modules are also configured to present both a node-specific view for the data storage node from which the user input was received and an aggregated, global view of the data storage objects in the distributed data storage system.