System and method for intelligent, globally distributed network storage

• US 7,509,420 B2
• Filed: 02/05/2001
• Issued: 03/24/2009
• Est. Priority Date: 02/18/2000
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Moulton, Gregory Hagan, Whitehill, Stephen B.
• Primary Examiner(s): PHILLIPS, HASSAN A
• Application Number:
  US09/777,002
• Publication Number:
  US 20010034795A1
• Time in Patent Office: 2,969 Days
• Field of Search: 709/201, 709/204, 709/205, 709/213, 709/214, 709/215
• US Class Current: 709/226
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A data storage system comprising a plurality of storage nodes, each node existing at a physical location having one or more contexts. Interface mechanisms couple to each storage node to communicate storage access requests with the storage node. Data storage management processes select one or more of the storage nodes to serve a data storage request based at least in part upon the particular contexts of each of the storage nodes.

Claims

1. A data storage system comprising:

a plurality of storage nodes, each node existing at a physical location and each storage node having associated contexts that define characteristics of each storage node, wherein first contexts including a political context, an economic context, and a geographic context are related to at least the physical location of each storage node and wherein second contexts including a network topological context are related to at least attributes of each storage node;

interface mechanisms coupled to each storage node for communicating data storage requests with the storage node, at least one data storage access request including performance criteria that define storage characteristics that are desired for data associated with the data storage access requests; and

data storage management processes that select one or more of the storage nodes to serve the at least one data storage access request based at least in part upon the first contexts and the second contexts that satisfy the performance criteria, wherein the first contexts and the second contexts are used to discriminate differences between the plurality of storage nodes and identify the selected one or more storage nodes to serve the at least one data storage access request.

2. The system of claim 1 wherein the data storage management processes comprise computer-implemented processes executing in at least one of the storage nodes.

3. The system of claim 1 wherein the data storage management processes comprise computer-implemented processes executing in all of the storage nodes.

4. The system of claim 1 wherein the performance criteria identify storage characteristics including one or more of cost, location, security, availability, or network connectivity.

5. The system of claim 1 wherein the data storage management processes comprise processes for matching the performance criteria to the first and second contexts of the storage nodes.

6. The data storage system of claim 1 wherein the data storage management processes present a unitary logical volume of data storage to external devices generating the storage access requests to the selected one or more storage nodes.

7. The data storage system of claim 6 wherein the selected one or more storage nodes are selected such that the first and second contexts of the selected one or more storage nodes that are associated with the unitary logical volume satisfy the performance criteria associated with the storage access requests.

8. The data storage system of claim 1 further comprising encryption mechanisms coupled to the interface mechanisms for encrypting storage access requests during communication between nodes.

9. The data storage system of claim 1 further comprising authentication mechanisms coupled to the interface mechanisms for authenticating storage nodes before communicating storage requests.

10. A method of managing distributed data storage comprising the acts of:

providing a plurality of distributed storage nodes, each node existing at a physical location and each node having one or more associated contexts that relate to storage characteristics of each node, the contexts including first contexts including a political context, an economic context, and a geographic context that are associated to at least the physical location and second contexts including a network topological context that are associated with attributes of the storage node;

receiving a data storage task in one of the storage nodes;

determining desired criteria associated with the received data storage task, wherein the desired criteria define storage characteristics for the data storage task;

selecting one or more of the plurality of storage nodes by matching the desired criteria to the associated first and second contexts, wherein the selected one or more of the plurality of storage nodes have first and second contexts that satisfy the desired criteria; and

executing the storage task in the one or more selected storage nodes.

11. The method of claim 10 wherein the selected storage nodes comprise at least two storage nodes where neither of the at least two storage nodes individually satisfy the desired criteria, but collectively the at least two storage nodes satisfy the desired-criteria.

12. The method of claim 10 wherein the selected storage nodes comprise at least two storage nodes and the at least two storage nodes are located in different geographical locations.

13. The method of claim 10 wherein the selected storage nodes comprise at least two storage nodes and the at least two storage nodes are located in different areas of a single data center.

14. The method of claim 10 wherein the selected storage nodes comprise at least two storage nodes and the at least two storage nodes are connected via different network backbones in a single data center.

15. The method of claim 10 wherein the selected storage nodes comprise at least two storage nodes and the at least two storage nodes are located in different data centers.

16. The method of claim 10 wherein the selected storage nodes comprise at least two storage nodes and the at least two storage nodes are located in different cities.

17. The method of claim 10 wherein the selected storage nodes comprise at least two storage nodes and the political context of the at least two storage nodes indicates that the at least two storage nodes are located in different political jurisdictions.

18. The method of claim 10 wherein the selection is based upon the economic context that includes socio-economic attributes of the physical location of the data storage node.

19. The method of claim 10 wherein the act of storing comprises storing the data according to a distributed parity scheme analogous to parity distribution found in RAID subsystems.

20. The method of claim 19 wherein the parity paradigm comprises an N-dimensional parity mechanisms where “

N”

is greater than three.

21. The method of claim 10 wherein the act of storing comprises storing the data in a manner such that the data stored in any one storage node cannot be used in any meaningful fashion without the availability of some or all of the data stored in other storage nodes.

22. A data storage service comprising:

receiving data storage access requests from a file system, the storage access requests including performance criteria that defines storage requirements for data associated with the storage access requests;

maintaining a state information data structure including state information describing the contexts of a number of network-accessible storage devices, wherein first contexts include a political context, an economic context, and a geographic context that are related to a physical location of the network-accessible storage devices and second contexts include a network topological context that is related to attributes of the network-accessible storage devices and wherein the first and second contexts are used to discriminate between storage characteristics of the number of network-accessible storage devices; and

using the first and second contexts from the state information to allocate capacity within the network-accessible storage devices to handle the received data storage request by using network-accessible storage devices having contexts that satisfy the performance criteria.

23. The data storage service of claim 22 wherein the act of maintaining the state information data structure comprises:

detecting a change in state information associated with at least one of the network-accessible storage devices; and

updating the state information associated with the at least one network-accessible storage device to include the change in state information.

24. The data storage service of claim 22 further comprising:

dynamically re-allocating capacity within the network-accessible storage devices in response to detecting a change in their associated contexts.

25. The data storage service of claim 24 wherein the dynamic re-allocating is done in the absence of an externally generated data storage access request.