Systems and apparatuses for aggregating nodes to form an aggregated virtual storage for a virtualized desktop environment

• US 9,277,010 B2
• Filed: 12/21/2012
• Issued: 03/01/2016
• Est. Priority Date: 12/21/2012
• Status: Active Grant

• Current Assignee: Atlantis Computing, Inc. (hiveIO, Inc.)
• Original Assignee: Atlantis Computing, Inc. (hiveIO, Inc.)
• Inventors: Iyer, Kartikeya, Venkatesh, Chetan C
• Primary Examiner(s): Ibrahim, Mohamed
• Application Number:
  US13/725,939
• Publication Number:
  US 20140181236A1
• Time in Patent Office: 1,166 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0614 Improving the reliability of storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Embodiments of the invention relate generally to software, data storage, and virtualized computing and processing resources. More specifically, systems and apparatuses are described for aggregating nodes to form an aggregated virtual storage for a virtualized desktop environment. In one embodiment, a virtual storage system includes servers including processors and memories, and an aggregated virtual storage including the memories, each of the memories being associated with a corresponding server. Also included is a storage aggregator processor coupled to a memory including executable instructions to generate a data structure for storage in each memory in an associated server in the servers, each of the data structures being configured to store a reference to duplicative data stored in a first number of servers in the servers. The duplicative data provides redundancy when a second number of servers, or fewer, in the servers are inaccessible.

Claims

1. A virtual storage system comprising:

a subset of servers including a plurality of processors and a plurality of memories;

an aggregated virtual storage comprising the plurality of memories, each of the plurality of memories being associated with a corresponding server in the subset of servers; and

a storage aggregator processor coupled to a memory including executable instructions to disperse duplicative data comprising copies of original data by distributing the copies of original data and the original data among the subset of servers or a portion thereof, and generate a data structure for storage in each memory in an associated server in the subset of servers, each of the data structures being configured to store a reference to duplicative data stored in a first number of servers in the subset of servers, the memory further including executable instructions to translate between disk memory access requests and access requests with the aggregated virtual storage,

wherein the duplicative data provides redundancy when a second number of servers, or fewer, in the subset of servers are inaccessible.

2. The virtual storage system of claim 1, wherein the memory further comprises executable instructions to:

generate a data representation as the reference to the duplicative data associated with a sector,

identify a node identifier for each server of the first number of servers, and

generate metadata including the reference to the duplicative data and the node identifier.

3. The virtual storage system of claim 2, wherein the memory including the executable instructions to generate the data representation as the reference further comprise executable instructions to:

generate a hash value as the data representation.

4. The virtual storage system of claim 2, the memory including the executable instructions to disperse the duplicative data further comprise executable instructions to:

stripe the data representation over the first number of servers, and

disperse parity data over the subset of servers or a portion thereof.

5. The virtual storage system of claim 2, wherein the memory further comprises executable instructions to:

read metadata from the subset of servers in parallel;

determine that the second number of servers or fewer are inaccessible;

identify data representations having node identifiers associated with the second number of servers;

identify other node identifiers for the data representations; and

read the duplicative data for the data representations at the other node identifiers.

6. The virtual storage system of claim 1, wherein the subset of servers comprises:

a server rack;

a plurality of housings in the server rack, each of the housings configured to support one of the servers; and

a communications bus coupling each of the servers in the subset of servers to each other and to the storage aggregator processor.

7. The virtual storage system of claim 1, wherein each of the plurality of memories comprise:

one or more disks or solid state memory, or both.

8. The virtual storage system of claim 1, wherein the memory further comprises:

executable instructions to populate the data structures with identical metadata.

9. A node storage aggregator comprising:

interfaces configured to exchange data with nodes;

a hash generator configured to generate a hash value for a portion of data;

a metadata generator configured to generate data structures to store metadata in each of the nodes, the metadata including the hash value for the portion of data and node identifiers specifying a subset of nodes at which the portion of data is to be stored;

a duplication module configured to generate duplicate portions of the data, and to store each of the duplicate portions of the data at each node in the subset of nodes; and

a redundant node access module configured to read the metadata in parallel from each of the nodes, to determine non-responsive nodes in the subset of nodes, and to extract at least one duplicate portion of the data from responsive nodes in the subset of nodes.

10. The node storage aggregator of claim 9, further comprising:

disperse controller configured to determine the subset of nodes, and to provide the node identifiers to the metadata generator; and

a data disperser configured to disperse the duplicate portions of the data over the subset of nodes,

wherein the disperse controller determines a quantity for the responsive nodes to ensure data redundancy for a maximum quantity for the non-responsive nodes.

11. The node storage aggregator of claim 10, wherein the disperse controller further comprises:

a striping module configured to stripe the duplicate portions of the data over the subset of nodes.

12. The node storage aggregator of claim 11, wherein the striping module is further configured to:

store parity data in each node of the subset of nodes.

13. The node storage aggregator of claim 9, wherein the subset of nodes constitute an aggregate virtual storage.

14. The node storage aggregator of claim 13, further comprising:

a write controller configured to control write operations to the aggregate virtual storage, and to provide write information including a sector number; and

a node write-read repository coupled to a disperse controller to receive the node identifiers, and configured to store associations among data representing the sector number, the hash value, and the node identifiers.

15. The node storage aggregator of claim 13, wherein the redundant node access module is further configured to analyze the metadata to detect a first node identifier associated with one of the non-responsive nodes relevant to the hash value, and to detect a second node identifier associated with one of the responsive nodes relevant to the hash value,

wherein a redundancy read request is generated to retrieve one of the duplicate portions of the data.

16. A system including one or more processors and one or more memories, the system comprising:

an aggregated virtual storage comprising subsets of memories in subsets of servers;

a hypervisor comprising one or more computing devices including processors and memory including executable instructions configured to provide a virtualization layer for a plurality of virtual desktop machines;

a deduplication application including executable instructions to deduplicate data received from a virtual desktop machine as the data is written to the aggregate virtual storage; and

a node storage aggregator configured to disperse duplicative data comprising copies of original data by distributing the copies of original data and the original data among the subset of servers or a portion thereof, and generate a data structure for storage in each memory in a subset of memories in a subset of servers, each of the data structures being further configured to store a reference to duplicative data stored in a first number of servers in the subset of servers,

wherein the duplicative data provides redundancy when a second number of servers in the subset of servers are inaccessible.

17. The system of claim 16, further comprising:

a metadata generator configured to generate the data structures to include metadata in each memory in the subset of memories, the metadata including a hash value for duplicative data and node identifiers specifying the first number of servers at which the duplicative data are to be stored.

18. A method, comprising:

exchanging data with nodes through interfaces of a node storage aggregator;

generating, with a processor, a hash value for a portion of data;

generating data structures to store metadata in each of the nodes, the metadata including the hash value for the portion of data and node identifiers specifying a subset of nodes at which the portion of data is to be stored;

generating duplicate portions of the data;

storing each of the duplicate portions of the data at each node in the subset of nodes; and

reading the metadata in parallel from each of the nodes to determine non-responsive nodes in the subset of nodes, and to extract at least one duplicate portion of the data from responsive nodes in the subset of nodes.

19. The method of claim 18, further comprising:

determining a minimum quantity of redundant nodes for the subset of nodes, to ensure data redundancy, as a total number of nodes less a maximum tolerated quantity of non-responsive nodes; and

dispersing a number of duplicate portions of the data over the subset of nodes, the number corresponding to the minimum quantity.