Method for dispersing and collating I/O's from virtual machines for parallelization of I/O access and redundancy of storing virtual machine data

• US 9,069,472 B2
• Filed: 12/21/2012
• Issued: 06/30/2015
• Est. Priority Date: 12/21/2012
• Status: Active Grant

• Current Assignee: Atlantis Computing, Inc. (hiveIO, Inc.)
• Original Assignee: Atlantis Computing, Inc. (hiveIO, Inc.)
• Inventors: Venkatesh, Chetan, Iyer, Kartikeya
• Primary Examiner(s): PORTKA, GARY J
• Application Number:
  US13/725,942
• Publication Number:
  US 20140181366A1
• Time in Patent Office: 921 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 12/08 in hierarchically structured memory systems, e.g. virtual memory systems
• CPC Class Codes: G06F 2009/45579 I/O management, e.g. providing access to device drivers or storage
• CPC Class Codes: G06F 2009/45583 Memory management, e.g. access or allocation
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0614 Improving the reliability of storage systems
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: G06F 9/455 Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects

Abstract

Methods and systems to disperse and collate I/O from virtual machines (VMs) among a plurality of near line controllers for parallelization of I/O'"'"'s (parallel reads and parallel writes) and for providing redundancy for stored VM data is disclosed.

Claims

1. A method, comprising:

configuring a non-transitory data storage medium to store a virtual aggregate including N nodes, where N is an integer that is ≧

3, each node having a node block size;

receiving data to be stored among the N nodes;

parsing the data into M data blocks where M is an integer that is ≧

1;

performing a hashing function on each data block to generate metadata (MD) comprising a hash table that is identical for copies of the MD;

selecting for each data block a selected node of the N nodes, wherein the selecting comprises;

mirroring the MD in a metadata field of the selected node,

mirroring a data payload of the data block in a mirrored data field of the selected node, and

striping the data payload of the data block in a striped data field of one or more other nodes of the N nodes that are not the selected node; and

repeating the selecting until the MD and data payloads of all the M data blocks have been mirrored and striped into at least a portion of the N nodes.

2. The method of claim 1, wherein the non-transitory data storage medium comprises a storage medium selected from the group consisting of random access memory (RAM), volatile RAM, non-volatile RAM, at least one hard disk drive (HDD), at least one solid-state drive (SSD), and at least one RAM drive.

3. The method of claim 2, wherein the volatile RAM comprises dynamic random access memory (DRAM).

4. The method of claim 2, wherein the non-volatile RAM comprises Flash memory.

5. The method of claim 1, wherein M is ≦

N.

6. The method of claim 1, wherein M is ≧

N and M includes a plurality of duplicate data blocks and only one of each duplicate data block is parsed into the M data blocks.

7. The method of claim 1, wherein the virtual aggregate is configured to replace, to emulate, or to replace and emulate locally attached storage for a plurality of servers.

8. The method of claim 7, wherein the plurality of servers are configured in a server rack.

9. The method of claim 7, wherein the plurality of servers are configured in a plurality of server racks.

10. The method of claim 7, wherein the locally attached storage that is replaced, emulated, or replaced and emulated by the virtual aggregate comprises at least one hard disk drive (HDD).

11. The method of claim 1, wherein the virtual aggregate is configured to replace, to emulate, or to replace and emulate a storage area network (SAN), network attached storage (NAS), or both.

12. The method of claim 1, wherein the MD includes parity data.

13. The method of claim 1, wherein the MD includes a plurality of pointers to mirrored data fields, striped data fields, or both in other nodes in the virtual aggregate.

14. The method of claim 1, wherein data payloads in at least a portion of the data received comprises virtual machine (VM) data.

15. The method of claim 14, wherein the VM data comprises one or more data types selected from the group consisting of an operating system (OS) image, and OS runtime, and application image, and an application runtime.

16. The method of claim 1, wherein the virtual aggregate is in data communication with a plurality of servers and each server including a least one central processing unit (CPU), at least one random access memory (RAM), and the virtual aggregate is operative to replace, to emulate, or to replace and emulate one or more locally attached hard disk drives (HDD) in each server.

17. The method of claim 1, wherein the node block size is 4 kB.

18. The method of claim 1, wherein each data block has a size that does not exceed the node block size.

19. The method of claim 1, wherein a selected one or more of mirroring the MD, mirroring the data payload, or stripping the data payload occurs in parallel or substantially in parallel across a plurality of the N nodes.

20. The method of claim 1, wherein the MD is included in a metadata hash table generated by the hashing function and the metadata hash table and its associated MD are stored in the metadata field of the selected node.

21. The method of claim 1, wherein the MD identifies a set in a slot.

22. The method of claim 21, wherein the slot identifies a sector.

23. The method of claim 22, wherein the sector identifies a node in the virtual aggregate.

24. The method of claim 1, wherein writing data to one or more of the N nodes of the virtual aggregate occurs in parallel.

25. The method of claim 1, wherein reading data from one or more of the N nodes of the virtual aggregate occurs in parallel.

26. The method of claim 1, wherein all the N nodes include identical copies of the MD.

27. A non-transitory computer readable medium including instructions that, when executed by a processor, cause the processor to perform a method, comprising:

configuring a non-transitory data storage medium to store a virtual aggregate including N nodes, where N is an integer that is >

3, each node having a node block size;

receiving data to be stored among the N nodes;

parsing the data into M data blocks where M is an integer that is ≧

1;

performing a hashing function on each data block to generate metadata (MD) comprising a hash table that is identical for copies of the MD;

selecting for each data block a selected node of the N nodes, wherein the selecting comprises;

mirroring the MD in a metadata field of the selected node,

mirroring a data payload of the data block in a mirrored data field of the selected node, and

striping the data payload of the data block in a striped data field of one or more other nodes of the N nodes that are not the selected node; and

repeating the selecting until the MD and data payloads of all the M data blocks have been mirrored and striped into at least a portion of the N nodes.

28. A system comprising:

a memory; and

a processor coupled with the memory to;

configure a non-transitory data storage medium to store a virtual aggregate including N nodes, where N is an integer that is >

3, each node having a node block size;

receive data to be stored among the N nodes;

parse the data into M data blocks where M is an integer that is ≧

1;

perform a hashing function on each data block to generate metadata (MD) comprising a hash table that is identical for copies of the MD;

select for each data block a selected node of the N nodes, wherein the selection comprises;

mirror the MD in a metadata field of the selected node,

mirror a data payload of the data block in a mirrored data field of the selected node, and

stripe the data payload of the data block in a striped data field of one or more other nodes of the N nodes that are not the selected node; and

repeat the selection until the MD and data payloads of all the M data blocks have been mirrored and striped into at least a portion of the N nodes.