Implementing Fault Domain And Latency Requirements In A Virtualized Distributed Storage System

• US 20190213085A1
• Filed: 01/11/2018
• Published: 07/11/2019
• Est. Priority Date: 01/11/2018
• Status: Abandoned Application

• Current Assignee: Robin Systems, Inc. (Rakuten Group, Inc.)
• Original Assignee: Robin Systems, Inc. (Rakuten Group, Inc.)
• Inventors: Alluboyina, Ravi Kumar, Vallala, Shravan Kumar, Mageswaran, Manjunath, Doshi, Tushar
• Application Number:
  US15/868,652
• Publication Number:
  US 20190213085A1
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/1438 Restarting or rejuvenating
• CPC Class Codes: G06F 11/1446 Point-in-time backing up or restoration of persistent data
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A new snapshot of a storage volume is created by instructing computing nodes to suppress write requests. Once pending write requests from the computing nodes are completed, storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot and finalizes and performs garbage collection with respect to segments allocated to the previous snapshot. Subsequent write requests to the storage volume are then performed on the segments allocated to the new snapshot. An orchestration layer implements a multi-role application that is provisioned with virtualized storage and computation resources. A snapshot of the application may be created and used to rollback or clone the application. Clones snapshots of storage volumes may be gradually populated with data from prior snapshots to reduce loading on a primary snapshot.

Claims

1. A method comprising, by a distributed storage system:

defining a bundle defining a plurality of applications, provisioning for the plurality of applications, and a fault domain constraint;

provisioning one or more compute nodes according to the provisioning for the plurality of applications;

instantiating instances of the plurality of applications on the one or more compute nodes as defined by the bundle;

provisioning one or more virtual storage volumes according to the provisioning for the plurality of applications, the one or more storage volumes corresponding to one or more storage nodes and implemented according to a disk virtualization scheme;

wherein provisioning the one or more virtual storage volumes further comprises distributing the one or more virtual storage volumes across the one or more storage nodes according to the fault domain constraint.

2. The method of claim 1, wherein the disk virtualization scheme does not expose identities of the one or more storage nodes to the instances of the plurality of applications.

3. The method of claim 1, wherein:

the fault domain constraint specifies that a first storage volume of the one or more virtual storage volumes is a redundant copy of a second volume of the one or more virtual storage volumes; and

wherein provisioning the one or more virtual storage volumes across the one or more storage nodes according to the fault domain constraint comprises constraining the first storage volume and the second storage volume to reside on different storage nodes of the one or more storage nodes.

4. The method of claim 1, wherein the bundle further defines a latency requirement;

wherein instantiating instances of the plurality of applications on the one or more compute nodes as defined by the bundle comprises locating, by the distributed storage system, the instances according to the latency requirement.

5. The method of claim 4, where locating, by the distributed storage system, the instances according to the latency requirement comprises selecting the one or more compute nodes as having a network latency with respect to the one or more storage nodes that meets the latency requirement.

6. The method of claim 4, where locating, by the distributed storage system, the instances according to the latency requirement comprises selecting the one or more compute nodes to be the same as the one or more storage nodes according to the latency requirement.

7. The method of claim 1, wherein the bundle defines a performance requirement, the method further comprising selecting, by the distributed storage system, the one or more storage nodes and one or more compute nodes according to the performance requirement.

8. The method of claim 1, wherein the bundle defines a memory requirement, the method further comprising selecting, by the distributed storage system, the one or more compute nodes according to the memory requirement.

9. The method of claim 1, wherein the bundle further includes cross-namespace definitions between applications of the plurality of applications, the method further comprising, by the distributed storage system:

linking name-spaces of one or more instances of the plurality of applications such that at least one of (a) one or more variables of a first instance of a first application of the plurality of applications is accessible by a second instance of a second application of the plurality of applications and (b) one or more functions of the first instance of the first application of the plurality of applications is accessible by the second instance of the second application of the plurality of applications.

10. The method of claim 1, wherein the bundle further defines configuration parameters for each application of the plurality of application, the method further comprising configuring the instances of the plurality of applications according to the configuration parameters.

11. A method comprising, by a distributed storage system:

a network;

a plurality of storage nodes operably coupled to the network;

a plurality of compute nodes operably coupled to the network;

an orchestration computer coupled to the network and comprising one or more processing devices and one or more one or more memory devices operably coupled to the one or more processing devices, the one or more memory devices storing executable code effective to cause the one or more processing devices to;

define a bundle defining a plurality of applications, provisioning for the plurality of applications, and a fault domain constraint;

provision one or more compute nodes of the plurality of compute nodes according to the provisioning for the plurality of applications;

instantiate instances of the plurality of applications on the one or more compute nodes as defined by the bundle;

provision one or more virtual storage volumes according to the provisioning for the plurality of applications, the one or more storage volumes corresponding to one or more storage nodes of the plurality of storage nodes and implemented according to a disk virtualization scheme, wherein provisioning the one or more virtual storage volumes further includes distributing the one or more virtual storage volumes across the one or more storage nodes according to the fault domain constraint.

12. The system of claim 11, wherein the disk virtualization scheme does not expose identities of the one or more storage nodes to the instances of the plurality of applications.

13. The system of claim 11, wherein:

the fault domain constraint specifies that a first storage volume of the one or more virtual storage volumes is a redundant copy of a second volume of the one or more virtual storage volumes; and

wherein the executable code is further effective to cause the one or more processing devices to distribute provisioning of the one or more virtual storage volumes across the one or more storage nodes according to the fault domain constraint by constraining the first storage volume and the second storage volume to reside on different storage nodes of the one or more storage nodes.

14. The system of claim 11, wherein the bundle further defines a latency requirement;

wherein the executable code is further effective to cause the one or more processing devices to invoke instantiation of instances of the plurality of applications on the one or more compute nodes as defined by the bundle by selecting the one or more compute nodes according to the latency requirement.

15. The system of claim 14, wherein the executable code is further effective to cause the one or more processing devices to select the one or more compute nodes according to the latency requirement by selecting the one or more compute nodes as having a network latency with respect to the one or more storage nodes that meets the latency requirement.

16. The system of claim 14, wherein the executable code is further effective to cause the one or more processing devices to select the one or more compute nodes according to the latency requirement by selecting the one or more compute nodes to be the same as the one or more storage nodes according to the latency requirement.

17. The system of claim 11, wherein the bundle defines a processing requirement, the executable code being further effective to cause the one or more processing devices to select the one or more compute nodes according to the latency requirement by selecting the one or more storage nodes and one or more compute nodes according to both the latency requirement and the processing requirement.

18. The system of claim 11, wherein the bundle defines a memory requirement, the executable code being further effective to cause the one or more processing devices to select the one or more compute nodes according to the memory requirement.

19. The system of claim 11, wherein the bundle further includes cross-namespace definitions between applications of the plurality of applications, the executable code being further effective to cause the one or more processing devices to:

link name-spaces of one or more instances of the plurality of applications such that at least one of (a) one or more variables of a first instance of a first application of the plurality of applications is accessible by a second instance of a second application of the plurality of applications and (b) one or more functions of the first instance of the first application of the plurality of applications is accessible by the second instance of the second application of the plurality of applications.

20. The system of claim 11, wherein the bundle further defines configuration parameters for each application of the plurality of application, the executable code being further effective to cause the one or more processing devices to configure the instances of the plurality of applications according to the configuration parameters.