Failure resilient distributed replicated data storage system
First Claim
1. A system comprising:
- a plurality of storage zones, each storage zone comprising a plurality of nodes wherein each node comprises a plurality of storage devices and a controller, the controller including a processor and memorya first node of a plurality of nodes included in a first zone of the plurality of storage zones, the first node having instructions which when executed cause a first processor included in a first controller in the first node to perform actions including;
receiving a storage request from a client of the system to store a data itempartitioning the data item into a plurality of data objects, each data object including a storage policy identifier and a data portioncreating a plurality of parity objects for the data item including creating at least one local parity object for one of the plurality of storage zones based on the received group of parity objects and the received group of data objects within the local storage zonecreating reassembly instructions for the data item including creating local reassembly instructions, the local reassembly instructions including reference to the local parity objectstoring groups of at least three of the data objects and groups of at least two of the parity objects and the reassembly instructions among the nodes included in the storage zones according to a predetermined set of criteriawherein each of the data objects are included in at least one of the groups of data objects, but each data object is included in less than all the groups of data objectswherein some of the data objects are duplicated among the storage zoneswherein each of the parity objects are included in at least one of the groups of parity objects, but each parity object is included in less than all the groups of parity objectswherein some of the parity objects are duplicated among the storage zoneswherein the predetermined set of criteria ensures that the data item can be reconstructed by accessing a subset of the plurality of data objects and the plurality of the parity objects.
2 Assignments
0 Petitions
Accused Products
Abstract
A failure resilient distributed replicated data storage system is described herein. The storage system includes zones that are independent, and autonomous from each other. The zones include nodes that are independent and autonomous. The nodes include storage devices. When a data item is stored, it is partitioned into a plurality of data objects and a plurality of parity objects calculated. Reassembly instructions are created for the data item. The data objects and parity objects are spread across all nodes and zones in the storage system. Reassembly instructions are also spread across the zones. When a read request is received, the data item is prepared from the lowest latency nodes according to the reassembly instructions. This provides for data resiliency while keeping the amount of storage space required relatively low.
19 Citations
21 Claims
-
1. A system comprising:
-
a plurality of storage zones, each storage zone comprising a plurality of nodes wherein each node comprises a plurality of storage devices and a controller, the controller including a processor and memory a first node of a plurality of nodes included in a first zone of the plurality of storage zones, the first node having instructions which when executed cause a first processor included in a first controller in the first node to perform actions including; receiving a storage request from a client of the system to store a data item partitioning the data item into a plurality of data objects, each data object including a storage policy identifier and a data portion creating a plurality of parity objects for the data item including creating at least one local parity object for one of the plurality of storage zones based on the received group of parity objects and the received group of data objects within the local storage zone creating reassembly instructions for the data item including creating local reassembly instructions, the local reassembly instructions including reference to the local parity object storing groups of at least three of the data objects and groups of at least two of the parity objects and the reassembly instructions among the nodes included in the storage zones according to a predetermined set of criteria wherein each of the data objects are included in at least one of the groups of data objects, but each data object is included in less than all the groups of data objects wherein some of the data objects are duplicated among the storage zones wherein each of the parity objects are included in at least one of the groups of parity objects, but each parity object is included in less than all the groups of parity objects wherein some of the parity objects are duplicated among the storage zones wherein the predetermined set of criteria ensures that the data item can be reconstructed by accessing a subset of the plurality of data objects and the plurality of the parity objects. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A system comprising:
-
a plurality of storage zones, each storage zone comprising a plurality of storage nodes wherein each storage node comprises a plurality of storage devices and a controller, the controller including a processor and memory a first node of a plurality of nodes included in a first zone of the plurality of storage zones, the first node having instructions which when executed cause a first processor included in a first controller in the first node to perform actions including; receiving a storage request from a client of the system to store a data item, the storage request including (a) the data item or a data item reference and (b) a storage criteria partitioning the data item into a plurality of data objects, each data object including a storage policy identifier and a data portion according to the storage criteria creating a plurality of parity objects for the data item according to the storage criteria including creating at least one local parity object for the zone based on the received group of parity objects and the received group of data objects within the local storage zone creating reassembly instructions for the data item according to the storage criteria including creating local reassembly instructions, the local reassembly instructions including reference to the local parity object storing the data objects and the parity objects among the nodes among the storage zones according to the storage criteria storing the reassembly instructions among the storage nodes among the storage zones wherein each of the data objects are included in at least one of the plurality of storage zones, but each data object is included in less than all the plurality of storage zones wherein some of the data objects are duplicated among the storage zones wherein each of the parity objects are included in at least one of the plurality of storage zones but each parity object is included in less than all the plurality of storage zones wherein some of the parity objects are duplicated among the storage zones wherein the reassembly instructions specify how the data item can be reconstructed from a subset of the plurality of data objects and the plurality of parity objects. - View Dependent Claims (12, 13, 14, 15, 16, 17, 18, 19, 20, 21)
-
Specification