Multi-layered data redundancy coding techniques
First Claim
Patent Images
1. A computer-implemented method, comprising:
- processing a plurality of requests to store archives on a data storage system having a set of volumes by at least;
applying a redundancy code to the archives to generate a plurality of shards, a subset of which include original data of the archives;
layering the plurality of shards into a plurality of groups such that;
a first group of the plurality of groups includes the subset of shards having the original data and at least one of the shards outside the subset of shards, anda second group of the plurality of groups includes the first group and at least a subset of a remainder of the plurality of shards;
storing the first group of shards on a first subset of the set of volumes such that any member shard of the first group is regenerable, via the redundancy code, using a first quorum quantity of other shards in the first group, andstoring the second group of shards on a second subset of the set of volumes such that any member shard of the second group is regenerable, via the redundancy code, using a second quorum quantity of other shards in the second group; and
as a result of detecting that a particular shard stored on the first subset of the set of volumes is unavailable, regenerating the particular shard using the second subset of the set of volumes.
1 Assignment
0 Petitions
Accused Products
Abstract
Techniques and methods for generating and implementing multiple layers of redundancy coded data are disclosed. For example, a redundancy coding scheme may include data elements that include data that is unencoded relative to the input, yet may still fully participate in providing redundancy to any data element in a given set. In a layered scheme, the input may include a bundle or group of encoded (or unencoded) data elements, thereby nesting two or more layers of redundancy coding. The specific amount of redundancy generated by such a scheme may be adjusted and adapted to failure characteristics of the entity on which the data elements are stored.
195 Citations
20 Claims
-
1. A computer-implemented method, comprising:
-
processing a plurality of requests to store archives on a data storage system having a set of volumes by at least; applying a redundancy code to the archives to generate a plurality of shards, a subset of which include original data of the archives; layering the plurality of shards into a plurality of groups such that; a first group of the plurality of groups includes the subset of shards having the original data and at least one of the shards outside the subset of shards, and a second group of the plurality of groups includes the first group and at least a subset of a remainder of the plurality of shards; storing the first group of shards on a first subset of the set of volumes such that any member shard of the first group is regenerable, via the redundancy code, using a first quorum quantity of other shards in the first group, and storing the second group of shards on a second subset of the set of volumes such that any member shard of the second group is regenerable, via the redundancy code, using a second quorum quantity of other shards in the second group; and as a result of detecting that a particular shard stored on the first subset of the set of volumes is unavailable, regenerating the particular shard using the second subset of the set of volumes. - View Dependent Claims (2, 3, 4)
-
-
5. A system, comprising:
at least one computing device that implements one or more services, wherein the one or more services; apply at least one redundancy code to archives associated with storage requests to the one or more services, to generate a plurality of shards, a subset of which include original data of the archives; group the plurality of shards into a first group and a second group, such that; the first group includes the subset of shards having the original data and a first subset of a remainder of the plurality of shards, and the second group includes a portion of the first group and a subset of a remainder of the plurality of shards not in the first group; store the first group on a first set of volumes associated with the one or more services, such that any member of the first group is regenerable using a subset of a remainder of the first group, store the second group on a second set of volumes associated with the one or more services, such that any member of the second group is regenerable using a subset of a remainder of the second group; and as a result of detecting that a particular shard stored on the first set of volumes is unavailable, regenerate the particular shard using the second set of volumes. - View Dependent Claims (6, 7, 8, 9, 10, 11, 12)
-
13. A non-transitory computer-readable storage medium having stored thereon executable instructions that, as a result of execution by one or more processors of a computer system, cause the computer system to:
-
as a result of receiving requests to store archives, generate a plurality of shards associated with the archives by applying at least one redundancy code to the archives, a subset of the plurality of shards including original data of the archives; layer the plurality of shards into at least a first group and a second group, the first group including at least the subset of shards having the original data and a first subset of a remainder of the plurality of shards, the second group including the first group and at least a subset of a remainder of the plurality of shards not in the first group; store the first group on a first set of volumes associated with the computer system, such that any shard of the first group is regenerable using only other shards in the first group; store the second group on a second set of volumes associated with the computer system, such that any shard of the second group is regenerable using other shards in the second group; and as a result of detecting that a particular shard stored on the first subset of the set of volumes is unavailable, regenerate the particular shard using the second subset of the set of volumes. - View Dependent Claims (14, 15, 16, 17, 18, 19, 20)
-
Specification