Incremental updates of grid encoded data storage systems
First Claim
Patent Images
1. A computer-implemented method, comprising:
- generating a grid of shards, the grid of shards forming a partitioned data set, the grid of shards being indexed by row and column, the grid of shards comprising a set of data shards, a set of derived shards, and a set of shards containing a predetermined data value, the set of derived shards comprising a set of horizontally-derived shards and a set of vertically-derived shards, wherein;
the partitioned data set includes one or more shards containing one or more portions of the data set and one or more shards generated by applying an erasure code to the one or more shards containing the one or more portions of the data set; and
each shard of the grid of shards has a corresponding row and corresponding column and is configured such that;
the shard is reproducible from other shards associated with the row and reproducible from other shards associated with the column;
if the shard is a horizontally-derived shard of the set of horizontally-derived shards, the shard is derived based at least in part on a set of data shards associated with the row; and
if the shard is a vertically-derived shard of the set of vertically-derived shards, the shard is derived based at least in part on a set of shards associated with the column; and
storing a data object using the grid of shards by at least;
storing a copy of the data object in a storage device corresponding to a first shard of the grid of shards to produce an updated first shard, the updated first shard having a first corresponding row and a first corresponding column;
updating a second shard of the grid of shards based at least in part on a first set of shards of the grid of shards, the first set of shards including the updated first shard and one or more shards of the set of shards containing the predetermined data value, each shard of the first set of shards having a same corresponding row as the second shard;
updating a third shard of the grid of shards based at least in part on a second set of shards of the grid of shards, the second set of shards including the updated first shard and one or more shards of the set of shards containing the predetermined data value, each shard of the second set of shards having a same corresponding column as the third shard; and
updating a fourth shard of the grid of shards based at least in part on a third set of shards of the grid of shards, the third set of shards including the updated second shard and one or more shards of the set of shards containing the predetermined data value, each shard of the third set of shards having the same corresponding column as the fourth shard.
1 Assignment
0 Petitions
Accused Products
Abstract
Techniques for incrementally updating grid encoding data storage systems are described herein. A grid of shards with a plurality of virtual shards is created where each virtual shard is a representation of a shard in the grid of shards that is not backed by a data storage device and where each shard of the grid of shards has an index value. Data is then stored in the grid of shards by updating a shard to store the data and by also updating a second shard based on a set of shards with the same index value as the shard updated to store the data.
257 Citations
20 Claims
-
1. A computer-implemented method, comprising:
-
generating a grid of shards, the grid of shards forming a partitioned data set, the grid of shards being indexed by row and column, the grid of shards comprising a set of data shards, a set of derived shards, and a set of shards containing a predetermined data value, the set of derived shards comprising a set of horizontally-derived shards and a set of vertically-derived shards, wherein; the partitioned data set includes one or more shards containing one or more portions of the data set and one or more shards generated by applying an erasure code to the one or more shards containing the one or more portions of the data set; and each shard of the grid of shards has a corresponding row and corresponding column and is configured such that; the shard is reproducible from other shards associated with the row and reproducible from other shards associated with the column; if the shard is a horizontally-derived shard of the set of horizontally-derived shards, the shard is derived based at least in part on a set of data shards associated with the row; and if the shard is a vertically-derived shard of the set of vertically-derived shards, the shard is derived based at least in part on a set of shards associated with the column; and storing a data object using the grid of shards by at least; storing a copy of the data object in a storage device corresponding to a first shard of the grid of shards to produce an updated first shard, the updated first shard having a first corresponding row and a first corresponding column; updating a second shard of the grid of shards based at least in part on a first set of shards of the grid of shards, the first set of shards including the updated first shard and one or more shards of the set of shards containing the predetermined data value, each shard of the first set of shards having a same corresponding row as the second shard; updating a third shard of the grid of shards based at least in part on a second set of shards of the grid of shards, the second set of shards including the updated first shard and one or more shards of the set of shards containing the predetermined data value, each shard of the second set of shards having a same corresponding column as the third shard; and updating a fourth shard of the grid of shards based at least in part on a third set of shards of the grid of shards, the third set of shards including the updated second shard and one or more shards of the set of shards containing the predetermined data value, each shard of the third set of shards having the same corresponding column as the fourth shard. - View Dependent Claims (2, 3, 4, 5)
-
-
6. A system, comprising at least one computing device that implements one or more services, wherein the one or more services:
-
generate a grid of shards, the grid of shards at least indexed by a first index, the grid of shards including a plurality of virtual shards, each virtual shard of the plurality of virtual shards having at least a corresponding first index, each virtual shard of the plurality of virtual shards configured to represent a shard of the grid of shards without storage of shard data in a data storage device; and store a data object in the grid of shards by at least; storing a copy of the data object in a storage device corresponding to a first shard of the grid of shards to produce an updated first shard, the updated first shard having at least a first corresponding first index; and updating a second shard of the grid of shards based at least in part on a first set of shards of the grid of shards, the first set of shards including the updated first shard and one or more first virtual shards, each shard of the first set of shards having a same corresponding first index as the second shard. - View Dependent Claims (7, 8, 9, 10, 11, 12, 13)
-
-
14. A non-transitory computer-readable storage medium having stored thereon executable instructions that, as a result of execution by one or more processors of a computer system, cause the computer system to at least:
-
generate a grid of shards, each shard of the grid of shards indexed by a corresponding set of indices, each set of indices having a predetermined number of indices, the set of indices at least including a first index, the grid of shards including a plurality of virtual shards, each virtual shard of the plurality of virtual shards configured to represent a shard of the grid of shards without storage of shard data on a storage device; and store a data object in the grid of shards by at least; storing a copy of the data object in a storage device corresponding to a first shard of the grid of shards to produce an updated first shard, the updated first shard having at least a first corresponding first index; and updating a second shard of the grid of shards based at least in part on a first set of shards of the grid of shards, the first set of shards including the updated first shard and one or more virtual shards, each shard of the first set of shards having a same corresponding first index as the second shard. - View Dependent Claims (15, 16, 17, 18, 19, 20)
-
Specification