Techniques and systems for data segregation in data storage systems

• US 9,940,474 B1
• Filed: 09/29/2015
• Issued: 04/10/2018
• Est. Priority Date: 09/29/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Franklin, Paul David, Theimer, Marvin Michael
• Primary Examiner(s): Shaw, Brian F
• Application Number:
  US14/869,887
• Time in Patent Office: 924 Days
• CPC Class Codes: G06F 21/6227 where protection concerns the structure of data, e.g. records, types, queries

Abstract

A computer system, such as a data storage system, implements techniques for segregating and controlling access to data stored in multiple regions. In some embodiments, redundancy coded shards generated from the data and stored in durable storage of a data storage system is allocated across multiple regions, but in a fashion that prevents actors with access to regions outside that of a “home” region from recovering a sufficient number of unique shards to regenerate the data represented thereby. In some embodiments, encryption is used to segregate the data by encrypting the generated shards, then storing the cryptographic information on or otherwise controlling access on hosts or other devices of only the home region.

Claims

1. A computer-implemented method, comprising:

under the control of one or more computer systems configured with executable instructions,

in response to receiving a data storage request whose fulfillment involves data being stored in a plurality of volumes of durable storage of a data storage system, the data storage request having a home region of a plurality of regions, storing the data by at least;

using a redundancy code, redundancy coding the data into a plurality of shards having a first quantity of members, a subset of the plurality of shards having a second quantity of members that corresponds to a minimum quantity of shards capable of recreating the data via the redundancy code;

selecting a partitioning of the plurality of shards that includes a first partition of shards and a second partition of shards, the second partition of shards insufficient for reconstructing the data; and

storing the plurality of shards such that;

the first partition is stored among a first set of volumes of the plurality of volumes, the first set of volumes being located in the home region; and

the second partition is stored among a second set of volumes that are located outside of the home region; and

tracking shard storage in the second set of volumes so as to prevent a total number of shards of the plurality of shards from being stored among the second set of volumes such that a total number of shards of the plurality of shards that is stored outside of the home region is insufficient for recreation of the data.

2. The computer-implemented method of claim 1, wherein the data storage request has the home region as a result of being received by an entity within the home region.

3. The computer-implemented method of claim 1, wherein the redundancy code includes an erasure code that is implemented to generate the shard set as a bundle of bundle-encoded shards, and the subset of the selected shard set includes an original form of the data.

4. The computer-implemented method of claim 1, wherein at least one of the shards in the remainder of the selected shard set is encrypted, and cryptographic information sufficient to decrypt the encrypted shard is controlled by an entity within the home region.

5. A system, comprising:

at least one computing device configured to implement one or more services, wherein the one or more services are configured to;

in response to receiving data storage requests, process the data storage requests by at least;

generating, using a redundancy code, a plurality of shards from data associated with the data storage requests;

allocating a first subset of the plurality of shards to a first region; and

allocating a second subset of the plurality of shards to a second region, the second subset being insufficient for regeneration of the data without at least one member of the first subset; and

monitor the second subset of the plurality of shards allocated to the second region so as to ensure that the second subset remains insufficient for generation of the data if one or more additional shards of the plurality of shards are allocated to the second region.

6. The system of claim 5, wherein the first region corresponds to a home region associated with the data.

7. The system of claim 5, wherein the plurality of shards lacks identity shards.

8. The system of claim 5, wherein the first subset of the plurality of shards includes identity shards.

9. The system of claim 5, wherein the plurality of shards includes grid encoded shards.

10. The system of claim 5, wherein the second region includes a plurality of regions outside of a home region associated with the data.

11. The system of claim 5, wherein the services are further configured to monitor the second subset by tracking a history of all shards allocated to the second region.

12. The system of claim 11, wherein the services are further configured to use the tracked history to prevent additional shards from being allocated to the second region if the tracked history indicates that a quantity of unique shards allocated to the second region is equal to or greater than a predetermined quantity of unique shards of the plurality of shards sufficient to regenerate the data.

13. A non-transitory computer-readable storage medium having stored thereon executable instructions that, when executed by one or more processors of a computer system, cause the computer system to at least:

in response to receiving data storage requests, process the data storage requests by at least;

generating, using a redundancy code, a plurality of shards from data associated with the data storage requests;

encrypting at least some of the plurality of shards;

storing cryptographic information sufficient for decrypting the encrypted portion of the plurality of shards in a first region of a plurality of regions;

preventing access to the cryptographic information to a remainder of the plurality of regions;

allocating a first subset of the plurality of shards to the first region; and

allocating a second subset of the plurality of shards to the remainder of the plurality of regions, the second subset being insufficient for regeneration of the data without access to the cryptographic information, the second subset including at least the encrypted portion of the plurality of shards.

14. The non-transitory computer-readable storage medium of claim 13, wherein the plurality of shards include identity shards that include the data in original form.

15. The non-transitory computer-readable storage medium of claim 13, wherein the instructions further comprise instructions that, when executed by the one or more processors, cause the computer system to process the data storage requests by encrypting the plurality of shards by using at least a portion of the cryptographic information, and wherein the computer system interacts with a key management system within the first region so as to obtain the cryptographic information from the key management system.

16. The non-transitory computer-readable storage medium of claim 13, wherein the instructions further comprise instructions that, when executed by the one or more processors, cause the computer system to process the data storage requests by preventing access to the cryptographic information by at least disallowing entities outside of the first region from accessing the cryptographic information.

17. The non-transitory computer-readable storage medium of claim 13, wherein the redundancy code is an erasure code.

18. The non-transitory computer-readable storage medium of claim 13, wherein a subset of the plurality of shards having fewer shards than that of the plurality of shards, when processed using the redundancy code, is capable of recreating the data.

19. The non-transitory computer-readable storage medium of claim 13, wherein the first subset of the encrypted plurality of shards includes encrypted derived shards.

20. The non-transitory computer-readable storage medium of claim 13, wherein the second subset of the encrypted plurality of shards includes encrypted derived shards.