Policy-based storage in a dispersed storage network

• US 10,372,686 B2
• Filed: 07/28/2016
• Issued: 08/06/2019
• Est. Priority Date: 12/29/2009
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Grube, Gary W., Resch, Jason K.
• Primary Examiner(s): Herzog, Madhuri R
• Application Number:
  US15/222,078
• Publication Number:
  US 20160335202A1
• Time in Patent Office: 1,104 Days
• CPC Class Codes: G06F 11/00 Error detection; Error correction; Monitoring error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00
• CPC Class Codes: G06F 11/08 Error detection or correction by redundancy in data representation, e.g. by using checking codes
• CPC Class Codes: G06F 11/1084 Degraded mode, e.g. caused by single or multiple storage removals or disk failures
• CPC Class Codes: G06F 16/1827 Management specifically adapted to NAS management of storage area networks [SAN] G06F3/067
• CPC Class Codes: G06F 21/60 Protecting data
• CPC Class Codes: G06F 21/62 Protecting access to data via a platform, e.g. using keys or access control rules
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 2212/1052 Security improvement
• CPC Class Codes: G06F 2221/2137 Time limited access, e.g. to a computer or data
• CPC Class Codes: G06F 2221/2141 Access rights, e.g. capability lists, access control lists, access tables, access matrices
• CPC Class Codes: G06F 2221/2151 Time stamp
• CPC Class Codes: H04L 2012/6467 Information loss recovery, e.g. error correction, prediction
• CPC Class Codes: H04L 63/10 for controlling access to devices or network resources
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method for execution by a dispersed storage and task (DST) processing unit operates to receive a write threshold number of slices of a data object and an access policy; determine a current timestamp that indicates a current time value; and store the write threshold number of slices, the access policy, and the timestamp in a plurality of storage units of a dispersed storage network (DSN).

Claims

1. A method comprises:

storing, by a computing device, a plurality of slice retrieval information for various instances of an access policy, wherein the various instances of the access policy comprises a set of instances of the access policy that each indicate a subset of a set of pillars of a vault that are available during a corresponding one of a plurality of time periods, wherein a read threshold number of the set of pillars of the vault are not available in any single one of the plurality of time periods;

receiving, by the computing device, a first data access request;

generating, by the computing device, a first timestamp for the first data access request, wherein the first timestamp is generated by associating a first current time with the first data access request;

determining, by the computing device, a first instance of the various instances of the access policy based on the first timestamp corresponding to a first time period of the plurality of time periods;

determining, by the computing device, a first one of the plurality of slice retrieval information based on the first instance;

retrieving, by the computing device, a first set of encoded data slices of a data segment in accordance with the first one of the plurality of slice retrieval information, wherein the first set of encoded data slices includes less than a read threshold number of encoded data slices;

receiving, by the computing device, a second data access request;

generating, by the computing device, a second timestamp for the second data access request, wherein the second timestamp is generated by associating a second current time with the second data access request;

determining, by the computing device, a second instance of the various instances of the access policy based on the second timestamp corresponding to a second time period of the plurality of time periods;

determining, by the computing device, a second one of the plurality of slice retrieval information based on the second instance;

retrieving, by the computing device, a second set of encoded data slices of the data segment in accordance with the second one of the plurality of slice retrieval information, wherein the second set of encoded data slices includes less than the read threshold number of encoded data slices; and

recovering, by the computing device, the data segment by utilizing the first set of encoded data slices and the second set of encoded data slices, wherein a union of the first set of encoded data slices and the second set of encoded data slices includes at least the read threshold number of encoded slices.

2. The method of claim 1, wherein the various instances of the access policy further comprises:

a third instance of the access policy in which one or more vaults are unavailable for a first third time period;

a fourth instance of the access policy in which one or more pillars are unavailable for a fourth time period;

a fifth instance of the access policy in which one or more storage units are unavailable for a fifth time period;

a sixth instance of the access policy includes time-based access privileges of a user device; and

a seventh instance of the access policy that includes one or more of the third, fourth, fifth, and sixth instances.

3. The method of claim 1, wherein the plurality of slice retrieval information further comprises:

third slice retrieval information that includes a list of vaults that are available a third time period;

fourth slice retrieval information that includes a list of pillars that are available a fourth time period;

fifth slice retrieval information that includes a list of storage units that are available a fifth time period;

sixth slice retrieval information that includes a list of time-based access privileges of a user device; and

seventh slice retrieval information that includes one or more of the third, fourth, fifth, and sixth slice retrieval information, wherein the first one of the plurality of slice retrieval information includes one of the third, fourth, fifth, sixth, and seventh slice retrieval information.

4. The method of claim 1, wherein the access policy is predetermined based on a write threshold number of encoded data slices.

5. The method of claim 1, wherein storing one of the plurality of slice retrieval information includes:

retrieving, for a previous data access request, an encoded data slice in accordance with one instance of the various instances of the access policy; and

storing, within local memory of the computing device, the encoded data slice for the first one of the plurality of slice retrieval information, wherein the first one of the plurality of slice retrieval information corresponds to the first instance of the various instances of the access policy;

wherein at least one of the encoded data slices of the first set of encoded data slices is determined in accordance with the first data access request based on the first one of the plurality of slice retrieval information, and wherein the at least one of the encoded data slices is retrieved from the local memory of the computing device; and

wherein the at least one of the encoded data slices retrieved from the local memory of the computing device includes the encoded data slice retrieved for the previous data access request.

6. The method of claim 5, wherein storing another one of the plurality of slice retrieval information comprises:

retrieving, for a second previous data access request, a slice name in accordance with another instance of the various instances of the access policy; and

storing, within local memory of the computing device, the slice name for the another one of the plurality of slice retrieval information, wherein the another one of the plurality of slice retrieval information corresponds to another instance of the access policy.

7. The method of claim 6 further comprises:

determining, in accordance with a third data access request, available storage units based on the first one of the plurality of slice retrieval information; and

retrieving one or more of the encoded data slices from the available storage units.

8. A processing system of a dispersed storage (DS) processing unit comprises:

at least one processor;

a memory that stores operational instructions, that when executed by the at least one processor cause the processing system to;

store a plurality of slice retrieval information for various instances of an access policy, wherein the various instances of the access policy comprises a set of instances of the access policy that each indicate a subset of a set of pillars of a vault that are available during a corresponding one of a plurality of time periods, wherein a read threshold number of the set of pillars of the vault are not available in any single one of the plurality of time periods;

receive a first data access request;

generate a first timestamp for the first data access request, wherein the first timestamp is generated by associating a first current time with the first data access request;

determine a first instance of the various instances of the access policy based on the first timestamp corresponding to a first time period of the plurality of time periods;

determine, a first one of the plurality of slice retrieval information based on the first instance;

retrieve a first set of encoded data slices of a data segment in accordance with the first one of the plurality of slice retrieval information, wherein the first set of encoded data slices includes less than a read threshold number of encoded data slices;

receive a second data access request;

generate a second timestamp for the second data access request, wherein the second timestamp is generated by associating a second current time with the second data access request;

determine a second instance of the various instances of the access policy based on the second timestamp corresponding to a second time period of the plurality of time periods;

determine a second one of the plurality of slice retrieval information based on the second instance;

retrieve a second set of encoded data slices of the data segment in accordance with the second one of the plurality of slice retrieval information, wherein the second set of encoded data slices includes less than the read threshold number of encoded data slices; and

recover the data segment by utilizing the first set of encoded data slices and the second set of encoded data slices, wherein a union of the first set of encoded data slices and the second set of encoded data slices includes at least the read threshold number of encoded slices.

9. The processing system of claim 8, wherein the various instances of the access policy further comprises:

a third instance of the access policy in which one or more vaults are unavailable for a third time period;

a fourth instance of the access policy in which one or more pillars are unavailable for a fourth time period;

a fifth instance of the access policy in which one or more storage units are unavailable for a fifth time period;

a sixth instance of the access policy includes time-based access privileges of a user device; and

a seventh instance of the access policy that includes one or more of the third, fourth, fifth, and sixth instances.

10. The processing system of claim 8, wherein the plurality of slice retrieval information further comprises:

first third slice retrieval information that includes a list of vaults that are available a third time period;

fourth slice retrieval information that includes a list of pillars that are available a fourth time period;

fifth slice retrieval information that includes a list of storage units that are available a fifth time period;

sixth slice retrieval information that includes a list of time-based access privileges of a user device; and

seventh slice retrieval information that includes one or more of the third, fourth, fifth, and sixth slice retrieval information, wherein the first one of the plurality of slice retrieval information includes one of the third, fourth, fifth, sixth, and seventh slice retrieval information.

11. The processing system of claim 8, wherein the access policy is predetermined based on a write threshold number of encoded data slices.

12. The processing system of claim 8, wherein storing one of the plurality of slice retrieval information includes:

retrieving, for a previous data access request, an encoded data slice in accordance with one instance of the various instances of the access policy; and

storing, within local memory of the DS processing unit, the encoded data slice for the first one of the plurality of slice retrieval information, wherein the first one of the plurality of slice retrieval information corresponds to the first instance of the various instances of the access policy;

wherein at least one of the encoded data slices of the first set of encoded data slices is determined in accordance with the first data access request based on the first one of the plurality of slice retrieval information, and wherein the at least one of the encoded data slices is retrieved from the local memory of the DS processing unit; and

wherein the at least one of the encoded data slices retrieved from the local memory of the DS processing unit includes the encoded data slice retrieved for the previous data access request.

13. The processing system of claim 12, wherein the processor is operable to store another one of the plurality of slice retrieval information by:

retrieving, for a second previous data access request, a slice name in accordance with another instance of the access policy; and

storing, within local memory of the DS processing unit, the slice name for the another one of the plurality of slice retrieval information, wherein the another one of the plurality of slice retrieval information corresponds to another instance of the access policy.

14. The processing system of claim 13, wherein the processor is further operable to:

determine, in accordance with a third data access request, available storage units based on the first one of the plurality of slice retrieval information; and

retrieve one or more of the encoded data slices from the available storage units.

15. A non-transitory computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by a processing system of a dispersed storage network (DSN) that includes a processor and a memory, causes the processing system to;

store a plurality of slice retrieval information for various instances of an access policy, wherein the various instances of the access policy comprises a set of instances of the access policy that each indicate a subset of a set of pillars of a vault that are available during a corresponding one of a plurality of time periods, wherein a read threshold number of the set of pillars of the vault are not available in any single one of the plurality of time periods;

receive a first data access request;

generate a first timestamp for the first data access request, wherein the first timestamp is generated by associating a first current time with the first data access request;

determine a first instance of the various instances of the access policy based on the first timestamp corresponding to a first time period of the plurality of time periods;

determine, a first one of the plurality of slice retrieval information based on the first instance;

retrieve a first set of encoded data slices of a data segment in accordance with the first one of the plurality of slice retrieval information, wherein the first set of encoded data slices includes less than a read threshold number of encoded data slices;

receive a second data access request;

generate a second timestamp for the second data access request, wherein the second timestamp is generated by associating a second current time with the second data access request;

determine a second instance of the various instances of the access policy based on the second timestamp corresponding to a second time period of the plurality of time periods;

determine a second one of the plurality of slice retrieval information based on the second instance;

retrieve a second set of encoded data slices of the data segment in accordance with the second one of the plurality of slice retrieval information, wherein the second set of encoded data slices includes less than the read threshold number of encoded data slices; and

recover the data segment by utilizing the first set of encoded data slices and the second set of encoded data slices, wherein a union of the first set of encoded data slices and the second set of encoded data slices includes at least the read threshold number of encoded slices.

16. The non-transitory computer readable storage medium of claim 15, wherein the various instances of the access policy further comprises:

a third instance of the access policy in which one or more vaults are unavailable for a third time period;

a fourth instance of the access policy in which one or more pillars are unavailable for a fourth time period;

a fifth instance of the access policy in which one or more storage units are unavailable for a fifth time period;

a sixth instance of the access policy includes time-based access privileges of a user device; and

a seventh instance of the access policy that includes one or more of the third, fourth, fifth, and sixth instances.

17. The non-transitory computer readable storage medium of claim 15, wherein the plurality of slice retrieval information comprises:

further comprises;

third slice retrieval information that includes a list of vaults that are available a third time period;

fourth slice retrieval information that includes a list of pillars that are available a fourth time period;

fifth slice retrieval information that includes a list of storage units that are available a fifth time period;

sixth slice retrieval information that includes a list of time-based access privileges of a user device; and

seventh slice retrieval information that includes one or more of the third, fourth, fifth, and sixth slice retrieval information, wherein the first one of the plurality of slice retrieval information includes one of the third, fourth, fifth, sixth, and seventh slice retrieval information.

18. The non-transitory computer readable storage medium of claim 15, wherein the access policy is predetermined based on a write threshold number of encoded data slices.

19. The non-transitory computer readable storage medium of claim 15, wherein storing one of the plurality of slice retrieval information includes:

retrieving, for a previous data access request, an encoded data slice in accordance with one instance of the various instances of the access policy; and

storing, within local memory, the encoded data slice for the first one of the plurality of slice retrieval information, wherein the first one of the plurality of slice retrieval information corresponds to the first instance of the various instances of the access policy;

wherein at least one of the encoded data slices of the first set of encoded data slices is determined in accordance with the first data access request based on the first one of the plurality of slice retrieval information, and wherein the at least one of the encoded data slices is retrieved from the local memory; and

wherein the at least one of the encoded data slices retrieved from the local memory includes the encoded data slice retrieved for the previous data access request.

20. The non-transitory computer readable storage medium of claim 19, wherein the processing system is operable to:

store another one of the plurality of slice retrieval information by;

retrieving, for a second previous data access request, a slice name in accordance with another instance of the access policy;

storing, within local memory, the slice name for the another one of the plurality of slice retrieval information, wherein the another one of the plurality of slice retrieval information corresponds to another instance of the access policy; and

when receiving a third data access request;

determine, in accordance with the third data access request, available storage units based on the another one of the plurality of slice retrieval information; and

retrieve one or more of the encoded data slices from the available storage units.