Distributed storage time synchronization based on storage delay

• US 9,344,500 B2
• Filed: 06/30/2014
• Issued: 05/17/2016
• Est. Priority Date: 12/29/2009
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Resch, Jason K., Grube, Gary W., Markison, Timothy W.
• Primary Examiner(s): HERZOG, MADHURI R
• Application Number:
  US14/319,282
• Publication Number:
  US 20140317226A1
• Time in Patent Office: 687 Days
• Field of Search: 707687-699, 726 1- 4, 726/21, 726 26- 30, 713164-167, 713189-194, 709201-203, 709217-229
• US Class Current: 1/1
• CPC Class Codes: G06F 11/00 Error detection; Error correction; Monitoring error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00
• CPC Class Codes: G06F 11/08 Error detection or correction by redundancy in data representation, e.g. by using checking codes
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 21/60 Protecting data
• CPC Class Codes: G06F 21/62 Protecting access to data via a platform, e.g. using keys or access control rules
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 2221/2137 Time limited access, e.g. to a computer or data
• CPC Class Codes: G06F 2221/2141 Access rights, e.g. capability lists, access control lists, access tables, access matrices
• CPC Class Codes: H04L 2012/6467 Information loss recovery, e.g. error correction, prediction
• CPC Class Codes: H04L 63/10 for controlling access to devices or network resources
• CPC Class Codes: H04L 63/108 when the policy decisions are valid for a limited amount of time
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins with a processing module receiving a data retrieval request and obtaining a real-time indicator corresponding to when the data retrieval request was received. The method continues with the processing module determining a time-based data access policy based on the data retrieval request and the real-time indicator and accessing a plurality of dispersed storage (DS) units in accordance with the time-based data access policy to retrieve encoded data slices. The method continues with the processing module decoding the threshold number of encoded data slices in accordance with an error coding dispersal storage function when a threshold number of the encoded data slices have been retrieved.

Claims

1. A method for synchronizing availability of data slices for retrieval from distributed storage units based on storage transmission delays, comprising:

determining transmission time delays between a processing unit, having a processor, and a plurality of distributed storage units communicating with the processing unit, in which the plurality of distributed storage units are configured to receive and store n number of encoded data slices of a data segment by having a respective distributed storage unit store a respective one of the n encoded data slices of the data segment and in which at least a threshold number k of the n encoded data slices are needed to reconstruct the data segment, where k is less than n;

selecting a set of distributed storage units based on the transmission time delays;

determining a longest transmission time delay from respective transmission time delays for the selected set of distributed storage units;

determining corresponding additional time delay value to add to the respective transmission time delays for the selected set of distributed storage units, in order to synchronize total delay between the processing unit and respective ones of the selected set of distributed storage units to substantially coincide with the longest transmission time delay; and

transmitting n encoded data slices from the processing unit to the plurality of distributed storage units, along with corresponding additional time delay values to the selected set of distributed storage units, in which the corresponding additional time delay values are to be utilized at the selected set of distributed storage units to delay access to received encoded data slices at the selected set of distributed storage units for an additional time period determined by the corresponding additional time delay value.

2. The method of claim 1, wherein when selecting the set of distributed storage units, all of the plurality of distributed storage units are selected as the selected set of distributed storage units.

3. The method of claim 1, wherein when selecting the set of distributed storage units, a subset of the plurality of distributed storage units is selected as the selected set of distributed storage units.

4. The method of claim 1, wherein when selecting the set of distributed storage units, a subset of the plurality of distributed storage units is selected as the selected set of distributed storage units and the subset is selected based on shortest transmission time delays.

5. The method of claim 1, wherein when selecting the set of distributed storage units, k number of distributed storage units are selected as the selected set of distributed storage units.

6. The method of claim 1, wherein when selecting the set of distributed storage units, k number of distributed storage units are selected as the selected set of distributed storage units and the k number of distributed storage units are selected based on shortest transmission time delays.

7. The method of claim 1, wherein when transmitting the n encoded data slices, a timer is to be used at the selected set of distributed storage units to prevent access to the received encoded data slices, in which the timer commences upon receiving the encoded slices and expires based on the corresponding additional time delay value and in which corresponding access for encoded data slice retrieval is granted upon expiration of the timer for respective ones of the selected set of distributed storage units.

8. A method for synchronizing availability of data slices for retrieval from distributed storage units based on storage transmission delays, comprising:

determining transmission time delays between a processing unit, having a processor, and a plurality of distributed storage units communicating with the processing unit, in which the plurality of distributed storage units are configured to receive and store n number of encoded data slices of a data segment by having a respective distributed storage unit store a respective one of the n encoded data slices of the data segment and in which at least a threshold number k of the n encoded data slices are needed to reconstruct the data segment, where k is less than n;

selecting a set of distributed storage units based on the transmission time delays;

determining a longest transmission time delay from respective transmission time delays for the selected set of distributed storage units;

determining corresponding additional time delay value to add to the respective transmission time delays for the selected set of distributed storage units, in order to synchronize total delay between the processing unit and respective ones of the selected set of distributed storage units to substantially coincide with the longest transmission time delay; and

transmitting n encoded data slices from the processing unit to the plurality of distributed storage units, but in which the processor delays transmitting respective encoded data slices to the selected set of distributed storage units by a time period determined by the corresponding additional time delay values to synchronize arrival of the respective encoded data slices to the selected set of distributed storage units.

9. The method of claim 8, wherein when selecting the set of distributed storage units, all of the plurality of distributed storage units are selected as the selected set of distributed storage units.

10. The method of claim 8, wherein when selecting the set of distributed storage units, a subset of the plurality of distributed storage units is selected as the selected set of distributed storage units.

11. The method of claim 8, wherein when selecting the set of distributed storage units, a subset of the plurality of distributed storage units is selected as the selected set of distributed storage units and the subset is selected based on shortest transmission time delays.

12. The method of claim 8, wherein when selecting the set of distributed storage units, k number of distributed storage units are selected as the selected set of distributed storage units.

13. The method of claim 8, wherein when selecting the set of distributed storage units, k number of distributed storage units are selected as the selected set of distributed storage units and the k number of distributed storage units are selected based on shortest transmission time delays.

14. The method of claim 8, further comprising utilizing a timer, in which the processor uses the timer to delay transmitting respective encoded data slices to the selected set of distributed storage units.

15. A method for synchronizing availability of data slices for retrieval from distributed storage units based on storage transmission delays, comprising:

determining a first set of transmission time delays between a first processing unit, having a first processor, and a plurality of distributed storage units communicating with the first processing unit, in which the plurality of distributed storage units are configured to receive and store first n number of encoded data slices of a first data segment from the first processing unit by having a respective distributed storage unit store a respective one of the first n encoded data slices of the first data segment and in which at least a threshold number k of the first n encoded data slices are needed to reconstruct the first data segment, where k is less than n;

communicating between the first processing unit and a second processing unit, which has a second processor, to receive at the first processing unit a second set of transmission time delays from the second processing unit, in which the second processing unit and the plurality of distributed storage units are configured to receive and store second n number of encoded data slices of a second data segment by having respective distributed storage unit store respective ones of the second n encoded data slices of the second data segment and in which at least the threshold number k of the second n encoded data slices are needed to reconstruct the second data segment;

selecting a set of distributed storage units based on the first set and second set of transmission time delays;

determining a longest transmission time delay from respective transmission time delays for the selected set of distributed storage units;

determining corresponding additional time delay value to add to the respective first and second sets of transmission time delays for the selected set of distributed storage units, in order to synchronize total delay between the first and second processing units and respective ones of the selected set of distributed storage units to substantially coincide with the longest transmission time delay for the first and second processing units; and

communicating the corresponding additional time delay value to the second processing unit from the first processing unit in order to time manage transmission of the first n encoded data slices from the first processing unit to the plurality of distributed storage units and transmission of the second n encoded data slice from the second processing unit to the plurality of distributed storage units, but in which the first and second processors delay transmitting respective encoded data slices to the selected set of distributed storage units by a time period determined by the corresponding additional time delay values to synchronize arrival of the respective encoded data slices from the first and second processing units to the selected set of distributed storage units.

16. The method of claim 15, wherein when selecting the set of distributed storage units, all of the plurality of distributed storage units are selected as the selected set of distributed storage units.

17. The method of claim 15, wherein when selecting the set of distributed storage units, a subset of the plurality of distributed storage units is selected as the selected set of distributed storage units.

18. The method of claim 15, wherein when selecting the set of distributed storage units, a subset of the plurality of distributed storage units is selected as the selected set of distributed storage units and the subset is selected based on shortest transmission time delays.

19. The method of claim 15, wherein when selecting the set of distributed storage units, k number of distributed storage units are selected as the selected set of distributed storage units.

20. The method of claim 15, wherein when selecting the set of distributed storage units, k number of distributed storage units are selected as the selected set of distributed storage units and the k number of distributed storage units are selected based on shortest transmission time delays.