Non-temporarily storing temporarily stored data in a dispersed storage network

• US 9,110,833 B2
• Filed: 05/08/2013
• Issued: 08/18/2015
• Est. Priority Date: 06/25/2012
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Gladwin, S. Christopher, Markison, Timothy W., Dhuse, Greg, Shirley, Thomas Franklin Jr., Leggette, Wesley, Resch, Jason K., Grube, Gary W.
• Primary Examiner(s): BUTLER, SARAI E
• Application Number:
  US13/889,557
• Publication Number:
  US 20130346795A1
• Time in Patent Office: 832 Days
• Field of Search: 714/6.2, 714/6.1
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 12/1458 by checking the subject access rights
• CPC Class Codes: G06F 21/00 Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/06 Digital input from, or digital output to, record carriers , e.g. RAID, emulated record carriers or networked record carriers
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H03M 13/2903 Methods and arrangements specifically for encoding, e.g. parallel encoding of a plurality of constituent codes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins by a dispersed storage (DS) processing module receiving data from a plurality of sources and queuing the data for storage in a dispersed storage network (DSN). The method continues with the DS processing module utilizing temporary dispersed storage error encoding parameters for efficient and reliable error encoded temporary storage of the data in the DSN. For a data object of the data temporarily stored in the DSN, the method continues with the DS processing module determining non-temporary storage parameters for the data object, retrieving first encoded data slices regarding the data object from the DSN, reconstructing the data object from the first encoded data slices in accordance with the temporary dispersed storage error encoding parameters, encoding the reconstructed data object in accordance with the non-temporary storage parameters for the data object to produce second encoded data slices, and storing the second encoded data slices in the DSN.

Claims

1. A method for execution by a processing module of one or more computing devices, the method comprises:

receiving data from a plurality of sources;

queuing the data for storage in a dispersed storage network (DSN);

utilizing temporary dispersed storage error encoding parameters for efficient and reliable error encoded temporary storage of the data in the DSN;

for a data object of the data temporarily stored in the DSN in accordance with the temporary dispersed storage error encoding parameters;

within a time period corresponding to reliability of the temporary dispersed storage error encoding parameters, determining non-temporary storage parameters for the data object;

retrieving first encoded data slices regarding the data object from the DSN, wherein the data object was error encoded in accordance with the temporary dispersed storage error encoding parameters to produce the first encoded data slices;

reconstructing the data object from the first encoded data slices in accordance with the temporary dispersed storage error encoding parameters;

encoding the reconstructed data object in accordance with the non-temporary storage parameters for the data object to produce second encoded data slices; and

storing the second encoded data slices in the DSN.

2. The method of claim 1 further comprises:

determining a loading of a queue corresponding to the queuing of the data;

determining a desired reliability-duration of the error encoded temporary storage based on the loading; and

determining the temporary dispersed storage error encoding parameters based on the loading and the desired reliability-duration.

3. The method of claim 2 further comprises:

adjusting the temporary dispersed storage error encoding parameters based on variations of the loading of the queue.

4. The method of claim 1, wherein the non-temporary storage parameters comprises at least one of:

a level of compression of the data object prior to the error encoding of the data object;

non-temporary dispersed storage error encoding parameters with an objective of one or more of;

long term storage reliability, ease of access, and storage size within the DSN;

indicating that the first encoded data slices are to be stored as non-temporary data slices; and

disposition of the first encoded data slices.

5. The method of claim 1 further comprises:

ascertaining source information regarding a source of the data object;

ascertaining user information regarding a user of the data object;

ascertaining content information regarding content of the data object; and

determining the non-temporary storage parameters based on at least one of the source information, the user information, and the content information.

6. The method of claim 1 further comprises:

establishing the non-temporary storage parameters to correspond to the temporary dispersed storage error encoding parameters when reliability-duration of the temporary dispersed storage error encoding parameters corresponds to a desired reliability-duration of non-temporary storage of the data object.

7. The method of claim 1 further comprises:

determining the time period corresponding to the reliability of the temporary dispersed storage error encoding parameters based on a calculated risk of loss of the data due to storage device failures involved when the data is stored using the temporary dispersed storage error encoding parameters.

8. The method of claim 1 further comprises:

queuing processing of data objects temporarily stored in the DSN in accordance with the temporary dispersed storage error encoding parameters; and

adjusting processing priority of one or more of the data objects based on elapsed time since temporary storage and the time period.

9. The method of claim 1 further comprises:

retrieving the data object from a queue;

encoding the retrieved data object of the data in accordance with the temporary dispersed storage error encoding parameters to produce the first encoded data slices;

storing the first encoded data slices in temporary memory of the DSN; and

storing the second encoded data slices in non-temporary memory of the DSN.

10. The method of claim 9 further comprises:

deleting the first encoded data slices from the temporary memory when the second encoded data slices are stored in the non-temporary memory.

11. A dispersed storage (DS) module comprises:

a first module, when operable within a computing device, causes the computing device to;

receive data from a plurality of sources; and

queue the data for storage in a dispersed storage network (DSN);

a second module, when operable within the computing device, causes the computing device to;

utilizing temporary dispersed storage error encoding parameters for efficient and reliable error encoded temporary storage of the data in the DSN; and

a third module, when operable within the computing device, causes the computing device to;

for a data object of the data temporarily stored in the DSN in accordance with the temporary dispersed storage error encoding parameters;

within a time period corresponding to reliability of the temporary dispersed storage error encoding parameters, determine non-temporary storage parameters for the data object;

retrieve first encoded data slices regarding the data object from the DSN, wherein the data object was error encoded in accordance with the temporary dispersed storage error encoding parameters to produce the first encoded data slices;

reconstruct the data object from the first encoded data slices in accordance with the temporary dispersed storage error encoding parameters;

encode the reconstructed data object in accordance with the non-temporary storage parameters for the data object to produce second encoded data slices; and

store the second encoded data slices in the DSN.

12. The DS module of claim 11 further comprises:

the second module further functions to;

determine a loading of the queue of the data;

determine a desired reliability-duration of the error encoded temporary storage based on the loading; and

determine the temporary dispersed storage error encoding parameters based on the loading and the desired reliability-duration.

13. The DS module of claim 12 further comprises:

the second module further functions to adjust the temporary dispersed storage error encoding parameters based on variations of the loading of the queue.

14. The DS module of claim 11, wherein the non-temporary storage parameters comprises at least one of:

a level of compression of the data object prior to the error encoding of the data object;

non-temporary dispersed storage error encoding parameters with an objective of one or more of;

long term storage reliability, ease of access, and storage size within the DSN;

indicating that the first encoded data slices are to be stored as non-temporary data slices; and

disposition of the first encoded data slices.

15. The DS module of claim 11 further comprises:

the third module further functions to;

ascertaining source information regarding a source of the data object;

ascertaining user information regarding a user of the data object;

ascertaining content information regarding content of the data object; and

determining the non-temporary storage parameters based on at least one of the source information, the user information, and the content information.

16. The DS module of claim 11 further comprises:

the third module further functions to establish the non-temporary storage parameters to correspond to the temporary dispersed storage error encoding parameters when reliability-duration of the temporary dispersed storage error encoding parameters corresponds to a desired reliability-duration of non-temporary storage of the data object.

17. The DS module of claim 11 further comprises:

the third module further functions to determine the time period corresponding to the reliability of the temporary dispersed storage error encoding parameters based on a calculated risk of the loss of data due to storage device failures involved when the data is stored using the temporary dispersed storage error encoding parameters.

18. The DS module of claim 11 further comprises:

the third module further functions to;

queue processing of data objects temporarily stored in the DSN in accordance with the temporary dispersed storage error encoding parameters; and

adjust processing priority of one or more of the data objects based on elapsed time since temporary storage and the time period.

19. The DS module of claim 11 further comprises:

the second module further functions to;

retrieve the data object from a queue;

encode the retrieved data object of the data in accordance with the temporary dispersed storage error encoding parameters to produce the first encoded data slices; and

store the first encoded data slices in temporary memory of the DSN; and

the third module further functions to;

store the second encoded data slices in non-temporary memory of the DSN.

20. The DS module of claim 19 further comprises:

the third module further functions to delete the first encoded data slices from the temporary memory when the second encoded data slices are stored in the non-temporary memory.