Performance optimized storage vaults in a dispersed storage network

• US 10,073,652 B2
• Filed: 07/25/2016
• Issued: 09/11/2018
• Est. Priority Date: 09/24/2015
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Hegde, Harsha, Leggette, Wesley B.
• Primary Examiner(s): Merant, Guerrier
• Application Number:
  US15/218,743
• Publication Number:
  US 20170091030A1
• Time in Patent Office: 778 Days
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0614 Improving the reliability of storage systems
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0661 Format or protocol conversion arrangements
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 8/65 Updates security arrangements therefor G06F21/57
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/154 Error and erasure correction, e.g. by using the error and erasure locator or Forney polynomial
• CPC Class Codes: H03M 13/3761 using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H04L 63/061 for key exchange, e.g. in peer-to-peer networks cryptographic mechanisms or cryptographic arrangements for key agreement H04L9/0838
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/34 involving the movement of software or configuration parameters network booting or remote initial program loading [RIPL] G06F9/4416
• CPC Class Codes: H04L 67/60 Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources admission control or resource allocation H04L47/70

Abstract

Methods for use in a dispersed storage network (DSN) to enable rapid retrieval of data. A first set of storage units of the DSN is configured as a performance optimized internal vault that utilizes dispersed error encoding/decoding parameters which provide improved data access performance in relation to a second set of storage units. Upon receiving a data object for storage, a computing device of the DSN determines, based on at least one performance criterion, to store the data object in the performance optimized internal vault as opposed to the second set of storage units. The data object is then dispersed storage error encoded, in accordance with dispersal parameters associated with storage of data in the internal vault, to produce a plurality of sets of performance encoded data slices, wherein the data object is segmented and each resulting data segment is encoded into a respective set of performance encoded data slices.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the DSN having storage resources including a first set of storage units and a second set of storage units, the first set of storage units configured as an internal storage vault, the method comprises:

receiving a data object for storage;

determining, based on at least one performance criterion, to store the data object in the internal storage vault as opposed to the second set of storage units;

dispersed storage error encoding the data object, in accordance with dispersal parameters associated with storage of data in the internal storage vault, to produce a plurality of sets of performance encoded data slices, wherein the data object is segmented into a plurality of data segments and wherein each data segment is dispersed storage error encoded into a respective set of performance encoded data slices; and

facilitating storage of the plurality of sets of performance encoded data slices in the internal storage vault.

2. The method of claim 1, wherein the dispersal parameters associated with the internal storage vault differ from second dispersal parameters associated with storage of data in the second set of storage units.

3. The method of claim 2, wherein each of the plurality of sets of performance encoded data slices includes n number of performance encoded data slices, and wherein the second set of storage units is configured to store sets of encoded data slices that each include a second number of encoded data slices, the second number being greater than n.

4. The method of claim 1, wherein the at least one performance criterion includes at least one of:

a data type, a data size indicator, a data owner identifier, system registry information, or available capacity level of the internal storage vault.

5. The method of claim 1, wherein the at least one performance criterion includes at least one of:

an estimated access frequency level, a DSN network loading level, an updated desired access performance level, or a desired data storage reliability level.

6. The method of claim 1 further comprises:

receiving a data recovery request for the data object;

identifying the set of storage units of the internal storage vault;

issuing read slice requests, to the identified set of storage units, for the performance encoded data slices of the plurality of sets of performance encoded data slices;

receiving the performance encoded data slices; and

for each respective set of performance encoded data slices, dispersed error decoding at least a decode threshold number of received performance encoded data slices to reproduce a data segment of the plurality of data segments.

7. The method of claim 6, wherein identifying the storage units of the internal storage vault includes at least one of:

interpreting a directory entry, interpreting a dispersed hierarchical index entry, identifying the storage units based on a data type indicator in the data recovery request, or receiving identifiers corresponding to the storage units.

8. The method of claim 1 further comprises:

determining to migrate the data object from the internal storage vault to the second set of storage units;

retrieving the plurality of sets of performance encoded data slices from the internal storage vault;

dispersed storage error decoding the sets of performance encoded data slices to reproduce the data object;

dispersed storage error encoding the data object, in accordance with dispersal parameters associated with the second set of storage units, to produce a plurality of sets of encoded data slices, wherein the data object is segmented into a plurality of data segments and wherein each data segment is dispersed storage error encoded into a respective set of encoded data slices; and

facilitating storage of the plurality of sets of encoded data slices in the second set of storage units.

9. The method of claim 8, wherein determining to migrate is based on an update to the at least one performance criterion.

10. The method of claim 1, further comprises:

obtaining the dispersal parameters associated with storage of data in the internal storage vault by at least one of;

extracting the dispersal parameters from system registry information, receiving the dispersal parameters in conjunction with the data object, interpreting a query response, determining a desired performance level, or matching dispersal encoding parameters to a determined performance level.

11. A computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), causes the one or more computing devices to;

determine, based on at least one performance criterion, to store a data object in a first set of storage units of the DSN as opposed to a second set of storage units of the DSN, the first set of storage units configured as an internal storage vault;

dispersed storage error encode the data object, in accordance with dispersal parameters associated with storage of data in the internal storage vault, to produce a plurality of sets of performance encoded data slices, wherein the data object is segmented into a plurality of data segments and wherein each data segment is dispersed storage error encoded into a respective set of performance encoded data slices; and

facilitate storage of the plurality of sets of performance encoded data slices in the internal storage vault.

12. The computer readable storage medium of claim 11, wherein the dispersal parameters associated with the internal storage vault differ from second dispersal parameters associated with storage of data in the second set of storage units.

13. The computer readable storage medium of claim 12, wherein each of the plurality of sets of performance encoded data slices includes n number of performance encoded data slices, and wherein the second set of storage units is configured to store sets of encoded data slices that each include a second number of encoded data slices, the second number being greater than n.

14. The computer readable storage medium of claim 11, wherein the at least one performance criterion includes at least one of:

a data type, a data size indicator, a data owner identifier, system registry information, or available capacity level of the internal storage vault.

15. The computer readable storage medium of claim 11, wherein the at least one performance criterion includes at least one of:

an estimated access frequency level, a DSN network loading level, or an updated desired access performance level.

16. A computing device of a group of computing devices of a dispersed storage network (DSN), the DSN having storage resources including a first set of storage units and a second set of storage units, the first set of storage units configured as an internal storage vault, the computing device comprises:

an interface;

a local memory; and

a processing module operably coupled to the interface and the local memory, wherein the processing module operates to;

receive, via the interface, a data object for storage in the DSN;

determine, based on at least one performance criterion, to store the data object in the internal storage vault as opposed to the second set of storage units;

dispersed storage error encode the data object, in accordance with dispersal parameters associated with storage of data in the internal storage vault, to produce a plurality of sets of performance encoded data slices, wherein the data object is segmented into a plurality of data segments and wherein each data segment is dispersed storage error encoded into a respective set of performance encoded data slices; and

facilitate storage of the plurality of sets of performance encoded data slices in the internal storage vault.

17. The computing device of claim 16, wherein the dispersal parameters associated with the internal storage vault differ from second dispersal parameters associated with storage of data in the second set of storage units.

18. The computing device of claim 17, wherein each of the plurality of sets of performance encoded data slices includes n number of performance encoded data slices, and wherein the second set of storage units is configured to store sets of encoded data slices that each include a second number of encoded data slices, the second number being greater than n.

19. The computing device of claim 16, wherein the at least one performance criterion includes at least one of:

a data type, a data size indicator, a data owner identifier, system registry information, available capacity level of the internal storage vault, an estimated access frequency level, a DSN network loading level, an updated desired access performance level, or a desired data storage reliability level.

20. The computing device of claim 16, wherein the processing module further functions to:

receive a data recovery request for the data object;

identify the set of storage units of the internal storage vault;

issue read slice requests to the identified set of storage units, via the interface, for the performance encoded data slices of the plurality of sets of performance encoded data slices;

receive, via the interface, the performance encoded data slices; and

for each respective set of performance encoded data slices, dispersed error decode at least a decode threshold number of received performance encoded data slices to reproduce a data segment of the plurality of data segments.