Maintaining data storage in accordance with an access metric

• US 10,102,069 B2
• Filed: 01/06/2017
• Issued: 10/16/2018
• Est. Priority Date: 06/05/2014
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Volvovski, Ilya, Cabral, Bruno Hennig, Motwani, Manish, Cocagne, Thomas Darrel, Markison, Timothy W., Grube, Gary W., Leggette, Wesley, Resch, Jason K., Storm, Michael Colin, Dhuse, Greg, Vedpathak, Yogesh Ramesh, Khadiwala, Ravi
• Primary Examiner(s): Bryan, Jason B
• Application Number:
  US15/400,767
• Publication Number:
  US 20170116082A1
• Time in Patent Office: 648 Days
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/0775 Content or structure details of the error report, e.g. specific table structure, specific error fields
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 2211/1004 Adaptive RAID, i.e. RAID system adapts to changing circumstances, e.g. RAID1 becomes RAID5 as disks fill up
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method includes identifying a data object for retrieval. The data object is encoded in accordance with first encoded parameters and stored as a plurality of sets of encoded data slices in a set of storage units. The method further includes determining whether an access metric regarding the data object exceeds an access threshold. When the access metric is equal to or exceeds the access threshold, the data object is further encoded in accordance with second encoding parameters and stored as a second plurality of sets of encoded data slices in a second set of storage units. The method further includes issuing retrieval requests to the second set of storage units regarding the second plurality of encoded data slices in accordance with a read threshold of the second encoding parameters. The method further includes recovering the data object from the second plurality of encoded data slices.

Claims

1. A method for execution by a computing device of a dispersed storage network (DSN), the method comprises:

identifying a data object for retrieval from DSN memory, wherein the data object is stored as a plurality of sets of encoded data slices in a set of storage units of the DSN, wherein the data object is dispersed storage error encoded with first encoding parameters to produce the plurality of sets of encoded data slices;

determining whether an access metric regarding the data object exceeds an access threshold, wherein, when the access metric is equal to or exceeds the access threshold, the data object is further stored as a second plurality of sets of encoded data slices in a second set of storage units of the DSN, wherein the data object is further dispersed storage error encoded with second encoding parameters to produce the second plurality of sets of encoded data slices, and wherein a first decode threshold and a first pillar width of the first encoding parameters are greater than a second decode threshold and a second pillar width of the second encoding parameters;

when the access metric exceeds the access threshold;

issuing a plurality of sets of retrieval requests to the second set of storage units regarding the second plurality of encoded data slices in accordance with a read threshold of the second encoding parameters; and

recovering the data object from received encoded data slices of the second plurality of encoded data slices.

2. The method of claim 1 further comprises:

when the access metric is less than the access threshold;

issuing a first plurality of sets of retrieval requests to the set of storage units regarding the first plurality of encoded data slices in accordance with a read threshold of the first encoding parameters; and

recovering the data object from received encoded data slices of the first plurality of encoded data slices.

3. The method of claim 1, wherein the access metric comprises one or more of:

network traffic loading information;

access rate of the data object; and

storage unit loading of a set of storage units storing the first plurality of sets of encoded data slices.

4. The method of claim 1 further comprises:

the second decode threshold number is less than or equal to one-half of the first decode threshold number.

5. The method of claim 1 further comprises:

the second decode threshold number is less than or equal to one-half of a total number of encoded data slices in a set of the second plurality of sets of encoded data slices.

6. The method of claim 1, wherein the recovering the data object from received encoded data slices of the second plurality of encoded data slices comprises:

for a first data segment of the data object;

receiving the second decode threshold number of encoded data slices of a first set of encoded data slices of the second plurality of sets of encoded data slices; and

dispersed storage error decoding the second decode threshold number of encoded data slices of the first set to recover the first data segment;

for a second data segment of the data object;

receiving the second decode threshold number of encoded data slices of a second set of encoded data slices of the second plurality of sets of encoded data slices; and

dispersed storage error decoding the second decode threshold number of encoded data slices of the second set to recover the first data segment; and

combining the first and second data segments to produce at least a portion of the data object.

7. A computer readable memory comprises:

at least one memory section that stores operational instructions that, when executed by a computing device of a dispersed storage network (DSN), causes the computing device to;

identify a data object for retrieval from DSN memory, wherein the data object is stored as a plurality of sets of encoded data slices in a set of storage units of the DSN, wherein the data object is dispersed storage error encoded with first encoding parameters to produce the plurality of sets of encoded data slices;

determine whether an access metric regarding the data object exceeds an access threshold, wherein, when the access metric is equal to or exceeds the access threshold, the data object is further stored as a second plurality of sets of encoded data slices in a second set of storage units of the DSN, wherein the data object is further dispersed storage error encoded with second encoding parameters to produce the second plurality of sets of encoded data slices, and wherein a first decode threshold and a first pillar width of the first encoding parameters are greater than a second decode threshold and a second pillar width of the second encoding parameters;

when the access metric exceeds the access threshold;

issue a plurality of sets of retrieval requests to the second set of storage units regarding the second plurality of encoded data slices in accordance with a read threshold of the second encoding parameters; and

recover the data object from received encoded data slices of the second plurality of encoded data slices.

8. The computer readable memory of claim 7, wherein the at least one memory section further stores operational instructions that, when executed by the computing device, causes the computing device to:

when the access metric is less than the access threshold;

issue a first plurality of sets of retrieval requests to the set of storage units regarding the first plurality of encoded data slices in accordance with a read threshold of the first encoding parameters; and

recover the data object from received encoded data slices of the first plurality of encoded data slices.

9. The computer readable memory of claim 7, wherein the access metric comprises one or more of:

network traffic loading information;

access rate of the data object; and

storage unit loading of a set of storage units storing the first plurality of sets of encoded data slices.

10. The computer readable memory of claim 7 further comprises:

the second decode threshold number is less than or equal to one-half of the first decode threshold number.

11. The computer readable memory of claim 7 further comprises:

the second decode threshold number is less than or equal to one-half of a total number of encoded data slices in a set of the second plurality of sets of encoded data slices.

12. The computer readable memory of claim 7, wherein the at least one memory section further stores operational instructions that, when executed by the computing device, causes the computing device to recover the data object from received encoded data slices of the second plurality of encoded data slices by:

for a first data segment of the data object;

receiving the second decode threshold number of encoded data slices of a first set of encoded data slices of the second plurality of sets of encoded data slices; and

dispersed storage error decoding the second decode threshold number of encoded data slices of the first set to recover the first data segment;

for a second data segment of the data object;

receiving the second decode threshold number of encoded data slices of a second set of encoded data slices of the second plurality of sets of encoded data slices; and

dispersed storage error decoding the second decode threshold number of encoded data slices of the second set to recover the first data segment; and

combining the first and second data segments to produce at least a portion of the data object.

13. A computing device of a dispersed storage network (DSN), the computing device comprises:

an interface;

a local memory; and

a processing module operably coupled to the interface and the local memory, wherein the processing module functions to;

identify a data object for retrieval from DSN memory, wherein the data object is stored as a plurality of sets of encoded data slices in a set of storage units of the DSN, wherein the data object is dispersed storage error encoded with first encoding parameters to produce the plurality of sets of encoded data slices;

determine whether an access metric regarding the data object exceeds an access threshold, wherein, when the access metric is equal to or exceeds the access threshold, the data object is further stored as a second plurality of sets of encoded data slices in a second set of storage units of the DSN, wherein the data object is further dispersed storage error encoded with second encoding parameters to produce the second plurality of sets of encoded data slices, and wherein a first decode threshold and a first pillar width of the first encoding parameters are greater than a second decode threshold and a second pillar width of the second encoding parameters;

when the access metric exceeds the access threshold;

issue, via the interface, a plurality of sets of retrieval requests to the second set of storage units regarding the second plurality of encoded data slices in accordance with a read threshold of the second encoding parameters; and

recover the data object from received encoded data slices of the second plurality of encoded data slices.

14. The computing device of claim 13, wherein the processing module is further operable to:

when the access metric is less than the access threshold;

issue, via the interface, a first plurality of sets of retrieval requests to the set of storage units regarding the first plurality of encoded data slices in accordance with a read threshold of the first encoding parameters; and

recover the data object from received encoded data slices of the first plurality of encoded data slices.

15. The computing device of claim 13, wherein the access metric comprises one or more of:

network traffic loading information;

access rate of the data object; and

storage unit loading of a set of storage units storing the first plurality of sets of encoded data slices.

16. The computing device of claim 13 further comprises:

the second decode threshold number is less than or equal to one-half of the first decode threshold number.

17. The computing device of claim 13 further comprises:

the second decode threshold number is less than or equal to one-half of a total number of encoded data slices in a set of the second plurality of sets of encoded data slices.

18. The computing device of claim 13, wherein the processing module further functions to recover the data object from received encoded data slices of the second plurality of encoded data slices by:

for a first data segment of the data object;

receiving, via the interface, the second decode threshold number of encoded data slices of a first set of encoded data slices of the second plurality of sets of encoded data slices; and

dispersed storage error decoding the second decode threshold number of encoded data slices of the first set to recover the first data segment;

for a second data segment of the data object;

receiving, via the interface, the second decode threshold number of encoded data slices of a second set of encoded data slices of the second plurality of sets of encoded data slices; and

dispersed storage error decoding the second decode threshold number of encoded data slices of the second set to recover the first data segment; and

combining the first and second data segments to produce at least a portion of the data object.