Maintaining data storage in accordance with an access metric

• US 9,606,867 B2
• Filed: 04/07/2015
• Issued: 03/28/2017
• Est. Priority Date: 06/05/2014
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Volvovski, Ilya, Cabral, Bruno Hennig, Motwani, Manish, Cocagne, Thomas Darrel, Markison, Timothy W., Grube, Gary W., Leggette, Wesley, Resch, Jason K., Storm, Michael Colin, Dhuse, Greg, Vedpathak, Yogesh Ramesh, Khadiwala, Ravi
• Primary Examiner(s): Bryan, Jason
• Application Number:
  US14/680,459
• Publication Number:
  US 20150355960A1
• Time in Patent Office: 721 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/0775 Content or structure details of the error report, e.g. specific table structure, specific error fields
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 2211/1004 Adaptive RAID, i.e. RAID system adapts to changing circumstances, e.g. RAID1 becomes RAID5 as disks fill up
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

When an access metric regarding an encoded data object exceeds an access threshold, a method begins by a processing module of a dispersed storage network (DSN) retrieving encoded data slices of a first plurality of sets of encoded data slices and recovering the data object utilizing first dispersed storage error encoding parameters. The method continues with the processing module re-encoding the recovered data object using second dispersed storage error encoding parameters to produce a re-encoded data object, where the re-encoded data object includes a second plurality of sets of encoded data slices. The method continues with the processing module outputting the second plurality of sets of encoded data slices to storage units of the DSN for storage therein and sending a message to retrieving devices of the DSN, where the message indicates use of the second plurality of sets of encoded data slices for the data object.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:

determining that an access metric regarding an encoded data object exceeds an access threshold, wherein the encoded data object is stored as a first plurality of sets of encoded data slices based on first dispersed storage error encoding parameters; and

when the access metric exceeds the access threshold;

retrieving a first decode threshold number of encoded data slices of each set of the first plurality of sets of encoded data slices;

decoding the first decode threshold number of encoded data slices of each set to recover the data object;

re-encoding the recovered data object using second dispersed storage error encoding parameters to produce a re-encoded data object, wherein the re-encoded data object includes a second plurality of sets of encoded data slices and wherein a second decode threshold number associated with the second dispersed storage error encoding parameters is less than the first decode threshold number;

outputting the second plurality of sets of encoded data slices to storage units of the DSN for storage therein; and

sending a message to retrieving devices of the DSN, wherein the message indicates use of the second plurality of sets of encoded data slices for the data object.

2. The method of claim 1, wherein the access metric comprises one or more of:

network traffic loading information;

access rate of the data object; and

storage unit loading of a set of storage units storing the first plurality of sets of encoded data slices.

3. The method of claim 1 further comprises:

the second decode threshold number is less than or equal to one-half of the first decode threshold number.

4. The method of claim 1 further comprises:

the second decode threshold number is less than or equal to one-half of a total number of encoded data slices in a set of the second plurality of sets of encoded data slices.

5. The method of claim 1 further comprises:

outputting, by each storage unit in a first subset of the storage units, an encoded data slice of a set of the second plurality of sets of encoded data slices in response to a first retrieval request for the re-encoded data object; and

outputting, by each storage unit in a second subset of the storage units, an encoded data slice of the set of the second plurality of sets of encoded data slices in response to a second retrieval request for the re-encoded data object.

6. The method of claim 5 further comprises:

receiving, by the storage units, the first and second retrieval requests for the re-encoded data object from first and second retrieving devices of the retrieving devices at substantially the same time.

7. The method of claim 1 further comprises:

when the access metric drops below the access threshold, sending a second message to the retrieving devices, wherein the second message indicates use of the first plurality of sets of encoded data slices for the data object.

8. The method of claim 7 further comprises:

sending deletion messages to the storage units, wherein the deletion messages instruct the storage units to delete the second plurality of sets of encoded data slices.

9. A non-transitory computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), causes the one or more computing devices to;

determine that an access metric regarding an encoded data object exceeds an access threshold, wherein the encoded data object is stored as a first plurality of sets of encoded data slices based on first dispersed storage error encoding parameters; and

when the access metric exceeds the access threshold;

retrieve a first decode threshold number of encoded data slices of each set of the first plurality of sets of encoded data slices;

decode the first decode threshold number of encoded data slices of each set to recover the data object;

re-encode the recovered data object using second dispersed storage error encoding parameters to produce a re-encoded data object, wherein the re-encoded data object includes a second plurality of sets of encoded data slices and wherein a second decode threshold number associated with the second dispersed storage error encoding parameters is less than the first decode threshold number;

output the second plurality of sets of encoded data slices to storage units of the DSN for storage therein; and

send a message to retrieving devices of the DSN, wherein the message indicates use of the second plurality of sets of encoded data slices for the data object.

10. The non-transitory computer readable storage medium of claim 9, wherein the access metric comprises one or more of:

network traffic loading information;

access rate of the data object; and

storage unit loading of a set of storage units storing the first plurality of sets of encoded data slices.

11. The non-transitory computer readable storage medium of claim 9 further comprises:

the second decode threshold number is less than or equal to one-half of the first decode threshold number.

12. The non-transitory computer readable storage medium of claim 9 further comprises:

the second decode threshold number is less than or equal to one-half of a total number of encoded data slices in a set of the second plurality of sets of encoded data slices.

13. The non-transitory computer readable storage medium of claim 9 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

output, by each storage unit in a first subset of the storage units, an encoded data slice of a set of the second plurality of sets of encoded data slices in response to a first retrieval request for the re-encoded data object; and

output, by each storage unit in a second subset of the storage units, an encoded data slice of the set of the second plurality of sets of encoded data slices in response to a second retrieval request for the re-encoded data object.

14. The non-transitory computer readable storage medium of claim 13 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

receive, by the storage units, the first and second retrieval requests for the re-encoded data object from first and second retrieving devices of the retrieving devices at substantially the same time.

15. The non-transitory computer readable storage medium of claim 9 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

when the access metric drops below the access threshold, send a second message to the retrieving devices, wherein the second message indicates use of the first plurality of sets of encoded data slices for the data object.

16. The non-transitory computer readable storage medium of claim 15 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

send deletion messages to the storage units, wherein the deletion messages instruct the storage units to delete the second plurality of sets of encoded data slices.

17. A computing device of a group of computing devices of a dispersed storage network (DSN), the computing device comprises:

an interface;

a local memory; and

a processing module operably coupled to the interface and the local memory, wherein the processing module functions to;

determine that an access metric regarding an encoded data object exceeds an access threshold, wherein the encoded data object is stored as a first plurality of sets of encoded data slices based on first dispersed storage error encoding parameters; and

when the access metric exceeds the access threshold;

retrieve a first decode threshold number of encoded data slices of each set of the first plurality of sets of encoded data slices;

decode the first decode threshold number of encoded data slices of each set to recover the data object;

re-encode the recovered data object using second dispersed storage error encoding parameters to produce a re-encoded data object, wherein the re-encoded data object includes a second plurality of sets of encoded data slices and wherein a second decode threshold number associated with the second dispersed storage error encoding parameters is less than the first decode threshold number;

output, via the interface, the second plurality of sets of encoded data slices to storage units of the DSN for storage therein; and

send, via the interface, a message to retrieving devices of the DSN, wherein the message indicates use of the second plurality of sets of encoded data slices for the data object.

18. The computing device of claim 17, wherein the access metric comprises one or more of:

network traffic loading information;

access rate of the data object; and

storage unit loading of a set of storage units storing the first plurality of sets of encoded data slices.

19. The computing device of claim 17 further comprises:

the second decode threshold number is less than or equal to one-half of the first decode threshold number.

20. The computing device of claim 17 further comprises:

the second decode threshold number is less than or equal to one-half of a total number of encoded data slices in a set of the second plurality of sets of encoded data slices.

21. The computing device of claim 17, wherein the processing module further functions to:

output, by each storage unit in a first subset of the storage units, an encoded data slice of a set of the second plurality of sets of encoded data slices in response to a first retrieval request for the re-encoded data object; and

output, by each storage unit in a second subset of the storage units, an encoded data slice of the set of the second plurality of sets of encoded data slices in response to a second retrieval request for the re-encoded data object.

22. The computing device of claim 21, wherein the processing module further functions to:

receive, by the storage units, the first and second retrieval requests for the re-encoded data object from first and second retrieving devices of the retrieving devices at substantially the same time.

23. The computing device of claim 17, wherein the processing module further functions to:

when the access metric drops below the access threshold, send, via the interface, a second message to the retrieving devices, wherein the second message indicates use of the first plurality of sets of encoded data slices for the data object.

24. The computing device of claim 23, wherein the processing module further functions to:

send, via the interface, deletion messages to the storage units, wherein the deletion messages instruct the storage units to delete the second plurality of sets of encoded data slices.