Expanding a dispersed storage network (DSN)

• US 10,223,036 B2
• Filed: 08/10/2016
• Issued: 03/05/2019
• Est. Priority Date: 08/10/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Leggette, Wesley B., Motwani, Manish, Ober, Brian F., Resch, Jason K.
• Primary Examiner(s): Dorman, Christian M
• Application Number:
  US15/233,712
• Publication Number:
  US 20180046403A1
• Time in Patent Office: 937 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/3006 where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems multiprogramming arrangements G06F9/46; allocation of resources G06F9/50
• CPC Class Codes: G06F 16/134 Distributed indices
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/5016 the resource being the memory
• CPC Class Codes: G06F 9/5083 Techniques for rebalancing the load in a distributed system
• CPC Class Codes: H04L 63/104 Grouping of entities
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A computing device includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and a processing module that is configured to perform various operations. The computing device monitors for condition(s) that triggers expansion of a private DSN memory that stores encoded data slices (EDSs), and when that condition occurs, the computing device generates a modified copy of the EDSs that includes a read and/or write threshold number of EDSs of the EDSs. The computing device transmits the modified copy of EDSs to a public DSN memory for storage within the public DSN memory. The computing device then services first read request and/or write request based on the private DSN memory that stores the plurality of EDSs and services second read request and/or write request based on public DSN memory that stores the modified copy of the plurality of EDSs.

Claims

1. A computing device comprising:

an interface configured to interface and communicate with a dispersed storage network (DSN);

memory that stores operational instructions; and

a processing module operably coupled to the interface and to the memory, wherein the processing module, when operable within the computing device based on the operational instructions, is configured to;

when a read expansion condition is met for a set of encoded data slices that is stored in a set of storage units of a first DSN, wherein the set of encoded data slices includes a pillar width number of encoded data slices, and wherein a decode threshold number of encoded data slices of the set of encoded data is needed to recover a data segment;

retrieve, via the interface, a read threshold number of encoded data slices of the set of encoded data slices from some of the storage units in the set of storage units, wherein the read threshold number is less than the pillar width number and greater than or equal to the decode threshold number;

send, via the interface, the read threshold number of encoded data slices to storage units of a second DSN for storage therein;

receive a plurality of read requests for the set of encoded data slices;

send, via the interface, a first group of the plurality of read requests to the set of storage units of the first DSN for processing; and

send, via the interface, a second group of the plurality of read requests to the storage units of the second DSN for processing.

2. The computing device of claim 1, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to:

when a write expansion condition is met for multitudes of sets of encoded data slices that are to be stored in the set of storage units of the first DSN;

determine a write threshold number of storage units of a second DSN, wherein the write threshold number is less than or equal to the pillar width number and greater than the read threshold number;

receive a plurality of write requests for the multitudes of sets of encoded data slices;

send, via the interface, a first group of the plurality of write requests to the set of storage units of the first DSN for processing; and

send, via the interface, a second group of the plurality of write requests to the write threshold number of storage units of the second DSN for processing, wherein the second group includes one or more write threshold number of sets of encoded data slices.

3. The computing device of claim 2, wherein the write expansion condition includes one or more of:

a change in available memory capacity associated with the first DSN;

currently stored data within the first DSN exceeds a storage capacity threshold associated with the first DSN;

currently stored data within the first DSN and expected inbound data to be stored within the first DSN is forecast to exceed the storage capacity threshold associated with the first DSN;

the currently stored data within the first DSN and actual inbound data to be stored within the first DSN exceed the storage capacity associated with the first DSN;

a change in operational status of available memory access resources of the first DSN to service write requests associated the currently stored data within the first DSN;

or

write requests associated with the currently stored data within the first DSN exceeds a first DSN access threshold.

4. The computing device of claim 2, wherein when the write expansion condition is no longer met:

send, via the interface, the one or more write threshold number of sets of encoded data slices to storage units of the first DSN; and

delete the one or more write threshold number of sets of encoded data slices stored in the second DSN.

5. The computing device of claim 1, wherein the read expansion condition includes one or more of:

a change in operational status of available memory access resources of the first DSN to service read requests associated the set of encoded data slices within the first DSN; and

read requests associated with the set of encoded data slices within the first DSN exceeds a first DSN access threshold.

6. The computing device of claim 1, wherein:

the first DSN includes one or more private storage units (SUs); and

the second DSN includes one or more public SUs.

7. The computing device of claim 1, wherein:

when the read expansion condition is no longer met;

delete the read threshold number of encoded data slices stored in the second DSN.

8. The computing device of claim 1, wherein the computing device includes a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, or a video game device.

9. The computing device of claim 1, wherein the DSN includes at least one of a wireless communication system, a wire lined communication systems, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).

10. A method for execution by a computing device, the method comprising:

when a read expansion condition is met for a set of encoded data slices that is stored in a set of storage units of a first DSN, wherein the set of encoded data slices includes a pillar width number of encoded data slices, and wherein a decode threshold number of encoded data slices of the set of encoded data is needed to recover a data segment;

retrieving, by the computing device, a read threshold number of encoded data slices of the set of encoded data slices from some of the storage units in the set of storage units, wherein the read threshold number is less than the pillar width number and greater than or equal to the decode threshold number;

sending, by the computing device, the read threshold number of encoded data slices to storage units of a second DSN for storage therein;

receiving, by the computing device, a plurality of read requests for the set of encoded data slices;

sending, by the computing device, a first group of the plurality of read requests to the set of storage units of the first DSN for processing; and

sending, by the computing device, a second group of the plurality of read requests to the storage units of the second DSN for processing.

11. The method of claim 10 further comprising:

when a write expansion condition is met for multitudes of sets of encoded data slices that are to be stored in the set of storage units of the first DSN;

determining, by the computing device, a write threshold number of storage units of a second DSN, wherein the write threshold number is less than or equal to the pillar width number and greater than the read threshold number;

receiving, by the computing device, a plurality of write requests for the multitudes of sets of encoded data slices;

sending, by the computing device, a first group of the plurality of write requests to the set of storage units of the first DSN for processing; and

sending, by the computing device, a second group of the plurality of write requests to the write threshold number of storage units of the second DSN for processing, wherein the second group includes one or more write threshold number of sets of encoded data slices.

12. The method of claim 11, wherein the write expansion condition includes one or more of:

a change in memory capacity associated with the first DSN;

currently stored data within the first DSN exceeds a storage capacity threshold associated with the first DSN;

currently stored data within the first DSN and expected inbound data to be stored within the first DSN is forecast to exceed the storage capacity threshold associated with the first DSN;

the currently stored data within the first DSN and actual inbound data to be stored within the first DSN exceed the storage capacity associated with the first DSN;

a change in operational status of available memory access resources of the first DSN to service write requests associated the currently stored data within the first DSN;

or

write requests associated with the currently stored data within the first DSN exceeds a first DSN access threshold.

13. The method of claim 11, wherein when the write expansion condition is no longer met:

sending the one or more write threshold number of sets of encoded data slices to storage units of the first DSN; and

deleting the one or more write threshold number of sets of encoded data slices stored in the second DSN.

14. The method of claim 10, wherein the read expansion condition includes one or more of:

a change in operational status of available memory access resources of the first DSN to service read requests associated the set of encoded data slices within the first DSN; and

read requests associated with the set of encoded data slices within the first DSN exceeds a first DSN access threshold.

15. The method of claim 10, wherein:

the first DSN memory includes one or more private storage units (SUs) within the DSN; and

the second DSN memory includes one or more public SUs.

16. The method of claim 10, wherein:

when the read expansion condition is no longer met;

deleting the read threshold number of encoded data slices stored in the second DSN.

17. The method of claim 10, wherein the computing device includes a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, or a video game device.

18. The method of claim 10, wherein the DSN includes at least one of a wireless communication system, a wire lined communication systems, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).