Predictive rebalancing according to future usage expectations

• US 10,257,276 B2
• Filed: 07/27/2018
• Issued: 04/09/2019
• Est. Priority Date: 09/30/2005
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Motwani, Manish, Kaczmarek, Joseph M., Resch, Jason K.
• Primary Examiner(s): Bradley, Matthew
• Application Number:
  US16/048,011
• Publication Number:
  US 20180337997A1
• Time in Patent Office: 256 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A computing device includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and processing circuitry operably coupled to the interface and to the memory. The processing circuitry is configured to execute the operational instructions to perform various operations and functions. The computing device detects at least one available memory device within a storage unit (SU). The computing device identifies storage capacities of each of the memory devices within the SU and identifies a DSN address range associated with the SU. The computing device maps the DSN address range to each of the memory devices within the SU based on the storage capacities to generate a memory mapping of the memory devices within the SU. The computing device then facilitates redistribution of some EDS from a first memory device to the at least one available memory device within the SU.

Claims

1. A computing device comprising:

an interface configured to interface and communicate with a dispersed or distributed storage network (DSN);

memory that stores operational instructions; and

processing circuitry operably coupled to the interface and to the memory, wherein the processing circuitry is configured to execute the operational instructions to;

detect at least one available memory device within a storage unit (SU) among a plurality of memory devices within the SU, wherein a plurality of SUs that includes the SU distributedly stores sets of encoded data slices (EDSs) associated with a data object, wherein at least some EDSs of the sets of EDSs associated with the data object that are stored in a first memory device of the plurality of memory devices within the SU are associated with a DSN address range, wherein the at least one available memory device includes a second memory device of the plurality of memory devices within the SU that is newly available within the SU having been added to the SU after the first memory device of the plurality of memory devices within the SU;

identify storage capacities of each of the plurality of memory devices within the SU;

identify the DSN address range associated with the SU;

map the DSN address range to each of the plurality of memory devices within the SU based on the storage capacities of each of the plurality of memory devices within the SU that have been identified to generate a memory mapping of the plurality of memory devices within the SU; and

facilitate redistribution of at least one of the at least some EDSs of the sets of EDSs associated with the data object that are stored in the first memory device of the plurality of memory devices within the SU to the at least one available memory device within the SU based on the memory mapping of the plurality of memory devices within the SU.

2. The computing device of claim 1, wherein the processing circuitry is further configured to execute the operational instructions to:

detect the at least one available memory device within the SU among the plurality of memory devices within the SU including based on at least one of to receive a message from at least one of another computing device or the SU, to perform a test, to interpret a test result, or to interpret a schedule.

3. The computing device of claim 1, wherein the processing circuitry is further configured to execute the operational instructions to:

identify the storage capacities of the plurality of memory devices within the SU based on at least one of to perform a lookup, to perform a performance test, to initiate a query, or to interpret a query response.

4. The computing device of claim 1, wherein the processing circuitry is further configured to execute the operational instructions to:

identify the DSN address range associated with the SU based on at least one of to interpret a system registry entry, to perform a lookup, to use a predetermination, to issue a query, or to interpret a query response.

5. The computing device of claim 1, wherein the processing circuitry is further configured to execute the operational instructions to:

map a first DSN address sub-range of the DSN address range to the first memory device of the plurality of memory devices within the SU; and

map a second DSN address sub-range of the DSN address range to the second memory device of the plurality of memory devices within the SU.

6. The computing device of claim 1, wherein:

the data object is segmented into a plurality of data segments;

a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce a set of EDSs that is of pillar width, wherein the set of EDSs is one of the sets of EDSs associated with the data object;

a decode threshold number of EDSs are needed to recover the data segment;

a read threshold number of EDSs provides for reconstruction of the data segment;

a write threshold number of EDSs provides for a successful transfer of the set of EDSs from a first at least one location in the DSN to a second at least one location in the DSN;

the set of EDSs is of pillar width and includes a pillar number of EDSs;

each of the decode threshold number, the read threshold number, and the write threshold number is less than the pillar number; and

the write threshold number is greater than or equal to the read threshold number that is greater than or equal to the decode threshold number.

7. The computing device of claim 1 further comprising:

another SU of the plurality of SUs within the DSN, a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, or a video game device.

8. The computing device of claim 1, wherein at least one of:

the computing device is located at a first premises that is remotely located from a second premises of at least one SU of the plurality of SUs within the DSN;

or

the DSN includes at least one of a wireless communication system, a wire lined communication system, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).

9. A computing device comprising:

an interface configured to interface and communicate with a dispersed or distributed storage network (DSN);

memory that stores operational instructions; and

processing circuitry operably coupled to the interface and to the memory, wherein the processing circuitry is configured to execute the operational instructions to;

detect at least one available memory device within a storage unit (SU) among a plurality of memory devices within the SU including based on at least one of to receive a message from at least one of another computing device or the SU, to perform a test, to interpret a test result, or to interpret a schedule, wherein a plurality of SUs that includes the SU distributedly stores sets of encoded data slices (EDSs) associated with a data object, wherein at least some EDSs of the sets of EDSs associated with the data object that are stored in a first memory device of the plurality of memory devices within the SU are associated with a DSN address range, wherein the at least one available memory device includes a second memory device of the plurality of memory devices within the SU that is newly available within the SU having been added to the SU after the first memory device of the plurality of memory devices within the SU;

identify storage capacities of each of the plurality of memory devices within the SU;

identify the DSN address range associated with the SU;

map the DSN address range to each of the plurality of memory devices within the SU based on the storage capacities of each of the plurality of memory devices within the SU that have been identified to generate a memory mapping of the plurality of memory devices within the SU including to;

map a first DSN address sub-range of the DSN address range to the first memory device of the plurality of memory devices within the SU; and

map a second DSN address sub-range of the DSN address range to the second memory device of the plurality of memory devices within the SU; and

facilitate redistribution of at least one of the at least some EDSs of the sets of EDSs associated with the data object that are stored in the first memory device of the plurality of memory devices within the SU to the at least one available memory device within the SU based on the memory mapping of the plurality of memory devices within the SU.

10. The computing device of claim 9, wherein the processing circuitry is further configured to execute the operational instructions to:

identify the storage capacities of the plurality of memory devices within the SU based on at least one of to perform a lookup, to perform a performance test, to initiate a query, or to interpret a query response.

11. The computing device of claim 9, wherein the processing circuitry is further configured to execute the operational instructions to:

identify the DSN address range associated with the SU based on at least one of to interpret a system registry entry, to perform a lookup, to use a predetermination, to issue a query, or to interpret a query response.

12. The computing device of claim 9, wherein:

the data object is segmented into a plurality of data segments;

a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce a set of EDSs that is of pillar width, wherein the set of EDSs is one of the sets of EDSs associated with the data object;

a decode threshold number of EDSs are needed to recover the data segment;

a read threshold number of EDSs provides for reconstruction of the data segment;

a write threshold number of EDSs provides for a successful transfer of the set of EDSs from a first at least one location in the DSN to a second at least one location in the DSN;

the set of EDSs is of pillar width and includes a pillar number of EDSs;

each of the decode threshold number, the read threshold number, and the write threshold number is less than the pillar number; and

the write threshold number is greater than or equal to the read threshold number that is greater than or equal to the decode threshold number.

13. The computing device of claim 9, wherein at least one of:

the computing device is located at a first premises that is remotely located from a second premises of at least one SU of the plurality of SUs within the DSN;

the computing device includes another SU of the plurality of SUs within the DSN, a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, or a video game device;

or

the DSN includes at least one of a wireless communication system, a wire lined communication system, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).

14. A method for execution by a computing device, the method comprising:

detecting, via an interface of the computing device configured to interface and communicate with a dispersed or distributed storage network (DSN), at least one available memory device within a storage unit (SU) among a plurality of memory devices within the SU, wherein a plurality of SUs that includes the SU distributedly stores sets of encoded data slices (EDSs) associated with a data object, wherein at least some EDSs of the sets of EDSs associated with the data object that are stored in a first memory device of the plurality of memory devices within the SU are associated with a DSN address range, wherein the at least one available memory device includes a second memory device of the plurality of memory devices within the SU that is newly available within the SU having been added to the SU after the first memory device of the plurality of memory devices within the SU;

identifying storage capacities of each of the plurality of memory devices within the SU;

identifying the DSN address range associated with the SU;

mapping the DSN address range to each of the plurality of memory devices within the SU based on the storage capacities of each of the plurality of memory devices within the SU that have been identified to generate a memory mapping of the plurality of memory devices within the SU; and

facilitating, via the interface, redistribution of at least one of the at least some EDSs of the sets of EDSs associated with the data object that are stored in the first memory device of the plurality of memory devices within the SU to the at least one available memory device within the SU based on the memory mapping of the plurality of memory devices within the SU.

15. The method of claim 14 further comprising:

detecting the at least one available memory device within the SU among the plurality of memory devices within the SU including based on at least one of to receive a message from at least one of another computing device or the SU, to perform a test, to interpret a test result, or to interpret a schedule.

16. The method of claim 14 further comprising:

identifying the storage capacities of the plurality of memory devices within the SU based on at least one of to perform a lookup, to perform a performance test, to initiate a query, or to interpret a query response.

17. The method of claim 14 further comprising:

identifying the DSN address range associated with the SU based on at least one of to interpret a system registry entry, to perform a lookup, to use a predetermination, to issue a query, or to interpret a query response.

18. The method of claim 14 further comprising:

mapping a first DSN address sub-range of the DSN address range to the first memory device of the plurality of memory devices within the SU; and

mapping a second DSN address sub-range of the DSN address range to the second memory device of the plurality of memory devices within the SU.

19. The method of claim 14, wherein:

the data object is segmented into a plurality of data segments;

a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce a set of EDSs that is of pillar width, wherein the set of EDSs is one of the sets of EDSs associated with the data object;

a decode threshold number of EDSs are needed to recover the data segment;

a read threshold number of EDSs provides for reconstruction of the data segment;

a write threshold number of EDSs provides for a successful transfer of the set of EDSs from a first at least one location in the DSN to a second at least one location in the DSN;

the set of EDSs is of pillar width and includes a pillar number of EDSs;

each of the decode threshold number, the read threshold number, and the write threshold number is less than the pillar number; and

the write threshold number is greater than or equal to the read threshold number that is greater than or equal to the decode threshold number.

20. The method of claim 14, wherein at least one of:

the computing device is located at a first premises that is remotely located from a second premises of at least one SU of the plurality of SUs within the DSN;

the computing device includes another SU of the plurality of SUs within the DSN, a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, or a video game device;

or

the DSN includes at least one of a wireless communication system, a wire lined communication system, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).