MINIMIZING DATA MOVEMENT THROUGH ROTATION OF SPARE MEMORY DEVICES

• US 20180109618A1
• Filed: 12/15/2017
• Published: 04/19/2018
• Est. Priority Date: 01/30/2015
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Volvovski, Ilya, Resch, Jason K., Viraraghavan, Praveen
• Granted Patent: US 10,440,116 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/07 Responding to the occurrence of a fault, e.g. fault tolerance
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: H03M 13/09 Error detection only, e.g. using cyclic redundancy check [CRC] codes or single parity bit
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/3761 using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A distributed storage network (DSN) includes a plurality of distributed storage task execution (DST EX) units, each which includes a plurality of distributed storage (DS) memories configured to store encoded data slices. A DST EX unit detects a physical storage failure associated with a first DS memory included in the DST EX unit, wherein the first DS memory is associated with a sub-DSN address range. An available spare DS memory included in the DST EX unit is selected, and slice mapping information is updated to associate the available spare DS memory with the sub-DSN address range. The DST EX unit facilitates rebuilding one or more encoded data slices associated with the sub-DSN address range to produce one or more rebuilt encoded data slices. The DST EX unit facilitates storage of the rebuilt encoded data slices in the available spare DS memory.

Claims

1. A method for use in a distributed storage network (DSN) including a plurality of distributed storage task execution (DST EX) units, each of the plurality of DST EX units including a plurality of distributed storage (DS) memories configured to store encoded data slices, the method comprising:

detecting, by a DST EX unit, a physical storage failure associated with a first DS memory included in the DST EX unit, wherein the first DS memory is associated with a sub-DSN address range;

selecting an available spare DS memory included in the DST EX unit;

updating slice mapping information to associate the available spare DS memory with the sub-DSN address range;

facilitating, by the DST EX unit, rebuilding one or more encoded data slices associated with the sub-DSN address range to produce one or more rebuilt encoded data slices; and

facilitating, by the DST EX unit, storing the one or more rebuilt encoded data slices in the available spare DS memory.

2. The method of claim 1, further comprising:

after storing the one or more rebuilt encoded data slices in the available spare DS memory, maintaining the first DS memory as a spare DS memory until a physical storage failure associated with a second DS memory is detected.

3. The method of claim 1, further comprising:

choosing the available spare DS memory from a plurality of available spare DS memories based on one or more of a capacity level, an estimated performance level, or a compatibility factor.

4. The method of claim 1, further comprising:

updating slice mapping information by updating a local table that includes the slice mapping information to;

associate the sub-DSN address range with the available spare DS memory; and

disassociate the sub-DSN address range from the first DS memory.

5. The method of claim 1, further comprising:

locally rebuilding the one or more encoded data slices, by the DST EX unit, to generate one or more rebuilt encoded data slices.

6. The method of claim 5, wherein locally rebuilding includes:

obtaining a decode threshold number of slice representations of a set of encoded data slices that includes an encoded data slice for rebuilding;

decoding the decode threshold number of slice representations to reproduce a data segment; and

encoding the data segment to generate the one or more rebuilt encoded data slices.

7. The method of claim 1, further comprising:

issuing a rebuild request to a rebuilding agent, the rebuild request including instructions to the rebuilding agent to send rebuilt encoded data slices to the DST EX unit;

receiving, at the DST EX unit, one or more rebuilt data slices from the rebuilding agent; and

storing the one or more rebuilt data slices in the available spare DS memory.

8. A distributed storage task execution (DST EX) unit for use in a distributed storage network (DSN), the DST EX unit comprising:

a processor and associated memory;

a plurality of distributed storage (DS) memories coupled to the processor and associated memory, the plurality of DS memories configured to store encoded data slices;

the processor and associated memory configured to;

detect a physical storage failure associated with a first DS memory, wherein the first DS memory is associated with a sub-DSN address range;

select an available spare DS memory from the plurality of DS memories;

update slice mapping information to associate the available spare DS memory with the sub-DSN address range;

facilitate rebuilding one or more encoded data slices associated with the sub-DSN address range to produce one or more rebuilt encoded data slices; and

facilitate storing the one or more rebuilt encoded data slices in the available spare DS memory.

9. The distributed storage task execution (DST EX) unit of claim 8, the processor and associated memory further configured to:

after storing the one or more rebuilt encoded data slices in the available spare DS memory, maintain the first DS memory as a spare DS memory until a physical storage failure associated with a second DS memory is detected.

10. The distributed storage task execution (DST EX) unit of claim 8, the processor and associated memory further configured to:

choose the available spare DS memory from a plurality of available spare DS memories based on one or more of a capacity level, an estimated performance level, or a compatibility factor.

11. The distributed storage task execution (DST EX) unit of claim 8, the processor and associated memory further configured to:

update a local table including slice mapping information to;

associate the sub-DSN address range with the available spare DS memory; and

disassociate the sub-DSN address range from the first DS memory.

12. The distributed storage task execution (DST EX) unit of claim 8, the processor and associated memory further configured to:

locally rebuild the one or more encoded data slices to generate one or more rebuilt encoded data slices.

13. The distributed storage task execution (DST EX) unit of claim 12, the processor and associated memory further configured to:

obtain a decode threshold number of slice representations of a set of encoded data slices that includes an encoded data slice for rebuilding;

decode the decode threshold number of slice representations to reproduce a data segment; and

encode the data segment to generate the one or more rebuilt encoded data slices.

14. The distributed storage task execution (DST EX) unit of claim 8, the processor and associated memory further configured to:

issue a rebuild request to a rebuilding agent, the rebuild request including instructions to the rebuilding agent to send rebuilt encoded data slices to the DST EX unit;

receive one or more rebuilt data slices from the rebuilding agent; and

store the one or more rebuilt data slices in the available spare DS memory.

15. A distributed storage network (DSN) comprising:

a set of distributed storage task execution (DST EX) units, at least one DST EX unit included in the set of DST EX units including;

a processor and associated memory;

a plurality of distributed storage (DS) memories coupled to the processor and associated memory, the plurality of DS memories configured to store encoded data slices;

the processor and associated memory configured to;

detect a physical storage failure associated with a first DS memory, wherein the first DS memory is associated with a sub-DSN address range;

select an available spare DS memory from the plurality of DS memories;

update slice mapping information to associate the available spare DS memory with the sub-DSN address range;

facilitate rebuilding one or more encoded data slices associated with the sub-DSN address range to produce one or more rebuilt encoded data slices; and

facilitate storing the one or more rebuilt encoded data slices in the available spare DS memory.

16. The distributed storage network (DSN) of claim 15, the processor and associated memory further configured to:

after storing the one or more rebuilt encoded data slices in the available spare DS memory, maintain the first DS memory as a spare DS memory until a physical storage failure associated with a second DS memory is detected.

17. The distributed storage network (DSN) of claim 15, the processor and associated memory further configured to:

choose the available spare DS memory from a plurality of available spare DS memories based on one or more of a capacity level, an estimated performance level, or a compatibility factor.

18. The distributed storage network (DSN) of claim 15, the processor and associated memory further configured to:

update a local table including slice mapping information to;

associate the sub-DSN address range with the available spare DS memory; and

disassociate the sub-DSN address range from the first DS memory.

19. The distributed storage network (DSN) of claim 15, the processor and associated memory further configured to:

locally rebuild the one or more encoded data slices to generate one or more rebuilt encoded data slices, wherein locally rebuilding includes;

obtaining a decode threshold number of slice representations of a set of encoded data slices that includes an encoded data slice for rebuilding;

decoding the decode threshold number of slice representations to reproduce a data segment; and

encoding the data segment to generate the one or more rebuilt encoded data slices.

20. The distributed storage network (DSN) of claim 15, further including:

a rebuilding module coupled to the at least one DST EX unit, the rebuilding module configured to rebuild encoded data slices on behalf of the at least one DST EX unit;

the processor and associated memory further configured to;

issue a rebuild request to a rebuilding agent, the rebuild request including instructions to the rebuilding agent to send rebuilt encoded data slices to the DST EX unit;

receive one or more rebuilt data slices from the rebuilding agent; and

store the one or more rebuilt data slices in the available spare DS memory.