OPTIMIZED CACHING OF SLICES BY A DS PROCESSING UNIT

• US 20170153978A1
• Filed: 11/16/2016
• Published: 06/01/2017
• Est. Priority Date: 11/30/2015
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Motwani, Manish, Wozniak, Ethan S.
• Granted Patent: US 10,073,658 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/108 Parity data distribution in semiconductor storages, e.g. in SSD
• CPC Class Codes: G06F 11/1088 Reconstruction on already foreseen single or plurality of spare disks
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 11/1402 Saving, restoring, recovering or retrying
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: G06F 12/0891 using clearing, invalidating or resetting means
• CPC Class Codes: G06F 15/17331 Distributed shared memory [DSM], e.g. remote direct memory access [RDMA]
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 2211/1007 Addressing errors, i.e. silent errors in RAID, e.g. sector slipping and addressing errors
• CPC Class Codes: G06F 2212/1024 Latency reduction
• CPC Class Codes: G06F 2212/154 Networked environment
• CPC Class Codes: G06F 2212/263 Network storage, e.g. SAN or NAS
• CPC Class Codes: G06F 2212/403 Error protection encoding, e.g. using parity or ECC codes
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0629 Configuration or reconfiguration of storage systems
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0644 : Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0665 : at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 : Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 : Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: G06F 9/4881 : Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06F 9/5083 : Techniques for rebalancing the load in a distributed system
• CPC Class Codes: H03M 13/1105 : Decoding
• CPC Class Codes: H03M 13/1515 : Reed-Solomon codes
• CPC Class Codes: H03M 13/3761 : using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H03M 13/616 : Matrix operations, especially for generator matrices or check matrices, e.g. column or row permutations
• CPC Class Codes: H03M 13/6502 : Reduction of hardware complexity or efficient processing
• CPC Class Codes: H04L 2101/604 : Address structures or formats
• CPC Class Codes: H04L 61/457 : containing identifiers of data entities on a computer, e.g. file names
• CPC Class Codes: H04L 63/101 : Access control lists [ACL]
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A computing device includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and a processing module operably coupled to the interface and memory such that the processing module, when operable within the computing device based on the operational instructions, is configured to perform various operations. A computing device receives a data access request involving a set of EDSs associated with a data object that are distributedly stored among storage units (SUs) including first SU(s) coupled via a local network of the DSN and second SU(s) remotely located to the computing device and coupled via an external network of the DSN. The computing device caches within the at least one memory therein a subset of EDSs stored within the second SU(s) remotely located to the computing device and coupled to the computing device via the external network.

Claims

1. A computing device comprising:

an interface configured to interface and communicate with a dispersed storage network (DSN);

at least one memory that stores operational instructions and at least some encoded data slices (EDSs); and

a processing module operably coupled to the interface and to the at least one memory, wherein the processing module, when operable within the computing device based on the operational instructions, is configured to;

receive a data access request involving a set of EDSs associated with a data object that are distributedly stored among a plurality of storage units (SUs) that includes a first at least one SU that is coupled to the computing device via a local network of the DSN and a second at least one SU that is remotely located to the computing device and is coupled to the computing device via an external network of the DSN, wherein the data object is segmented into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce the set of EDSs, wherein a decode threshold number of EDSs are needed to recover the data segment, wherein a read threshold number of EDSs provides for reconstruction of the data segment, wherein a write threshold number of EDSs provides for a successful transfer of the set of EDSs from a first at least one location in the DSN to a second at least one location in the DSN;

cache within the at least one memory a subset of EDSs stored within the second at least one SU that is remotely located to the computing device and is coupled to the computing device via the external network; and

process the data access request involving the set of EDSs associated with the data object based on a first at least one EDS of the set of EDSs stored within the first at least one SU via the local network and based on at least one of a second at least one EDS of the set of EDSs stored within the at least one memory of the computing device or a third at least one EDS of the set of EDSs stored within the second at least one SU via the external network.

2. The computing device of claim 1, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to process the data access request involving the set of EDSs associated with the data object including to:

retrieve firstly the second at least one EDS of the set of EDSs stored within the at least one memory of the computing device;

retrieve secondly the first at least one EDS of the set of EDSs stored within the first at least one SU via the local network; and

retrieve thirdly the third at least one EDS of the set of EDSs stored within the second at least one SU via the external network.

3. The computing device of claim 1, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to process the data access request involving the set of EDSs associated with the data object including to:

retrieve at least one of the decode threshold number of EDSs, the read threshold number of EDSs, or the write threshold number of EDSs from the first at least one EDS of the set of EDSs stored within the first at least one SU via the local network and the second at least one EDS of the set of EDSs stored within the at least one memory of the computing device.

4. The computing device of claim 1, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to:

determine a first revision level of a first EDS having a slice name of the set of EDSs stored within the first at least one SU with a second revision level of a second EDS having the slice name of the set of EDSs stored within the at least one memory of the computing device; and

when the first revision level compares unfavorably to the second revision level, request from the first at least one SU the first EDS having the slice name of the set of EDSs stored within the first at least one SU to replace the second EDS having the slice name of the set of EDSs stored within the at least one memory of the computing device.

5. The computing device of claim 1, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to:

process the data access request involving the set of EDSs associated with the data object such that the set of EDSs associated with the data are distributedly stored among the plurality of SUs that includes n SUs, wherein n is a positive integer greater than or equal to 2, such that a first approximately 1/n EDSs are stored within the first at least one SU and a second approximately 1/n EDSs are stored within the second at least one SU; and

cache within the at least one memory a sufficient number of EDSs stored within the second at least one SU that is remotely located to the computing device and is coupled to the computing device via the external network so that at least one of the decode threshold number of EDSs, the read threshold number of EDSs, or the write threshold number of EDSs may be retrieved from the at least one memory and the first at least one SU.

6. The computing device of claim 1, wherein:

the computing device is located at a first premises that is remotely located from at least one SU of the plurality of SUs within the DSN; and

the at least one memory includes a first memory that stores operational instructions and a second memory implemented that is separately from the first memory and that stores the at least some EDSs.

7. The computing device of claim 1 further comprising:

a SU of the plurality of SUs within the DSN, a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, or a video game device.

8. The computing device of claim 1, wherein the DSN includes at least one of a wireless communication system, a wire lined communication systems, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).

9. A computing device comprising:

an interface configured to interface and communicate with a dispersed storage network (DSN);

at least one memory that stores operational instructions and at least some encoded data slices (EDSs); and

a processing module operably coupled to the interface and to the at least one memory, wherein the processing module, when operable within the computing device based on the operational instructions, is configured to;

receive a data access request involving a set of EDSs associated with a data object that are distributedly stored among a plurality of storage units (SUs) that includes n SUs, wherein n is a positive integer greater than or equal to 2, such that a first approximately 1/n EDSs are stored within a first at least one SU that is coupled to the computing device via a local network of the DSN and a second approximately 1/n EDSs are stored within a second at least one SU that is remotely located to the computing device and is coupled to the computing device via an external network of the DSN, wherein the data object is segmented into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce the set of EDSs, wherein a decode threshold number of EDSs are needed to recover the data segment, wherein a read threshold number of EDSs provides for reconstruction of the data segment, wherein a write threshold number of EDSs provides for a successful transfer of the set of EDSs from a first at least one location in the DSN to a second at least one location in the DSN;

cache within the at least one memory a subset of EDSs stored within the second at least one SU that is remotely located to the computing device and is coupled to the computing device via the external network; and

process the data access request involving the set of EDSs associated with the data object based on a first at least one EDS of the set of EDSs stored within the first at least one SU via the local network and based on at least one of a second at least one EDS of the set of EDSs stored within the at least one memory of the computing device or a third at least one EDS of the set of EDSs stored within the second at least one SU via the external network including to;

retrieve firstly the second at least one EDS of the set of EDSs stored within the at least one memory of the computing device; and

retrieve secondly the first at least one EDS of the set of EDSs stored within the first at least one SU via the local network.

10. The computing device of claim 9, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to process the data access request involving the set of EDSs associated with the data object including to:

retrieve at least one of the decode threshold number of EDSs, the read threshold number of EDSs, or the write threshold number of EDSs from the first at least one EDS of the set of EDSs stored within the first at least one SU via the local network and the second at least one EDS of the set of EDSs stored within the at least one memory of the computing device.

11. The computing device of claim 9, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to:

determine a first revision level of a first EDS having a slice name of the set of EDSs stored within the first at least one SU with a second revision level of a second EDS having the slice name of the set of EDSs stored within the at least one memory of the computing device; and

when the first revision level compares unfavorably to the second revision level, request from the first at least one SU the first EDS having the slice name of the set of EDSs stored within the first at least one SU to replace the second EDS having the slice name of the set of EDSs stored within the at least one memory of the computing device.

12. The computing device of claim 9 further comprising:

a SU of the plurality of SUs within the DSN, a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, or a video game device.

13. The computing device of claim 9, wherein the DSN includes at least one of a wireless communication system, a wire lined communication systems, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).

14. A method for execution by a computing device, the method comprising:

receiving, via an interface configured to interface and communicate with a dispersed storage network (DSN), a data access request involving a set of encoded data slices (EDSs) associated with a data object that are distributedly stored among a plurality of storage units (SUs) that includes a first at least one SU that is coupled to the computing device via a local network of the DSN and a second at least one SU that is remotely located to the computing device and is coupled to the computing device via an external network of the DSN, wherein the data object is segmented into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce the set of EDSs, wherein a decode threshold number of EDSs are needed to recover the data segment, wherein a read threshold number of EDSs provides for reconstruction of the data segment, wherein a write threshold number of EDSs provides for a successful transfer of the set of EDSs from a first at least one location in the DSN to a second at least one location in the DSN;

caching within at least one memory of the computing device a subset of EDSs stored within the second at least one SU that is remotely located to the computing device and is coupled to the computing device via the external network; and

processing the data access request involving the set of EDSs associated with the data object based on a first at least one EDS of the set of EDSs stored within the first at least one SU via the local network and based on at least one of a second at least one EDS of the set of EDSs stored within the at least one memory of the computing device or a third at least one EDS of the set of EDSs stored within the second at least one SU via the external network.

15. The method of claim 14, wherein the processing the data access request involving the set of EDSs associated with the data object further comprising:

retrieving firstly the second at least one EDS of the set of EDSs stored within the at least one memory of the computing device;

retrieving secondly the first at least one EDS of the set of EDSs stored within the first at least one SU via the local network; and

retrieving thirdly retrieve the third at least one EDS of the set of EDSs stored within the second at least one SU via the external network.

16. The method of claim 14, wherein the processing the data access request involving the set of EDSs associated with the data object further comprising:

retrieving at least one of the decode threshold number of EDSs, the read threshold number of EDSs, or the write threshold number of EDSs from the first at least one EDS of the set of EDSs stored within the first at least one SU via the local network and the second at least one EDS of the set of EDSs stored within the at least one memory of the computing device.

17. The method of claim 14 further comprising:

determining a first revision level of a first EDS having a slice name of the set of EDSs stored within the first at least one SU with a second revision level of a second EDS having the slice name of the set of EDSs stored within the at least one memory of the computing device; and

when the first revision level compares unfavorably to the second revision level, requesting from the first at least one SU the first EDS having the slice name of the set of EDSs stored within the first at least one SU to replace the second EDS having the slice name of the set of EDSs stored within the at least one memory of the computing device.

18. The method of claim 14 further comprising:

processing the data access request involving the set of EDSs associated with the data object such that the set of EDSs associated with the data are distributedly stored among the plurality of SUs that includes n SUs, wherein n is a positive integer greater than or equal to 2, such that a first approximately 1/n EDSs are stored within the first at least one SU and a second approximately 1/n EDSs are stored within the second at least one SU; and

caching within the at least one memory a sufficient number of EDSs stored within the second at least one SU that is remotely located to the computing device and is coupled to the computing device via the external network so that at least one of the decode threshold number of EDSs, the read threshold number of EDSs, or the write threshold number of EDSs may be retrieved from the at least one memory and the first at least one SU.

19. The method of claim 14, wherein the computing device is a SU of the plurality of SUs within the DSN, a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, or a video game device.

20. The method of claim 14, wherein the DSN includes at least one of a wireless communication system, a wire lined communication systems, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).