Defragmenting slices in dispersed storage network memory

• US 9,661,075 B2
• Filed: 07/11/2014
• Issued: 05/23/2017
• Est. Priority Date: 08/29/2013
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Storm, Michael Colin, Resch, Jason K., Baptist, Andrew, Leggette, Wesley
• Primary Examiner(s): Bertram, Ryan
• Application Number:
  US14/328,861
• Publication Number:
  US 20150066879A1
• Time in Patent Office: 1,047 Days
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0614 Improving the reliability of storage systems
• CPC Class Codes: G06F 3/0638 Organizing or formatting or addressing of data
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins by a dispersed storage (DS) processing module receiving access requests, processing data set requests and issuing access responses. The method continues by monitoring slice access requests to generate access records by either storing time stamped access records indicating identities of slices requested by a timestamp or by commonality of slice names. The method continues with determining a correlation of two or more slice access based on the access records when a correlation is greater than a correlation threshold and identifying two or more slices for co-location. The method continues when the two or more slices are not co-located by selecting one or more of the two or more slices for migration to a common memory device.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:

monitoring slice access requests for encoded data slices stored within the DSN to generate access records;

correlating two or more slice access requests based on the access records to produce a correlation value;

when the correlation value is greater than a correlation threshold level, identifying two or more encoded data slices for co-location within the DSN; and

when the two or more encoded data slices for co-location are not stored utilizing a common storage configuration, identifying a fewest number of encoded data slices requiring migration or identifying a greatest number of encoded data slices not requiring migration that are already associated with the common storage configuration and selecting one or more of the two or more encoded data slices for migration to the common storage configuration;

determining the common storage configuration based on the access records; and

migrating the selected one or more encoded data slices to the common storage configuration.

2. The method of claim 1, wherein the monitoring slice access requests includes one or more of:

receiving access requests;

generating timestamp to access records associating identities of the encoded data slices; and

generating timestamp to access records associating identities of timestamps.

3. The method of claim 1, wherein the correlation of two or more slice access requests further comprises determining probability of processing the two or more slice access requests within a given timeframe to produce a correlation value.

4. The method of claim 1, wherein the correlation of two or more slice access requests further comprises determining probability of processing the two or more slice access requests with commonality of slice names to produce a correlation value.

5. The method of claim 1 further comprises the identification of two or more encoded data slices associated with a correlation value greater than a selection threshold.

6. The method of claim 1 further comprises the identifying of a common storage configuration of the greatest number of encoded data slices not requiring migration and determining a new storage configuration to facilitate enough storage capacity for all of the two or more encoded data slices.

7. The method of claim 1 further comprises the common storage configuration including one or more of:

a common memory;

a common vault;

a common stripe; and

a common storage container.

8. The method of claim 7 further comprises:

determining a co-location method within the common memory; and

migrating the selected one or more encoded data slices to the common memory in accordance with the co-location method.

9. The method of claim 8, wherein the co-location method includes one or more of:

anywhere in a memory device;

in adjacent memory sectors;

in similar memory sectors;

in a virtual storage container to facilitate co-reading;

in a common stripe across the common memory device; and

at least one other memory device.

10. The method of claim 1 further comprises updating slice names of the migrated one or more encoded data slices.

11. The method of claim 1, wherein the monitoring memory access includes one or more of:

receiving access requests, processing data set requests and issuing access responses.

12. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:

monitoring memory access;

storing memory access records indicating identities of two or more memory slices requested by slice name;

determining a correlation of the two or more memory slices based on commonality of the slice names; and

when a correlation is greater than a correlation threshold, identifying the two or more memory slices for co-location; and

selecting one or more of the two or more memory slices for migration to a common memory device within the DSN based on identifying a common storage configuration of the greatest number of memory slices not requiring migration and determining a new storage configuration to facilitate enough storage capacity for all of the two or more memory slices;

determining a co-location method within the common memory device; and

migrating the selected one or more memory slices to the common memory device in accordance with the co-location method.

13. The method of claim 12, wherein the monitoring memory access includes one or more of:

receiving access requests;

processing data set requests; and

issuing access responses.

14. The method of claim 12, wherein the co-location method includes one or more of:

anywhere in memory device;

in adjacent memory sectors;

in similar memory sectors;

in a virtual storage container to facilitate co-reading;

in a common stripe across the common memory device; and

at least one other memory device.

15. A dispersed storage (DS) module comprises:

a first module, when operable within a computing device, causes the computing device to;

monitor slice access requests for encoded data slices stored within the DSN to generate access records;

a second module, when operable within the computing device, causes the computing device to;

correlate two or more slice access requests based on the access records to produce a correlation value;

a third module, when operable within the computing device, causes the computing device to;

when the correlation value is greater than a correlation threshold level, identify two or more encoded data slices for co-location; and

a fourth module, when operable within the computing device, causes the computing device to;

when the two or more encoded data slices for co-location are not stored utilizing a common storage configuration, identify a fewest number of encoded data slices requiring migration or identify a greatest number of encoded data slices not requiring migration that are already associated with the common storage configuration and select one or more of the two or more encoded data slices for migration to the common storage configuration, determine the common storage configuration based on the access records, and migrate the selected one or more encoded data slices to the common storage configuration.

16. The dispersed storage (DS) module of claim 15, wherein the correlation of two or more slice access requests further comprises one or more of:

determining probability of processing the two or more slice access requests within a given timeframe to produce a correlation value; and

determining probability of processing the two or more encoded data slice access requests with commonality of slice names to produce a correlation value.