Secure data migration in a dispersed storage network

• US 9,514,132 B2
• Filed: 12/18/2012
• Issued: 12/06/2016
• Est. Priority Date: 01/31/2012
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Resch, Jason K., Leggette, Wesley
• Primary Examiner(s): NGUYEN, PHONG H
• Application Number:
  US13/718,985
• Publication Number:
  US 20130198130A1
• Time in Patent Office: 1,449 Days
• Field of Search: 707/661, 707/667, 707/609
• US Class Current: 1/1
• CPC Class Codes: G06F 11/08 Error detection or correction by redundancy in data representation, e.g. by using checking codes
• CPC Class Codes: G06F 11/1004 to protect a block of data words, e.g. CRC or checksum G06F11/1076 takes precedence; security arrangements for protecting computers or computer systems against unauthorized activity G06F21/00
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 11/1096 Parity calculation or recalculation after configuration or reconfiguration of the system
• CPC Class Codes: G06F 16/119 Details of migration of file systems migration mechanisms in storage systems G06F3/0647
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/23 Updating
• CPC Class Codes: G06F 21/602 Providing cryptographic facilities or services
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins by a dispersed storage (DS) processing module identifying a candidate dispersed storage (DS) unit of a dispersed storage network (DSN). In response to a data migration request, the method continues with the DS processing module receiving a migration receptiveness message from the candidate DS unit. In response to a data migration reference message, the method continues with the DS processing module receiving, from another DS unit of a set of DS units, a migration reference response that includes an indication of storage utilized by the other DS unit. The method continues with the DS processing module determining a storage utilization of the candidate DS unit based on the migration reference response and when the determined storage utilization compares favorably to the reported storage utilization, enabling transfer of data from a DS unit of another set of DS units to the candidate DS unit.

Claims

1. A method for secure data migration within a dispersed storage network (DSN), the method comprises:

identifying a candidate dispersed storage (DS) unit of the DSN, wherein data objects are divided into data partitions in accordance with a data partitioning protocol and the data objects are stored as sets of data partitions in a set of DS units that includes the candidate DS unit;

in response to a data migration request, receiving a migration receptiveness message from the candidate DS unit, wherein the migration receptiveness message includes an indication of availability to receive migrated data and a reported storage utilization for storing a first sub-set of data partitions of the sets of data partitions;

in response to a data migration reference message, receiving, from another DS unit of the set of DS units, a migration reference response that includes an indication of storage utilized by the other DS unit for storing a second sub-set of data partitions of the sets of data partitions;

determining a storage utilization of the candidate DS unit for storing the first sub-set of data partitions based on the migration reference response and the data partitioning protocol; and

when the determined storage utilization compares favorably to the reported storage utilization and the indication of availability to receive migrated data is favorable, enabling transfer of data partitions of other data objects from a DS unit of another set of DS units to the candidate DS unit, wherein determining that the indication of availability to receive migrated data is favorable by;

ascertaining storage capabilities of the candidate DS unit;

determining other storage obligations of the candidate DS unit;

determining available storage of the candidate DS unit based on the storage capabilities, the other storage obligations, and the reported storage utilization; and

when the available storage compares favorably to a storage availability threshold, determining that the indication of availability to receive migrated data is favorable.

2. The method of claim 1, wherein the identifying the candidate DS unit comprises:

identifying a storage trait of the candidate DS unit that is also a storage trait of the DS unit of the other set of DS units to yield a common storage trait, wherein the common storage trait includes one or more of a common site identifier, a common pillar number in a dispersed storage error encoding system, a reliability requirement of the DSN, access availability, and access speed.

3. The method of claim 1 further comprises:

the data partitioning protocol includes a distributed storage error encoding function, wherein the data objects are encoded in accordance with the distributed storage error encoding function to produce sets of encoded data slices, wherein, from set to set, size of the encoded data slices is substantially the same; and

generating a message indicating that the candidate DS unit may be compromised when the reported storage utilization does not substantially match the storage utilized by the other DS unit.

4. The method of claim 1, wherein the determining the storage utilization of the candidate DS unit comprises:

determining a storage utilization ratio between the candidate DS unit and the other DS unit based on the data partitioning protocol; and

multiplying the storage utilization ratio by the storage utilized by the other DS unit to determine the storage utilization by the candidate DS unit.

5. The method of claim 1 further comprises:

receiving, from multiple DS units of the set of DS units, multiple migration reference responses; and

determining the storage utilization of the candidate DS unit for storing the first sub-set of data partitions based on the migration reference response, the multiple migration reference responses, and the data partitioning protocol.

6. The method of claim 5 further comprises:

the determining the storage utilization of the candidate DS unit including;

for each of the multiple DS units and the other DS unit, determining a storage utilization ratio between the candidate DS unit and the other DS unit based on the data partitioning protocol to produce multiple storage utilization ratios;

for each of the multiple storage utilization ratios, multiplying the storage utilization ratio by the storage utilized by a corresponding one of the multiple DS units and the other DS unit to determine multiple storage utilizations by the candidate DS unit; and

when the multiple storage utilizations substantially match, selecting one of the multiple storage utilizations as the determined storage utilization; and

when the multiple storage utilizations do not substantially match, determining that the determined storage utilization compares unfavorably to the reported storage utilization.

7. The method of claim 1 further comprises:

updating address ranges of the candidate DS unit and the DS unit of the other set of DS units after the transfer is completed.

8. The method of claim 1 further comprises:

when the determined storage utilization compares unfavorably to the reported storage utilization;

identifying another candidate DS unit from yet another set of DS units; and

generating a message indicating that the candidate DS unit may have been compromised.