Process to migrate named objects to a dispersed or distributed storage network (DSN)

• US 10,229,002 B2
• Filed: 02/14/2017
• Issued: 03/12/2019
• Est. Priority Date: 01/04/2013
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Baptist, Andrew D., Borich, Franco V., Leggette, Wesley B.
• Primary Examiner(s): Kerveros, James C
• Application Number:
  US15/432,642
• Publication Number:
  US 20170153946A1
• Time in Patent Office: 756 Days
• Field of Search: 714763
• CPC Class Codes: G06F 11/1068 in sector programmable memories, e.g. flash disk G06F11/1072 takes precedence
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0652 Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A computing device includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and a processing module operably coupled to the interface and memory such that the processing module, when operable within the computing device based on the operational instructions, is configured to perform various operations. The computing device receives data object information for a data object and stores the data object information in a dispersed index of a dispersed or distributed storage network (DSN). The computing device also dispersed error encodes the data object to generate sets of encoded data slices (EDSs) (e.g., for data segments of the data object) and updates the index entry state of the dispersed index to moving to indicate that the data object is moving. The computing device distributedly stores the sets of EDSs among a storage units (SUs) of the DSN.

Claims

1. A computing device comprising:

an interface configured to interface and communicate with a dispersed or distributed storage network (DSN);

memory that stores operational instructions; and

a processing module operably coupled to the interface and to the memory, wherein the processing module, when operable within the computing device based on the operational instructions, is configured to;

receive, from a data storage system, data object information for a data object that is stored within the data storage system;

store the data object information in a dispersed index of a dispersed or distributed storage network (DSN) that is different than the data storage system including to update an index entry state of the dispersed index to indicate the data object is to be moved from the data storage system to the DSN;

dispersed error encode the data object to generate sets of encoded data slices (EDSs) corresponding to a plurality of data segments of the data object, wherein the data object is segmented into the plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce a set of EDSs of the sets of EDSs;

update the index entry state of the dispersed index to moving to indicate that the data object is moving from the data storage system to the DSN;

distributedly store the sets of EDSs among a plurality of storage units (SUs) of the DSN; and

when storage of a write threshold number of EDSs for the sets of EDSs is confirmed by the plurality of SUs, update the index entry state of the dispersed index to indicate that storage of the data object in the DSN is done.

2. The computing device of claim 1, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to operate in a move mode including to:

generate a delete data object request;

transmit the delete data object request to the data storage system after storage of the write threshold number of EDSs for the sets of EDSs is confirmed by the plurality of SUs; and

when deletion of the data object is confirmed by the data storage system, update the index entry state of the dispersed index to indicate that storage of the data object in the DSN is done and the data object is moved from the data storage system to the DSN.

3. The computing device of claim 1, wherein:

the sets of EDSs are of pillar width,

a decode threshold number of EDSs of the set of EDSs of the sets of EDSs are needed to recover the data segment of the plurality of data segments;

a read threshold number of EDSs of the set of EDSs of the sets of EDSs provides for reconstruction of the data segment of the plurality of data segments; and

the write threshold number of EDSs of the set of EDSs of the sets of EDSs provides for a successful transfer of the set of EDSs of the sets of EDSs from a first at least one location in the DSN to a second at least one location in the DSN.

4. The computing device of claim 1, wherein:

the data storage system is a legacy data storage system with respect to the DSN; and

the data storage system includes a RAID (redundant array of independent disks) system that stores the data object.

5. The computing device of claim 1, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to:

perform storage of a plurality of data objects that includes the data object within the DSN on a batch by batch basis, wherein each batch includes a respective subset of data objects of the plurality of data objects, and wherein each subset of data objects of the plurality of data objects include at least one of a same number of data objects or a same total size.

6. The computing device of claim 1, wherein the computing device is located at a first premises that is remotely located from at least one SU of the plurality of SUs within the DSN.

7. The computing device of claim 1 further comprising:

a SU of the plurality of SUs within the DSN, a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, or a video game device.

8. The computing device of claim 1, wherein the DSN includes at least one of a wireless communication system, a wire lined communication systems, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).

9. A computing device comprising:

an interface configured to interface and communicate with a dispersed or distributed storage network (DSN);

memory that stores operational instructions; and

a processing module operably coupled to the interface and to the memory, wherein the processing module, when operable within the computing device based on the operational instructions, is configured to;

receive, from a data storage system, data object information for a data object that is stored within the data storage system;

store the data object information in a dispersed index of a dispersed or distributed storage network (DSN) that is different than the data storage system including to update an index entry state of the dispersed index to indicate the data object is to be moved from the data storage system to the DSN;

dispersed error encode the data object to generate sets of encoded data slices (EDSs) corresponding to a plurality of data segments of the data object, wherein the data object is segmented into the plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce a set of EDSs of the sets of EDSs;

update the index entry state of the dispersed index to moving to indicate that the data object is moving from the data storage system to the DSN;

distributedly store the sets of EDSs among a plurality of storage units (SUs) of the DSN; and

when operating in a copy mode, when storage of a write threshold number of EDSs for the sets of EDSs is confirmed by the plurality of SUs, update the index entry state of the dispersed index to indicate that storage of the data object in the DSN is done;

when operating in a move mode;

generate a delete data object request;

transmit the delete data object request to the data storage system after storage of the write threshold number of EDSs for the sets of EDSs is confirmed by the plurality of SUs; and

when deletion of the data object is confirmed by the data storage system, update the index entry state of the dispersed index to indicate that storage of the data object in the DSN is done and the data object is moved from the data storage system to the DSN.

10. The computing device of claim 9, wherein:

the sets of EDSs are of pillar width,

a decode threshold number of EDSs of the set of EDSs of the sets of EDSs are needed to recover the data segment of the plurality of data segments;

a read threshold number of EDSs of the set of EDSs of the sets of EDSs provides for reconstruction of the data segment of the plurality of data segments; and

the write threshold number of EDSs of the set of EDSs of the sets of EDSs provides for a successful transfer of the set of EDSs of the sets of EDSs from a first at least one location in the DSN to a second at least one location in the DSN.

11. The computing device of claim 9, wherein:

the data storage system is a legacy data storage system with respect to the DSN; and

the data storage system includes a RAID (redundant array of independent disks) system that stores the data object.

12. The computing device of claim 9 further comprising:

a SU of the plurality of SUs within the DSN, a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, or a video game device.

13. The computing device of claim 9, wherein the DSN includes at least one of a wireless communication system, a wire lined communication systems, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).

14. A method for execution by a computing device, the method comprising:

receiving, from a data storage system and via an interface of the computing device that is configured to interface and communicate with a dispersed or distributed storage network (DSN), data object information for a data object that is stored within the data storage system;

storing the data object information in a dispersed index of a dispersed or distributed storage network (DSN) that is different than the data storage system including to update an index entry state of the dispersed index to indicate the data object is to be moved from the data storage system to the DSN;

dispersed error encoding the data object to generate sets of encoded data slices (EDSs) corresponding to a plurality of data segments of the data object, wherein the data object is segmented into the plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce a set of EDSs of the sets of EDSs;

updating the index entry state of the dispersed index to moving to indicate that the data object is moving from the data storage system to the DSN;

distributedly storing the sets of EDSs among a plurality of storage units (SUs) of the DSN; and

when storage of a write threshold number of EDSs for the sets of EDSs is confirmed by the plurality of SUs, updating the index entry state of the dispersed index to indicate that storage of the data object in the DSN is done.

15. The method of claim 14, when operating in a move mode, further comprising:

generating a delete data object request;

transmitting, via the interface of the computing device, the delete data object request to the data storage system after storage of the write threshold number of EDSs for the sets of EDSs is confirmed by the plurality of SUs; and

when deletion of the data object is confirmed by the data storage system, updating the index entry state of the dispersed index to indicate that storage of the data object in the DSN is done and the data object is moved from the data storage system to the DSN.

16. The method of claim 14, wherein:

the sets of EDSs are of pillar width,

a decode threshold number of EDSs of the set of EDSs of the sets of EDSs are needed to recover the data segment of the plurality of data segments;

a read threshold number of EDSs of the set of EDSs of the sets of EDSs provides for reconstruction of the data segment of the plurality of data segments; and

the write threshold number of EDSs of the set of EDSs of the sets of EDSs provides for a successful transfer of the set of EDSs of the sets of EDSs from a first at least one location in the DSN to a second at least one location in the DSN.

17. The method of claim 14, wherein:

the data storage system is a legacy data storage system with respect to the DSN; and

the data storage system includes a RAID (redundant array of independent disks) system that stores the data object.

18. The method of claim 14 further comprising:

performing storage of a plurality of data objects that includes the data object within the DSN on a batch by batch basis, wherein each batch includes a respective subset of data objects of the plurality of data objects, and wherein each subset of data objects of the plurality of data objects include at least one of a same number of data objects or a same total size.

19. The method of claim 14, wherein the computing device includes a SU of the plurality of SUs within the DSN, a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, or a video game device.

20. The method of claim 14, wherein the DSN includes at least one of a wireless communication system, a wire lined communication systems, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).