Efficient transfer of objects between containers on the same vault

• US 10,379,942 B2
• Filed: 09/27/2017
• Issued: 08/13/2019
• Est. Priority Date: 09/27/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Hegde, Harsha, Kala, Lakshmi Krishna Priya, Leggette, Wesley B., Motwani, Manish, Resch, Jason K., Scholl, Daniel J., Shah, Rohan P., Vedpathak, Yogesh R.
• Primary Examiner(s): Britt, Cynthia
• Application Number:
  US15/717,626
• Publication Number:
  US 20190095277A1
• Time in Patent Office: 685 Days
• CPC Class Codes: G06F 11/1044 with specific ECC/EDC distribution
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A method includes determining to transfer a data object from a first container of a vault to a second container of the vault. The method further includes determining first and second storage parameters for the first and second containers, respectively. When the first storage parameters substantially match the second storage parameters, the method further includes retrieving metadata of the data object from a first container structure associated with the first container. The method further includes modifying the metadata regarding the transferring of the data object from the first container to the second container to produce modified metadata. The method further includes adding the modified metadata to a second container structure associated with the second container such that the data object is accessible via the second container.

Claims

1. A method for execution by a computing device of a dispersed storage network (DSN), the method comprises:

determining to transfer a data object from a first container of a vault to a second container of the vault, wherein a vault is a logical memory structure supported by physical memory of storage units of the DSN, wherein the vault is logically divided into a plurality of containers that includes the first and second containers, wherein the data object is stored in the first container as a plurality of sets of encoded data slices;

determining first and second storage parameters for the first and second containers, respectively, wherein the first storage parameters include a first pillar number, a first decode threshold number, and a first encoding function and the second storage parameters include a second pillar number, a second decode threshold number, and a second encoding function;

when the first storage parameters substantially match the second storage parameters;

retrieving metadata of the data object from a first container structure associated with the first container, wherein the metadata includes one or more of;

permissions for the data object, DSN addresses of the plurality of sets of encoded data slices, and access control to the data object;

modifying the metadata regarding the transferring of the data object from the first container to the second container to produce modified metadata; and

adding the modified metadata to a second container structure associated with the second container such that the data object is accessible via the second container.

2. The method of claim 1 further comprises:

when the transfer is a migration, deleting the metadata of the data object from the first container structure after the modified metadata is added to the second container structure.

3. The method of claim 1 further comprises:

when the transfer is duplication, retaining the metadata of the data object in the first container structure after the modified metadata is added to the second container structure.

4. The method of claim 1, wherein the determination that the first storage parameters substantially match the second storage parameters comprises:

determining that the first pillar number substantially matches the second pillar number;

determining that the first decode threshold number substantially matches the second decode threshold number; and

determining that the first encoding function substantially matches the second encoding function.

5. The method of claim 1 further comprises:

when the first storage parameters does not substantially match the second storage parameters, determining whether the first encoding function substantially matches the second encoding function; and

when the first encoding function does not substantially match the second encoding function, performing the transferring of the data object by;

retrieving the plurality of sets of encoded data slices from the first container;

decoding the plurality of sets of encoded data slices to obtain the data object;

encoding the data object with the second storage parameters to produce a second plurality of sets of encoded data slices; and

storing the second plurality of sets of encoded data slices in the second container.

6. The method of claim 5 further comprises:

when the first encoding function does substantially match the second encoding function, determining whether the first decode threshold number substantially matches the second decode threshold number;

when the first decode threshold number does not substantially match the second decode threshold number, performing the transferring of the data object by;

retrieving the plurality of sets of encoded data slices from the first container;

decoding the plurality of sets of encoded data slices to obtain the data object;

encoding the data object with the second storage parameters to produce the second plurality of sets of encoded data slices; and

storing the second plurality of sets of encoded data slices in the second container.

7. The method of claim 6 further comprises:

when the first decode threshold number does substantially match the second decode threshold number and when the first pillar number does not substantially matches the second pillar number, determining whether the first pillar number is greater than the second pillar number; and

when the first pillar number is greater than the second pillar number, modifying the metadata to include DSN addresses for a plurality of modified sets of encoded data slices of the plurality of sets of encoded data slices, wherein a modified set of encoded data slices of the plurality of modified sets of encoded data slices includes the second pillar number of encoded data slices.

8. The method of claim 7 further comprises:

when the second pillar number is greater than the first pillar number;

generating, based on the first encoding function, one or more new encoded data slices for each set of the plurality of sets of encoded data slices;

storing the one or more new encoded data slices for each set of the plurality of sets of encoded data slices in the second container; and

modifying the metadata to include DSN addresses for the plurality of sets of encoded data slices and for the one or more new encoded data slices for each set of the plurality of sets of encoded data slices.

9. A computing device of a dispersed storage network (DSN), the computing device comprises:

an interface;

memory; and

a processing module operably coupled to the memory and the interface, wherein the processing module is operable to;

determine to transfer a data object from a first container of a vault to a second container of the vault, wherein a vault is a logical memory structure supported by physical memory of storage units of the DSN, wherein the vault is logically divided into a plurality of containers that includes the first and second containers, wherein the data object is stored in the first container as a plurality of sets of encoded data slices;

determine first and second storage parameters for the first and second containers, respectively, wherein the first storage parameters include a first pillar number, a first decode threshold number, and a first encoding function and the second storage parameters include a second pillar number, a second decode threshold number, and a second encoding function;

when the first storage parameters substantially match the second storage parameters;

retrieve metadata of the data object from a first container structure associated with the first container, wherein the metadata includes one or more of;

permissions for the data object, DSN addresses of the plurality of sets of encoded data slices, and access control to the data object;

modify the metadata regarding the transferring of the data object from the first container to the second container to produce modified metadata; and

add the modified metadata to a second container structure associated with the second container such that the data object is accessible via the second container.

10. The computing device of claim 9, wherein the processing module is further operable to:

when the transfer is a migration, delete the metadata of the data object from the first container structure after the modified metadata is added to the second container structure.

11. The computing device of claim 9, wherein the processing module is further operable to:

when the transfer is duplication, retain the metadata of the data object in the first container structure after the modified metadata is added to the second container structure.

12. The computing device of claim 9, wherein the processing module is operable to determine that the first storage parameters substantially match the second storage parameters by:

determining that the first pillar number substantially matches the second pillar number;

determining that the first decode threshold number substantially matches the second decode threshold number; and

determining that the first encoding function substantially matches the second encoding function.

13. The computing device of claim 9, wherein the processing module is further operable to:

when the first storage parameters does not substantially match the second storage parameters, determine whether the first encoding function substantially matches the second encoding function; and

when the first encoding function does not substantially match the second encoding function, perform the transferring of the data object by;

retrieving the plurality of sets of encoded data slices from the first container;

decoding the plurality of sets of encoded data slices to obtain the data object;

encoding the data object with the second storage parameters to produce a second plurality of sets of encoded data slices; and

storing the second plurality of sets of encoded data slices in the second container.

14. The computing device of claim 13, wherein the processing module is further operable to:

when the first encoding function does substantially match the second encoding function, determine whether the first decode threshold number substantially matches the second decode threshold number;

when the first decode threshold number does not substantially match the second decode threshold number, perform the transferring of the data object by;

retrieving the plurality of sets of encoded data slices from the first container;

decoding the plurality of sets of encoded data slices to obtain the data object;

encoding the data object with the second storage parameters to produce the second plurality of sets of encoded data slices; and

storing the second plurality of sets of encoded data slices in the second container.

15. The computing device of claim 14, wherein the processing module is further operable to:

when the first decode threshold number does substantially match the second decode threshold number and when the first pillar number does not substantially matches the second pillar number, determine whether the first pillar number is greater than the second pillar number; and

when the first pillar number is greater than the second pillar number, modify the metadata to include DSN addresses for a plurality of modified sets of encoded data slices of the plurality of sets of encoded data slices, wherein a modified set of encoded data slices of the plurality of modified sets of encoded data slices includes the second pillar number of encoded data slices.

16. The computing device of claim 15, wherein the processing module is further operable to:

when the second pillar number is greater than the first pillar number;

generate, based on the first encoding function, one or more new encoded data slices for each set of the plurality of sets of encoded data slices;

store the one or more new encoded data slices for each set of the plurality of sets of encoded data slices in the second container; and

modify the metadata to include DSN addresses for the plurality of sets of encoded data slices and for the one or more new encoded data slices for each set of the plurality of sets of encoded data slices.