Efficient migration to distributed storage
First Claim
Patent Images
1. A method for migrating objects to a distributed storage system, comprising:
- determining, by a migration service implemented at a target storage of the distributed storage system, a list of objects within a source storage to migrate to the target storage;
performing pull migration, by the migration service of the target storage against the source storage, for objects in the list of objects, the pull migration comprising;
querying the source storage to determine relationships among the list of objects to migrate;
generating a chunk layout for the objects to migrate based on the relationships, the generating including placing related objects, responsive to the querying, within contiguous chunk segments and/or within contiguous chunks; and
for each unencoded chunk within the chunk layout;
retrieving objects from source storage specified by the unencoded chunk within the chunk layout;
generating, by a chunk encoding service at the target storage, data and coded fragments for the unencoded chunk using the retrieved objects; and
storing the data and coded fragments to primary storage at the target storage.
7 Assignments
0 Petitions
Accused Products
Abstract
A computer program product, system, and method for determining a list of objects, within source storage, to migrate; generating a chunk layout for the objects to migrate; and for each unencoded chunk within the chunk layout: retrieving objects from source storage specified by the unencoded chunk within the chunk layout; generating data and coded fragments for the unencoded chunk using the retrieved objects; and storing the data and coded fragments to primary storage.
-
Citations
18 Claims
-
1. A method for migrating objects to a distributed storage system, comprising:
-
determining, by a migration service implemented at a target storage of the distributed storage system, a list of objects within a source storage to migrate to the target storage; performing pull migration, by the migration service of the target storage against the source storage, for objects in the list of objects, the pull migration comprising; querying the source storage to determine relationships among the list of objects to migrate; generating a chunk layout for the objects to migrate based on the relationships, the generating including placing related objects, responsive to the querying, within contiguous chunk segments and/or within contiguous chunks; and for each unencoded chunk within the chunk layout; retrieving objects from source storage specified by the unencoded chunk within the chunk layout; generating, by a chunk encoding service at the target storage, data and coded fragments for the unencoded chunk using the retrieved objects; and storing the data and coded fragments to primary storage at the target storage. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A system for migrating objects to a distributed storage system, comprising:
-
a processor at a target storage of the distributed storage system; a volatile memory; and a non-volatile memory storing computer program code that when executed on the processor causes the processor to execute a process operable to perform the operations of; determining, by a migration service implemented at the target storage, a list of objects within a source storage to migrate to the target storage; performing pull migration, by the migration service of the target storage against the source storage, for objects in the list of objects, the pull migration comprising; querying the source storage to determine relationships among the list of objects to migrate; generating a chunk layout for the objects to migrate based on the relationships, the generating including placing related objects, responsive to the querying, within contiguous chunk segments and/or within contiguous chunks; and for each unencoded chunk within the chunk layout; retrieving objects from source storage specified by the unencoded chunk within the chunk layout; generating, by a chunk encoding service at the target storage, data and coded fragments for the unencoded chunk using the retrieved objects; and storing the data and coded fragments to primary storage at the target storage. - View Dependent Claims (14, 15)
-
-
16. A computer program product tangibly embodied in a non-transitory computer-readable medium, the computer-readable medium storing program instructions for migrating objects to a distributed storage system, the instructions are executable to:
-
determine, by a migration service implemented at a target storage of the distributed storage system, a list of objects within a source storage to migrate to the target storage; perform pull migration, by the migration service of the target storage against the source storage, for objects in the list of objects, the pull migration comprising; querying the source storage to determine relationships among the list of objects to migrate; generating a chunk layout for the objects to migrate based on the relationships, the generating including placing related objects, responsive to the querying, within contiguous chunk segment and/or within contiguous chunks; and for each unencoded chunk within the chunk layout; retrieving objects from source storage specified by the unencoded chunk within the chunk layout; generate, by a chunk encoding service at the target storage, data and coded fragments for the unencoded chunk using the retrieved objects; and storing the data and coded fragments to primary storage at the target storage. - View Dependent Claims (17, 18)
-
Specification