REPLICATION OF DATA OBJECTS FROM A SOURCE SERVER TO A TARGET SERVER

• US 20130054523A1
• Filed: 08/30/2011
• Published: 02/28/2013
• Est. Priority Date: 08/30/2011
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Anglin, Matthew J., Cannon, David M., Dawson, Colin S., Fruchtman, Barry, Haye, Mark A., Martin, Howard N.
• Granted Patent: US 9,910,904 B2
• Time in Patent Office: Days
• US Class Current: 707/624
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation

Abstract

Data objects are replicated from a source storage managed by a source server to a target storage managed by a target server. A source list is built of objects at the source server to replicate to the target server. The target server is queried to obtain a target list of objects at the target server. A replication list is built indicating objects on the source list not included on the target list to transfer to the target server. For each object in the replication list, data for the object not already at the target storage is sent to the target server and metadata on the object is sent to the target server to cause the target server to include the metadata in an entry for the object in a target server replication database. An entry for the object is added to a source server replication database.

Claims

1. A computer program product for replicating objects from a source storage managed by a source server to a target storage managed by a target server, the computer program product comprising a computer readable storage medium having computer readable program code embodied therein that executes to cause the source server to perform operations, the operations comprising:

building a source list of objects at the source server to replicate to the target server;

querying the target server to obtain a target list of objects at the target server;

building a replication list indicating objects on the source list not included on the target list to transfer to the target server;

for each object in the replication list, performing;

sending data for the object not already at the target storage to the target server;

sending metadata on the object to the target server to cause the target server to include the metadata in an entry for the object in a target server replication database;

adding an entry for the object to a source server replication database.

2. The computer program product of claim 1, wherein the operations further comprise:

for each object in the replication list, including in the entry for the object in the source server replication database an identifier of the object at the source server and an identifier of the object at the target server.

3. The computer program product of claim 1, wherein the operations further comprise:

for each object on both the source list and the target list, performing;

sending metadata for the object to the target server to cause the target server to include the metadata in an entry for the object in the target server replication database; and

adding an entry for the object to the source server replication database.

4. The computer program product of claim 1, wherein each object is comprised of chunks, wherein the operations performed for each object on the replication list further comprise:

determining chunks in the objects; and

determining a first set of chunks in the object that are not stored at the target storage, wherein sending the data for the data object to the target server comprises sending the determined set of chunks to the target server to store in the target storage; and

determining a second set of chunks in the object that are currently stored at the target storage; and

sending chunk identifiers of chunks in the second set of chunks to the target server.

5. The computer program product of claim 1, wherein the operations further comprise:

querying the target server for an inventory of objects at the target server that satisfy a first criteria;

receiving from the target server the inventory of the objects;

determining objects in the source server matching those listed in the inventory from the target server;

sending metadata for the determined objects matching those listed in the inventory to the target server to cause the target server to include the metadata in an entry for the object in the target server replication database; and

adding an entry for the object to the source server replication database, wherein the built source list includes objects to replicate at the source server that satisfy a second criteria.

6. The computer program product of claim 5, wherein the first criteria comprises a client node originating the objects to replicate and wherein the second criteria comprises at least one of data type and filespace of objects at the client node.

7. The computer program product of claim 1, wherein the operations further comprise:

determining objects to delete that are on the target list but not on the source list; and

communicating with the target server to cause the target server to delete the determined objects to delete from the target server and target storage.

8. The computer program product of claim 1, wherein the source and target servers provide for at least one of separate and independently operating hardware and operating systems;

having the target server provide a hot standby mode for the source server; and

heterogeneous hardware and operating systems.

9. A computer program product for replicating objects from a source storage managed by a source server to a target storage managed by a target server, the computer program product comprising a computer readable storage medium having computer readable program code embodied therein that executes to cause the target server to perform operations, the operations comprising:

receiving a query from the source server for a target list of objects at the target server;

sending the source server the target list of the objects at the target server;

receiving from the source server data for objects to store in the target storage;

receiving from the source server metadata for the data received for the objects to replicate; and

adding an entry to a target server replication database for each object for which data is received including the metadata received for the object.

10. The computer program product of claim 9, wherein the operations further comprise:

for the entry for each object in the target server replication database, including an identifier of the object at the source server and an identifier of the object at the target server.

11. The computer program product of claim 9, wherein the operations further comprise:

receiving metadata from the source server for each object to replicate that is already stored in the target storage; and

adding to the target server replication database an entry for each object to replicate already stored in the target storage including the received metadata for the object.

12. The computer program product of claim 9, wherein each object is comprised of chunks, wherein receiving the data for at least one object comprises receiving only those chunks for the object from the source server that are not already stored in the target storage, wherein the operations further comprise:

receiving from the source server a list of chunks in the object currently stored in the target storage; and

for each object for which less than all chunks in the object were received from the target server, indicating in the entry for the object in the target server replication database links to the chunks in the list of chunks already stored in the target storage.

13. The computer program product of claim 9, wherein the operations further comprise:

receiving from the source server a query for an inventory of objects at the target server that satisfy a first criteria;

sending to the source server the inventory of the objects at the target server that satisfy the first criteria;

receiving metadata for objects at the source server matching those listed in the inventory; and

adding an entry to the target server replication database for each object for which metadata is received for the objects matching those listed in the inventory, wherein the entries include the metadata for the object, wherein the target list includes objects at the target server that satisfy a second criteria.

14. The computer program product of claim 9, wherein multiple source servers are replicated to the target server.

15. The computer program product of claim 9, wherein the operations further comprise:

maintaining a group of objects stored in the target storage; and

indicating the group of objects as incomplete in response to determining that not all the objects in the group have been received at the target server, wherein the group of objects indicated as incomplete are not made available to a client node during a restore operation.

16. The computer program product of claim 15, wherein the operations further comprise:

receiving objects for the group of objects indicated as incomplete; and

indicating the group of objects as complete in response to having received all the objects in the group, wherein objects in the group indicated as complete are available to the client node during a restore operation.

17. A system for replicating objects from a source storage to a target storage managed by a target server, comprising:

a processor; and

a computer readable storage medium having code executed by the processor to perform operations, the operations comprising;

building a source list of objects at the source storage to replicate to the target server;

querying the target server to obtain a target list of objects at the target server;

building a replication list indicating objects on the source list not included on the target list to transfer to the target server;

for each object in the replication list, performing;

sending data for the object not already at the target storage to the target server;

sending metadata on the object to the target server to cause the target server to include the metadata in an entry for the object in a target server replication database;

adding an entry for the object to a source server replication database.

18. The system of claim 17, wherein the operations further comprise:

for each object on both the source list and the target list, performing;

sending metadata for the object to the target server to cause the target server to include the metadata in an entry for the object in the target server replication database; and

adding an entry for the object to the source server replication database.

19. The system of claim 17, wherein each object is comprised of chunks, wherein the operations performed for each object on the replication list further comprise:

determining chunks in the objects; and

determining a first set of chunks in the object that are not stored at the target storage, wherein sending the data for the data object to the target server comprises sending the determined set of chunks to the target server to store in the target storage; and

determining a second set of chunks in the object that are currently stored at the target storage; and

sending chunk identifiers of chunks in the second set of chunks to the target server.

20. The system of claim 17, wherein the operations further comprise:

querying the target server for an inventory of objects at the target server that satisfy a first criteria;

receiving from the target server the inventory of the objects;

determining objects in the source server matching those listed in the inventory from the target server;

sending metadata for the determined objects matching those listed in the inventory to the target server to cause the target server to include the metadata in an entry for the object in the target server replication database; and

adding an entry for the object to the source server replication database, wherein the built source list includes objects to replicate at the source server that satisfy a second criteria.

21. The system of claim 17, wherein the operations further comprise:

determining objects to delete that are on the target list but not on the source list; and

communicating with the target server to cause the target server to delete the determined objects to delete from the target server and target storage.

22. A system for replicating objects from a source storage managed by a source server to a target storage, comprising:

a processor; and

a computer readable storage medium having code executed by the processor to perform operations, the operations comprising;

receiving a query from the source server for a target list of objects at the target server;

sending the source server the target list of the objects at the target server;

receiving from the source server data for objects to store in the target storage;

receiving from the source server metadata for the data received for the objects to replicate; and

adding an entry to a target server replication database for each object for which data is received including the metadata received for the object.

23. The system of claim 22, wherein the operations further comprise:

receiving metadata from the source server for each object to replicate that is already stored in the target storage; and

adding to the target server replication database an entry for each object to replicate already stored in the target storage including the received metadata for the object.

24. The system of claim 22, wherein each object is comprised of chunks, wherein receiving the data for at least one object comprises receiving only those chunks for the object from the source server that are not already stored in the target storage, wherein the operations further comprise:

receiving from the source server a list of chunks in the object currently stored in the target storage; and

for each object for which less than all chunks in the object were received from the target server, indicating in the entry for the object in the target server replication database links to the chunks in the list of chunks already stored in the target storage.

25. The system of claim 22, wherein the operations further comprise:

receiving from the source server a query for an inventory of objects at the target server that satisfy a first criteria;

sending to the source server the inventory of the objects at the target server that satisfy the first criteria;

receiving metadata for objects at the source server matching those listed in the inventory; and

adding an entry to the target server replication database for each object for which metadata is received for the objects matching those listed in the inventory, wherein the entries include the metadata for the object, wherein the target list includes objects at the target server that satisfy a second criteria.

26. The system of claim 22, wherein the operations further comprise:

maintaining a group of objects stored in the target storage; and

indicating the group of objects as incomplete in response to determining that not all the objects in the group have been received at the target server, wherein the group of objects indicated as incomplete are not made available to a client node during a restore operation.

27-36. -36. (canceled)