System and method for transferring related data objects in a distributed data storage environment

• US 7,062,541 B1
• Filed: 04/27/2000
• Issued: 06/13/2006
• Est. Priority Date: 04/27/2000
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Cannon, David M., Dawson, Colin S., Haye, Mark A.
• Primary Examiner(s): Lim, Krisna
• Assistant Examiner(s): STRANGE, AARON N
• Application Number:
  US09/563,547
• Time in Patent Office: 2,238 Days
• Field of Search: 709/200, 709/201, 709/202, 709/203, 709/217, 709/219, 709/204, 707/204, 707/4
• US Class Current: 709/219
• CPC Class Codes: G06F 3/0614 Improving the reliability of storage systems
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A system and method for transferring related data objects in a distributed data storage environment allows for data objects belonging to a common group to be transferred while preserving the grouping of the data objects. The data objects can be transferred in an order in which the data objects are stored, rather than in an order of grouping. The data objects are accompanied with attribute information including a unique group identifier when transferred from a source site to a target site. At the target site, the attribute information for each data object is received and analyzed. An import groups table listing the groups of data objects is used to associate each data object with its group as it is received. Data objects of groups for which any members have not been received are also listed in an unresolved objects table. When a group is complete, all data objects of the group are removed from the unresolved objects table. At the end of a data transfer session, all data objects still listed in the unresolved objects table are deleted.

Claims

1. A method for transferring a group of related data objects across a distributed data storage system having a source site and a target site, the method comprising:

assigning a group identifier configured to identify a cross-linked relationship between data objects to a group of data objects at a source site, the data objects of the group of data objects related by a predetermined policy configured to define the required data objects for a complete group of data objects;

locating data objects of the group of data objects dispersed over separate storage devices of the source site;

transferring the group of data objects in the order the data objects are accessed in the storage devices from the source site to a nonvolatile storage device of a target site, including concurrently transferring the group identifier together with each data object of the group of data objects;

tracking the group of data objects at the target site with the group identifier; and

determining whether the data objects of the transferred group of data objects comply with the policy.

2. The method of claim 1, further comprising entering information about each group of data objects within a listing at the target site.

3. The method of claim 2, wherein tracking the group of data objects at the target site further comprises determining whether all members of a group have been received by the target site.

4. The method of claim 2, wherein tracking the group of data objects comprises making an entry within the listing for each group of data objects, the entry including the group identifier and comparing a group identifier transmitted with data object to the entries in the listing to determine membership of the data object within a group of data objects.

5. The method of claim 1, further comprising transferring other data objects not within the group of data objects and transferring all data objects in the order stored on the storage devices, including interspersing the other data objects within the transfer of the data objects of the group of data objects.

6. The method of claim 1, wherein the source site comprises a backup server computer and wherein the group of data objects comprises a base file and one or more sub-files configured to be combined with the base file in order to restore a source file.

7. The method of claim 1, wherein the grouped data objects are interspersed with other data objects on storage volumes of the source site.

8. The method of claim 1, further comprising tracking information about groups of data objects transferred from the source site to the target site with a first listing and tracking which of the data objects are members of groups for which any members of the group have not been received with a second listing.

9. The method of claim 1, wherein transferring a group of data objects comprises copying the group of data objects onto a removable storage media at the source site, mounting the removable storage media onto a storage device of the target site and transferring the group of data objects in the order they appear on the removable storage media into a RAM buffer of the target site.

10. The method of claim 1, wherein transferring a group of data objects from a source site to a target site comprises transferring data objects of the group of data objects from separate storage volumes.

11. A system for transferring a group of data objects across a distributed data storage system having a source site and a target site, the system comprising:

an identifier assignment module located at the source site and configured to assign a group identifier configured to identify a cross-linked relationship between data objects to a group of data objects at a source site, the data objects of the group of data objects related by a predetermined policy configured to define the required data objects for a complete group of data objects;

an object locator module within the source site, the object locator module configured to locate data objects of the group of data objects dispersed over separate storage devices of the source site;

a transmission module located at the source site and configured to transfer the group of data objects in the order the data objects are accessed in the storage devices from the source site to a nonvolatile storage device of a target site, including concurrently transferring the group identifier together with each data object of the group of data objects;

a tracking module located at the target site and configured to track the group of data objects at the target site with the group identifier; and

a policy module configured to determine whether the data objects of the transferred group of data objects comply with the policy.

12. The system of claim 11, further comprising a listing within the target site, the listing configured to receive information about each group of data objects.

13. The system of claim 12, further comprising a comparison module located at the target site, the comparison module configured to compare the group identifier of one data object of the group of data objects against the information about each group of data objects entered within the listing to determine membership in the group of data objects.

14. The system of claim 12, further comprising a database entry module within the target site, the database entry module configured to make an entry within the listing for each grouped data object, the entry indicating the group to which the data object belongs.

15. The system of claim 11, wherein the transmission module is further configured to transfer other data objects not within the group of data objects and to transfer all data objects in the order stored on the storage devices, including interspersing the other data objects within the transfer of the data objects of the group of data objects.

16. The system of claim 11, wherein the source site comprises a backup server computer and wherein the group of data objects comprises a base file and one or more sub-files configured to be combined with the base file in order to restore a source file.

17. The system of claim 11, wherein the grouped data objects are interspersed with other data objects on storage volumes of the source site.

18. The system of claim 11, further comprising a first listing configured to track information about groups of data objects transferred from the source site to the target site and a second listing configured to track which of the data objects are members of groups for which any members of the group have not been received.