Efficiency of file synchronization in a linear tape file system

• US 9,372,868 B2
• Filed: 12/04/2013
• Issued: 06/21/2016
• Est. Priority Date: 12/04/2013
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Ashida, Takashi, Hasegawa, Tohru, Itagaki, Hiroshi, Mitsuma, Shinsuke, Watanabe, Terue
• Primary Examiner(s): To, Baoquoc N
• Assistant Examiner(s): Allen, Nicholas
• Application Number:
  US14/096,094
• Publication Number:
  US 20150154222A1
• Time in Patent Office: 930 Days
• Field of Search: 707/610
• US Class Current: 1/1
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity

Abstract

A mechanism is provided for improved file synchronization of a magnetic tape. A destination device determines whether one or more file checksums exist in an extended file attribute associated with an identified file. Responsive to one or more file checksums existing in the extended file attribute associated with the identified file, the destination device reads the one or more file checksums out of the extended file attribute without accessing the magnetic tape. The destination device transmits the one or more file checksums to a source device. The source device segments the data of the new file into fixed-length blocks that do not overlap one another. The source device calculates one or more new file checksums for each of the fixed-length blocks. The source device transmits the one or more new file checksums to the destination device.

Claims

1. A method, in a data processing system, for improved file synchronization of a magnetic tape, the method comprising:

determining, by a destination device, whether one or more file checksums exist in an extended file attribute associated with an identified file, wherein the extended file attribute of the identified file is in an index associated with the magnetic tape and wherein the index is in a local data structure associated with the destination device;

responsive to one or more file checksums existing in the extended file attribute associated with the identified file, reading, by the destination device, the one or more file checksums out of the extended file attribute without accessing the magnetic tape;

transmitting, by the destination device, the one or more file checksums to a source device;

upon receiving the one or more file checksums, setting, by the source device, a value of an offset variable and a value of a disagree_length variable to 0;

determining, by the source device, whether a sum of the value of the offset variable, the value of the disagree_length variable, and a predefined value of a block_length is less than or equal to a size of a modified version of the identified file thereby forming a new file, wherein the predefined value of the block_length is a length of a fixed-length block to calculate checksum;

responsive to the determining that the sum of the value of the offset variable, the value of the disagree_length variable, and the predefined value of the block_length is greater than the size of the new file, transmitting, by the source device, the data equivalent to the difference of the size of the new file minus the offset from the value of the offset as a literal instruction to the destination device;

segmenting, by the source device, data of the new file into fixed-length blocks that do not overlap one another;

calculating, by the source device, one or more new file checksums for each of the fixed-length blocks;

transmitting, by the source device, the one or more new file checksums to the destination device; and

storing, by the destination device, the one or more new file checksums in the extended file attributes associated with the new file in the index associated with the magnetic tape in the local data structure.

2. The method of claim 1, further comprising:

updating, by the destination device, one or more existing extents or adding, by the destination device, one or more new extents, wherein the one or more existing extents and the one or more new extents are in the index in the local data structure associated with the destination device; and

writing, by the destination device, to the magnetic tape utilizing the updated one or more existing extents or the one or more new extents.

3. The method of claim 2, wherein functions to update the one or more existing extents or add the one or more new extents is provided by a linear tape file system.

4. The method of claim 1, further comprising:

responsive to the one or more file checksums failing to exist in the extended file attribute associated with the file in the index in the local data structure associated with the destination device, reading out, by the destination device, all data of the file from the magnetic tape;

segmenting, by a destination device, the data of the file from the magnetic tape into fixed-length blocks that do not overlap one another;

calculating, by the destination device, one or more file checksums for each of the fixed-length blocks; and

transmitting, by the destination device, the one or more new checksums to the source device.

5. The method of claim 1, further comprising:

storing, by the destination device, the one or more new file checksums received from the source device in the extended file attribute as associated with the file in the index in the local data structure associated with the destination device.

6. The method of claim 1, further comprising:

responsive to an indication to dismount the magnetic tape, writing, by a linear tape file system, contents of the index from the local data structure to an index on the magnetic tape.

7. The method of claim 6, wherein the linear tape file system writes to the magnetic tape at a file-system level.

8. A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to:

determine, by a destination device, whether one or more file checksums exist in an extended file attribute associated with an identified file, wherein the extended file attribute of the identified file is in an index associated with the magnetic tape and wherein the index is in a local data structure associated with the destination device;

responsive to one or more file checksums existing in the extended file attribute associated with the identified file, read, by the destination device, the one or more file checksums out of the extended file attribute without accessing the magnetic tape;

transmit, by the destination device, the one or more file checksums to a source device;

upon receiving the one or more file checksums, setting, by the source device, a value of an offset variable and a value of a disagree_length variable to 0;

determining, by the source device, whether a sum of the value of the offset variable, the value of the disagree_length variable, and a predefined value of a block_length is less than or equal to a size of a modified version of the identified file thereby forming a new file, wherein the predefined value of the block_length is a length of a fixed-length block to calculate checksum;

responsive to the determining that the sum of the value of the offset variable, the value of the disagree_length variable, and the predefined value of the block_length is greater than the size of the new file, transmitting, by the source device, the data equivalent to the difference of the size of the new file minus the offset from the value of the offset as a literal instruction to the destination device;

segment, by the source device, data of the new file into fixed-length blocks that do not overlap one another;

calculate, by the source device, one or more new file checksums for each of the fixed-length blocks;

transmit, by the source device, the one or more new file checksums to the destination device; and

storing, by the destination device, the one or more new file checksums in the extended file attributes associated with the new file in the index associated with the magnetic tape in the local data structure.

9. The computer program product of claim 8, wherein the computer readable program further causes the computing device to:

update, by the destination device, one or more existing extents or add, by the destination device, one or more new extents, wherein the one or more existing extents and the one or more new extents are in the index in the local data structure associated with the destination device; and

write, by the destination device, to the magnetic tape utilizing the updated one or more existing extents or the one or more new extents.

10. The computer program product of claim 9, wherein functions to update the one or more existing extents or add the one or more new extents is provided by a linear tape file system.

11. The computer program product of claim 8, wherein the computer readable program further causes the computing device to:

responsive to the one or more file checksums failing to exist in the extended file attribute associated with the file in the index in the local data structure associated with the destination device, read out, by the destination device, all data of the file from the magnetic tape;

segment, by a destination device, the data of the file from the magnetic tape into fixed-length blocks that do not overlap one another;

calculate, by the destination device, one or more file checksums for each of the fixed-length blocks; and

transmit, by the destination device, the one or more new checksums to the source device.

12. The computer program product of claim 8, wherein the computer readable program further causes the computing device to:

store, by the destination device, the one or more new file checksums received from the source device in the extended file attribute as associated with the file in the index in the local data structure associated with the destination device.

13. The computer program product of claim 8, wherein the computer readable program further causes the computing device to:

responsive to an indication to dismount the magnetic tape, write, by a linear tape file system, contents of the index from the local data structure to an index on the magnetic tape.

14. The computer program product of claim 13, wherein the linear tape file system writes to the magnetic tape at a file-system level.

15. An apparatus comprising:

a destination device and a source device, wherein each of the destination device and the source device comprise;

a processor; and

a memory coupled to the processor,

wherein the memories comprise instructions which, when executed by the processors, cause the processors to;

determine, by the destination device, whether one or more file checksums exist in an extended file attribute associated with an identified file, wherein the extended file attribute of the identified file is in an index associated with the magnetic tape and wherein the index is in a local data structure associated with the destination device;

responsive to one or more file checksums existing in the extended file attribute associated with the identified file, read, by the destination device, the one or more file checksums out of the extended file attribute without accessing the magnetic tape;

transmit, by the destination device, the one or more file checksums to the source device;

upon receiving the one or more file checksums, setting, by the source device, a value of an offset variable and a value of a disagree_length variable to 0;

determining, by the source device, whether a sum of the value of the offset variable, the value of the disagree_length variable, and a predefined value of a block_length is less than or equal to a size of a modified version of the identified file thereby forming a new file, wherein the predefined value of the block_length is a length of a fixed-length block to calculate checksum;

responsive to the determining that the sum of the value of the offset variable, the value of the disagree_length variable, and the predefined value of the block_length is greater than the size of the new file, transmitting, by the source device, the data equivalent to the difference of the size of the new file minus the offset from the value of the offset as a literal instruction to the destination device;

segmenting, by the source device, data of the new file into fixed-length blocks that do not overlap one another;

calculating, by the source device, one or more new file checksums for each of the fixed-length blocks;

transmitting, by the source device, the one or more new file checksums to the destination device; and

storing, by the destination device, the one or more new file checksums in the extended file attributes associated with the new file in the index associated with the magnetic tape in the local data structure.

16. The apparatus of claim 15, wherein the instructions further cause the processors to:

update, by the destination device, one or more existing extents or add, by the destination device, one or more new extents, wherein the one or more existing extents and the one or more new extents are in the index in the local data structure associated with the destination device; and

write, by the destination device, to the magnetic tape utilizing the updated one or more existing extents or the one or more new extents.

17. The apparatus of claim 16, wherein functions to update the one or more existing extents or add the one or more new extents is provided by a linear tape file system.

18. The apparatus of claim 15, wherein the instructions further cause the processors to:

responsive to the one or more file checksums failing to exist in the extended file attribute associated with the file in the index in the local data structure associated with the destination device, read out, by the destination device, all data of the file from the magnetic tape;

segment, by a destination device, the data of the file from the magnetic tape into fixed-length blocks that do not overlap one another;

calculate, by the destination device, one or more file checksums for each of the fixed-length blocks; and

transmit, by the destination device, the one or more new checksums to the source device.

19. The apparatus of claim 15, wherein the instructions further cause the processors to:

store, by the destination device, the one or more new file checksums received from the source device in the extended file attribute as associated with the file in the index in the local data structure associated with the destination device.

20. The apparatus of claim 15, wherein the instructions further cause the processors to:

responsive to an indication to dismount the magnetic tape, write, by a linear tape file system, contents of the index from the local data structure to an index on the magnetic tape, wherein the linear tape file system writes to the magnetic tape at a file-system level.