Application recovery from network-induced data corruption

• US 8,620,874 B1
• Filed: 06/20/2011
• Issued: 12/31/2013
• Est. Priority Date: 06/10/2008
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Xavier, Felix, George, Tijin, Varghese, Kevin, Pattabiraman, Jayalakshmi
• Primary Examiner(s): Corrielus, Jean M
• Application Number:
  US13/164,558
• Time in Patent Office: 925 Days
• Field of Search: 707/690, 707/691, 707/697, 707/699, 709/251, 714/748, 714/750, 714/758
• US Class Current: 707/690
• CPC Class Codes: G06F 11/08 Error detection or correction by redundancy in data representation, e.g. by using checking codes
• CPC Class Codes: H04L 1/1809 Selective-repeat protocols

Abstract

A source application reads a body of data in data block sized units and calculates a checksum value for each data block before sending the data block, the calculated checksum value and the identifier. Upon receipt, a destination application independently calculates a checksum value for each received data block and compares the two checksums. Non-matching checksums indicate a network-induced error in the data block. Identifiers for the erroneous data blocks are transmitted to the source application after all of the data blocks have been initially transmitted. The source application thereafter resends only those data blocks identified. The destination application repeats the process of comparing checksums and transmitting identifiers to the source application until all of the data blocks of the body of data have been correctly received, and then uses the data blocks to recreate the body of data.

Claims

1. A method of transferring a set of padded data blocks, comprising:

assembling, by a source computer, the set of padded data blocks, each of the padded data blocks in the set including;

a data block, a data block identifier, and a data block source checksum, wherein the data block source checksum is generated by the source computer;

transmitting, from the source computer to a destination computer, the set of padded data blocks over a communications network;

upon receiving, by the destination computer, all of the set of padded data blocks from the communications network;

generating, by the destination computer, a data block destination checksum for each padded data block in the set of padded data blocks;

extracting, by the destination computer, the data block source checksum from each padded data block generated at the source computer;

comparing, by the destination computer, each data block source checksum with a corresponding data block destination checksum generated at the destination computer;

determining based on the comparing by the destination computer, a data block in the set of received padded data blocks is corrupt;

requesting the source computer to retransmit a subset of the set of padded data blocks containing the corrupt data block identified by the data block identifier after the destination computer has received all of the set of padded data blocks from the source computer; and

retransmitting by the source computer, to the destination computer, the subset of the set of padded data blocks containing the corrupt data block in response to the requesting.

2. A method as defined in claim 1, wherein:

a source application executed on the source computer;

a destination application executes on the destination computer; and

the method further comprising;

generating each data block source checksum for the set of padded data blocks at the source computer by the source application; and

detecting the corruption of the data block of the set of padded data blocks by the destination application.

3. A method as defined in claim 2, further comprising:

receiving, at the destination computer, the retransmitted subset; and

reconstructing the set of received padded data blocks by the destination application.

4. A method as defined in claim 3, wherein an operating system executes on each of the source and destination computers, each operating system implementing a transport protocol for sending the set of padded data blocks between the source and destination computers, the method further comprising:

requesting the operating system of the source computer to transfer the set of padded data blocks to the destination computer by the source application;

receiving the set of padded data blocks by the operating system of the destination computer; and

supplying the received set of padded data blocks to the destination application by the operating system of the destination computer.

5. A method of transferring a body of data from a source computer to a destination computer over a communications network, comprising:

dividing the body of data into a plurality of padded data blocks, each of the plurality of padded data blocks including a data block, a data block identifier and a data block source error detection information;

generating, by the source computer, the data block source error detection information for each padded data block of the plurality of padded data blocks;

sending, by the source computer, the body of data as a plurality of padded data blocks to the destination computer over the communications network;

upon receiving, by the destination computer, the body of data sent by the source computer;

generating, by the destination computer, a data block destination error detection information for each padded data block;

comparing, by the destination computer, the data block source error detection information with the corresponding data block destination error detection information;

determining based on the comparing by the destination computer, a padded data block in the set of received padded data blocks is corrupt;

requesting the source computer resend each padded data block of the body of data determined to have a corrupt data block identified by the data block identifier; and

resending each data block requested to be resent by the source computer to the destination computer over the communications network in response to receiving the request to resend from the destination computer.

6. A method as defined in claim 5, further comprising:

regenerating the data block source error detection information for each data block requested to be resent in response to receiving the request to resend; and

sending the regenerated data block source error detection information along with each data block requested to be resent by the source computer to the destination computer over the communications network in response to receiving the request to resend.

7. A method as defined in claim 5, further comprising:

generating the data block source error detection information for each data block of the set of padded data blocks by calculating a checksum for each data block, the checksum constituting the data block source error detection information.

8. A method as defined in claim 5, further comprising:

utilizing an error detecting transport protocol to send the data blocks and data block source error detection information from the source computer to the destination computer over the communications network and to receive the data blocks and data block source error detection information sent from the source computer by the destination computer; and

generating the data block source error detection information and determining one of the data blocks to be corrupted independently of any error detecting performed by the error detecting transport protocol.

9. A method as defined in claim 8, further comprising:

dividing each padded data block into a plurality of sub-data blocks;

utilizing the error detecting transport protocol to send and receive the padded data blocks by sending each sub-data block of each padded data block separately from the source computer to the destination computer; and

combining the received plurality of the sub-data blocks of each padded data block to reconstruct each padded data block on the destination computer.

10. A method as defined in claim 5, wherein the body of the data is stored on a mass data storage device of the source computer, the mass data storage device comprising a plurality of hard disk drives.

11. A computer system for transferring a set of padded data blocks from a source computer to a destination computer, comprising:

a source computer having a source storage medium upon which a body of data stored;

a destination computer having a destination storage medium;

a communications network communicating data between the source computer and the destination computer;

a set of executable instructions on the source computer operative to cause the source computer to;

divide the body of data into the set of padded data blocks, each of the set of padded data blocks including a data block, a data block identifier and a data block source error detection information for a data block, wherein the data block source error detection information is generated by the source computer;

send the set of padded data blocks to the destination computer over the communications network;

a set of executable instructions on the destination computer operative to cause the destination computer to, upon receipt of all of the set of padded data blocks;

receive the generated data block source error detection information from the source computer;

compare the data block source error detection information with the corresponding data block destination error detection information;

determine based on the comparing by the destination computer, a data block in the set of received padded data blocks is corrupt;

request the source computer to resend each padded data block having a corrupt data block identified by the data block identifier after all of the plurality of padded data blocks have been received; and

the set of executable instructions on the source computer further operative to cause the source computer to;

resend each data block requested to be resent to the destination computer over the communications network in response to receiving the request to resend.

12. A computer system as defined in claim 11, wherein the set of executable instructions on the source computer is further operative to:

update the data block source error detection information for each data block requested to be resent in response to the request to resend; and

send the updated data block source error detection information along with each data block requested to be resent to the destination computer in response to the request to resend.

13. A computer system as defined in claim 11, wherein the set of the executable instructions on the source computer and the set of executable instructions on the destination computer are operative to:

perform the sending, receiving, and determining operations in subsequent data transfers of data blocks in response to requests to resend data blocks determined to have been received corrupted in previous data transfers;

continuing the subsequent data transfers until the destination computer receives all of the set of padded data blocks; and

determine, by the destination computer, that all of the set of the padded data blocks are not corrupted.

14. A computer system as defined in claim 11, wherein:

the sets of executable instructions on the source computer and the destination computer implement a transport protocol for transferring data between the source computer and the destination computer over the communications network, the sending and receiving occurring through the implementation of the transport protocol; and

the implementation of the transport protocol performs error detection of the data transferred between the source and destination computers independently of the generating source and destination error detection information and determining one of the received padded data blocks to have been corrupted.

15. A computer system as defined in claim 11, wherein:

the source computer is a data storage server which supplies data storage services to client computers;

the source storage medium is a mass data storage device; and

the destination computer is a client of the source computer for data storage devices.

16. A computer system as defined in claim 15, wherein the mass data storage device comprises:

a plurality of hard disk drives having at least one rotating magnetic platter on which data is stored.

17. A computer system as defined in claim 11, wherein:

the set of executable instructions on the source computer includes executable instructions which are part of a source computer operating system and also includes other executable instructions which are part of a source computer application;

the set of executable instructions on the destination computer includes executable instructions which are part of a destination computer operating system and also includes other executable instructions which are part of a destination computer application;

the source computer application performs the dividing of the body of data into the plurality of padded data blocks, and supplies the plurality of padded data blocks and the data block source error detection information to the source computer operating system;

both the source computer operating system and the destination computer system implement a transport protocol for sending data between the source and destination computers;

the source computer operating system transfers the plurality of padded data blocks to the destination computer operating system using the implemented transfer protocol; and

the destination computer operating system supplies the received plurality of padded data blocks and the error detection information to the destination computer application.

18. A computer system as defined in claim 17, wherein:

the source computer application provides data storage services to the destination computer application, the destination computer application being a client of the source computer application for the data storage services.

19. A computer system as defined in claim 18, further comprising:

a data storage subsystem connected to the source computer, the source computer writing data to and reading data from the data storage subsystem, the data storage subsystem including the source storage medium.

20. A computer system as defined in claim 17, wherein:

the transport protocol defines a predetermined maximum data payload size for each transfer of data sent between the source computer operating system and the destination computer operating system over the communications network;

the source computer operating system divides each set of padded data blocks received from the source computer application into a plurality of sub-data blocks each having a size less than the maximum data payload size, the source computer operating system subsequently utilizing the transport protocol to send each sub-data block to the destination computer operating system; and

the destination computer operating system receives the sub-data blocks and reconstructs each data block from the sub-data blocks which compose each data block.