METHODS AND APPARATUS TO PERFORM FILE TRANSFERS IN DISTRIBUTED FILE SYSTEMS

• US 20080244030A1
• Filed: 03/30/2007
• Published: 10/02/2008
• Est. Priority Date: 03/30/2007
• Status: Active Grant

• Current Assignee: LinkedIn Corporation (Microsoft Corporation)
• Original Assignee: SBC Knowledge Ventures LP (AT&T, Inc.)
• Inventors: Leitheiser, Gregory
• Granted Patent: US 7,865,575 B2
• Time in Patent Office: Days
• US Class Current: 709/211
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: H04L 12/66 Arrangements for connecting between networks having differing types of switching systems, e.g. gateways
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]

Abstract

Methods and apparatus to send and receive files are disclosed. An example method for sending a file includes sending a list of data nodes storing a first portion of the file and a second portion of the file to a master node of a destination distributed file system, computing a first checksum for a first portion of the file, computing a second checksum for a second portion of the file, sending the first checksum and the second checksum to the destination distributed file system, sending the first portion of the file to a first data node of the destination distributed file system, and sending the second portion of the file to a second data node of the destination distributed file system.

Claims

1. A method for sending a file, the method comprising:

sending a list of data nodes storing a first portion of the file and a second portion of the file to a master node of a destination distributed file system;

computing a first checksum for a first portion of the file;

computing a second checksum for a second portion of the file;

sending the first checksum and the second checksum to the destination distributed file system;

sending the first portion of the file to a first data node of the destination distributed file system, wherein the first data node is assigned by the master node of the distributed file system; and

sending the second portion of the file to a second data node of the destination distributed file system, wherein the second data node is assigned by the master node of the distributed file system.

2. A method as defined in claim 1, wherein the data nodes in the list of data nodes are associated with a source distributed file system.

3. A method as defined in claim 1, further comprising receiving a request for a file from the master node of the destination distributed file system.

4. A method as defined in claim 1, wherein the list of data nodes identifies the first data node and the second data node.

5. A method as defined in claim 1, wherein computing the first checksum is performed by the first data node.

6. A method as defined in claim 1, wherein sending the list of data nodes is performed by a master node of a destination distributed file system associated with the data nodes in the list of data nodes.

32. An article of manufacture as defined in claim 31, wherein the machine readable instructions further cause the machine to receive a list of checksums associated with the host machines.

33. An article of manufacture as defined in claim 32, wherein the machine readable instructions further cause the machine to receive a first checksum from the first client.

34. An article of manufacture as defined in claim 33, wherein the machine readable instructions further cause the machine to compare a second checksum identified in the list of checksums and associated with the first host machine to the first checksum.

35. An article of manufacture as defined in claim 34, wherein the machine readable instructions further cause the machine to instruct the first client to retransmit the request for the first portion of the data from the first host machine identified in the list of host machines.

36. An article of manufacture as defined in claim 31, wherein the machine readable instructions further cause the machine to send a file transfer request to the server.

7. (canceled)

8. (canceled)

9. (canceled)

10. (canceled)

11. (canceled)

12. (canceled)

13. (canceled)

14. (canceled)

15. (canceled)

16. (canceled)

17. A system for sending a file, the system comprising:

a first data node to compute a first checksum for a first portion of the file and to send the first portion of the file to a third data node of a destination distributed file system;

a second data node to compute a second checksum for a second portion of the file and to send the second portion of the file to a fourth data node of the destination distributed file system; and

a master node to send a list of data nodes storing the first portion of the file and the second portion of the file to the destination distributed file system and to send the first checksum and the second checksum to the destination distributed file system.

18. A system as defined in claim 17, wherein the first data node and the second data node are associated with a source distributed file system.

19. A system as defined in claim 17, wherein the master node is further to receive a request for a file from the destination distributed file system.

20. A system as defined in claim 17, wherein the list of data nodes identifies the first data node and the second data node.

21. A system as defined in claim 17, further comprising a network interface associated with the destination distributed file system to communicatively couple the master node, the first data node, and the second data node to the destination distributed file system.

22. A system as defined in claim 21, wherein the network interface is to allow the first data node and the second data node to access the third data node and the fourth data node.

23. A system as defined in claim 22, further comprising a fifth data node associated with the destination distributed file system.

24. A system as defined in claim 23, wherein the network interface is to prevent access to the fifth data node by the first data node and the second data node.

25. (canceled)

26. (canceled)

27. (canceled)

28. (canceled)

29. (canceled)

30. (canceled)

31. An article of manufacture storing machine readable instructions which, when executed, cause a machine to:

receive a list of host machines from a server at a master node of a source distributed file system;

instruct a first client to request a first portion of data from a first host machine identified in the list of host machines at the master node; and

instruct a second client different from the first client to request a second portion of the data different from the first portion of the data from a second host machine identified in the list of host machines at the master node.

37. (canceled)

38. (canceled)

39. (canceled)

40. (canceled)