Distributed file sharing system

• US 20030233455A1
• Filed: 06/14/2002
• Published: 12/18/2003
• Est. Priority Date: 06/14/2002
• Status: Abandoned Application

• Current Assignee: Hurricane Electric LLC
• Original Assignee: Hurricane Electric LLC
• Inventors: Nelson, Scott, Leber, Mike
• Application Number:
  US10/170,632
• Publication Number:
  US 20030233455A1
• Time in Patent Office: Days
• US Class Current: 709/226
• CPC Class Codes: G02F 1/133512 Light shielding layers, e.g. black matrix G02F1/136209 takes precedence
• CPC Class Codes: G02F 1/1339 Gaskets; Spacers; Sealing of cells
• CPC Class Codes: G06F 16/1834 implemented based on peer-to-peer networks, e.g. gnutella p2p communication protocols H04L67/104
• CPC Class Codes: H04L 63/123 received data contents, e.g. message integrity
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/101 based on network conditions
• CPC Class Codes: H04L 67/104 Peer-to-peer [P2P] networks
• CPC Class Codes: H04L 67/1063 Discovery through centralising entities
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

A distributed file sharing system and a method for providing fast download of data from multiple data storage mediums. The system combines a network of peer client computers which provide data through optimized peer-to-peer communication links, and a server computer which provides an authentication code for checking the completeness and integrity of the downloaded data. The method involves sending a request for a file to the server computer; receiving back from the server an authentication code and a list of peer client computers that have the requested file or part of it; sending a request for the file to a subset of peer clients that yield the fastest download rate; receiving file data back from this subset of peer clients; reassembling the requested file using data sent by the peer clients; and checking the integrity and completeness of the reconstructed file by comparing a computed checksum of said reconstructed file with the authentication code.

Claims

1. A distributed file sharing system for fast download of data from multiple computer data storage mediums connected by peer-to-peer connections through a computer network comprising, in combination:

a plurality of client computers connected to the computer network;

a plurality of computer data storage mediums connected to a corresponding one of said client computers;

a plurality of client databases, each said client database stored on a corresponding one of said computer data storage mediums, and containing files or parts of files;

a file retrieval software application operational with each one of said client computers, said file retrieval software application comprising a module that establishes communication links with other ones of said client computers;

a module that measures an instantaneous speed of data flow through each one of said communication links and dynamically selects a communication link that yields a highest measured speed of data flow;

a module which communicates with said computer data storage medium and which can retrieve a complete file or file part; and

a module that reassembles parts of a file into a complete file and that performs a predetermined authentication procedure which computes an authentication code of the file;

a server computer connected to the computer network and connectable to each of said client computers;

a server computer storage medium connected to said server computer;

a server database stored on said server computer storage medium, said server database containing said list of files and said lists of locations of client computers associated with each one of said files; and

a file sharing software application operational with said server computer which maintains a list of files that are stored on client databases and a list of locations of client computers that store said files, which also computes an authentication code of a file, and which sends said authentication code and said locations of client computers to a client computer that requests the file.

2. A distributed file sharing system as claimed in claim 1 wherein the computer network is a global computer network such as Internet.

3. A distributed file sharing system as claimed in claim 2 wherein the peer-to-peer connections comprise a TCP/IP protocol over the Internet.

4. A distributed file sharing system as claimed in claim 2 wherein the peer-to-peer connections comprise a UDP protocol.

5. A distributed file sharing system as claimed in claim 2 wherein the peer-to-peer connections comprise a RS-232 connection standard.

6. A distributed file sharing system as claimed in claim 2 wherein the peer-to-peer connections comprise a RS-485 connection standard.

7. A distributed file sharing system as claimed in claim 2 wherein the peer-to-peer connections use a switch phone network.

8. A distributed file sharing system as claimed in claim 3 further comprising a client-server connection which uses a TCP/IP protocol over the Internet.

9. A distributed file sharing system as claimed in claim 3 further comprising a client-server connection which uses a UDP protocol.

10. A distributed file sharing system as claimed in claim 3 further comprising a client-server connection which uses a RS-232 connection standard.

11. A distributed file sharing system as claimed in claim 3 further comprising a client-server connection which uses a RS-485 connection standard.

12. A distributed file sharing system as claimed in claim 3 further comprising a client-server connection which uses a switch phone network.

13. A distributed file sharing system as claimed in claim 8 further comprising a peer-to-peer communication protocol wherein said peer-to-peer communication protocol is a HTTP protocol.

14. A distributed file sharing system as claimed in claim 8 further comprising a peer-to-peer communication protocol wherein said peer-to-peer communication protocol is a FTP protocol.

15. A distributed file sharing system as claimed in claim 13 further comprising a client-to-server communication protocol wherein said client-to-server communication protocol is a customized communication protocol which is optimized for the client-server connection.

16. A distributed file sharing system as claimed in claim 13 further comprising a client-to-server communication protocol wherein said client-to-server communication protocol is a FTP protocol.

17. A distributed file sharing system as claimed in claim 15 wherein said predetermined authentication procedure comprises a MD5 checksum algorithm.

18. A distributed file sharing system as claimed in claim 15 wherein said predetermined authentication procedure comprises a SHA1 checksum algorithm.

19. A distributed file sharing system as claimed in claim 15 wherein said predetermined authentication procedure comprises a CRC checksum algorithm.

20. A distributed file sharing system as claimed in claim 17 wherein said instantaneous speed of data flow is measured by a module which times a responsiveness of a channel during a download.

21. A distributed file sharing system as claimed in claim 17 wherein said instantaneous speed of data flow is measured by a module which determines a number of intervening routers on a path.

22. A distributed file sharing system as claimed in claim 17 wherein said instantaneous speed of data flow is measured by a module which determines a cost as measured by Border Gateway Protocol routing tables.

23. A distributed file sharing system as claimed in claim 17 wherein said instantaneous speed of data flow is determined by a module which uses pre-configured network topology maps.

24. A distributed file sharing system as claimed in claim 20 wherein said server computer is a workstation that runs a UNIX-like operating system.

25. A distributed file sharing system as claimed in claim 20 wherein said server computer is a workstation that runs a Linux operating system.

26. A distributed file sharing system as claimed in claim 24 wherein said server database is a Linux file system.

27. A distributed file sharing system as claimed in claim 24 wherein said server database is a SQL-compatible database.

28. A distributed file sharing system as claimed in claim 24 wherein said server database is a MySQL database.

29. A distributed file sharing system as claimed in claim 24 wherein said server database is a Network File System database.

30. A distributed file sharing system as claimed in claim 24 wherein said server database is a DBASE database.

31. A distributed file sharing system as claimed in claim 26 wherein said client database connected to a client computer is a native file system of an operating system run by said client computer.

32. A distributed file sharing system as claimed in claim 26 wherein said client database conneted to a client computer is a Linux file system.

33. A distributed file sharing system as claimed in claim 26 wherein said client database connected to a client computer is a SQL-compatible database.

34. A distributed file sharing system as claimed in claim 26 wherein said client database connected to a client computer is a MySQL database.

35. A distributed file sharing system as claimed in claim 26 wherein said client database connected to a client computer is a Network File System database.

36. A distributed file sharing system as claimed in claim 26 wherein said client database connected to a client computer is a DBASE database.

37. A method of using a distributed file sharing system for fast download of data from multiple computer data storage mediums connected by peer-to-peer connections through a computer network, said method comprising the steps of:

sending a request for a file from a client computer connected to the computer network to a server computer connected to the computer network;

receiving a response from the server computer through the computer network, said response comprising a list of locations of other client computers that presumably have said file, or a part of said file, and an authentication code corresponding to said file;

sending a query for the file to a pre-selected set of other client computers;

receiving a response from a subset of other client computers that actually have the file or a part of the file;

sending a formal request for the file to the subset of other client computers that actually have the file or a part of the file;

receiving data pertaining to the file or a part of the file from the subset of other client computers that actually have the file or a part of the file;

reassembling the file using the data received from said subset of other client computers;

computing a checksum of a reassembled file and performing a comparison between said checksum and the authentication code received from the server computer;

sending a status report comprising the comparison between the checksum and the authentication code back to the server computer.

38. A method of using a distributed file sharing system as claimed in claim 37 further comprising the steps of measuring an instantaneous speed of data flow from other peer client computers to a requesting client computer and determining a subset of other peer client computers that yield a highest measured speed of data flow.

39. A method of using a distributed file sharing system as claimed in claim 38 wherein the speed of data flow is measured by timing a responsiveness of a channel during a download.

40. A method of using a distributed file sharing system as claimed in claim 38 wherein the speed of data flow is measured by determining a number of intervening routers on a path.

41. A method of using a distributed file sharing system as claimed in claim 38 wherein the speed of data flow is measured by determining a cost as measured by Border Gateway Protocol routing tables.

42. A method of using a distributed file sharing system as claimed in claim 38 wherein the speed of data flow is determined by using pre-configured network topology maps.

43. A method of using a distributed file sharing system as claimed in claim 39 wherein said checksum of the reassembled file is computed using a MD5 checksum algorithm.

44. A method of using a distributed file sharing system as claimed in claim 39 wherein said checksum of the reassembled file is computed using a SHA1 checksum algorithm.

45. A method of using a distributed file sharing system as claimed in claim 39 wherein said checksum of the reassembled file is computed using a CRC checksum algorithm.

46. A method of using a distributed file sharing system as claimed in claim 43 further comprising the step of computing separate checksums for each part of the file received from other peer client computers when the comparison between the checksum of the reassembled file and the authentication code received from the server computer shows a discrepancy.

47. A method of using a distributed file sharing system as claimed in claim 46 further comprising the step of sending back to the server computer the separate checksums computed for each part of the file when the comparison between the checksum of the reassembled file and the authentication code received from the server computer shows a discrepancy.

48. A method of using a distributed file sharing system as claimed in claim 47 further comprising the steps of:

receiving from the server computer an updated list of locations of other client computers that presumably have the file, or a part of the file, when the comparison between the checksum of the reassembled file and the authentication code received from the server computer shows a discrepancy;

using said updated list of locations of other client computers to request and retrieve data pertaining to the file from client computers that actually have the file or a part of the file;

reassembling the file using the data received from the other client computers; and

computing a new checksum of a newly reassembled file and sending said new checksum to the server computer.

49. A method of providing a distributed file sharing system for fast download of data from multiple computer data storage mediums connected by peer-to-peer connections through a computer network, said method comprising the steps of:

receiving a request for a file from a requesting client computer connected to the computer network using a server computer;

sending a query for said file to a database connected to the server computer;

retrieving a list of locations of client computers that have the file or a part of the file and an authentication code from said database;

sending said list of locations of client computers and said authentication code back to the requesting client computer through the computer network.

50. A method of contributing with a serving client computer to a distributed file sharing system for fast download of data from multiple computer data storage mediums connected by peer-to-peer connections through a computer network, said method comprising the steps of:

receiving a request for a file from a requesting peer client computer connected to the computer network;

sending a query for said file to a database connected to the serving client computer;

retrieving information about the file from said database;

sending said information about the file back to the requesting peer client computer;

receiving a formal request from the requesting peer client computer for data from the file;

sending data from the file back to the requesting peer client computer.