Managed peer-to-peer applications, systems and methods for distributed data access and storage

• US 7,587,467 B2
• Filed: 11/13/2004
• Issued: 09/08/2009
• Est. Priority Date: 12/02/1999
• Status: Expired due to Fees

• Current Assignee: Western Digital Technologies Incorporated (Western Digital Corporation)
• Original Assignee: Western Digital Technologies Incorporated (Western Digital Corporation)
• Inventors: Rizal, Dharmarus, Hesselink, Lambertus, Bjornson, Eric S.
• Primary Examiner(s): WINDER, PATRICE L
• Application Number:
  US10/987,610
• Publication Number:
  US 20050144186A1
• Time in Patent Office: 1,760 Days
• Field of Search: 709/214, 709/216, 709/229
• US Class Current: 709/214
• CPC Class Codes: H04L 63/0209 Architectural arrangements, e.g. perimeter networks or demilitarized zones
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

Applications, systems and methods for efficiently accessing and controlling data of devices among multiple computers over a network. Strategic cache management processes are provided to manage the data in cache memory of the storage devices involved. Communication of data over the network may be managed by means of one or more connection servers which may also manage any or all of authentication, authorization, security, encryption and point-to-multipoint communications functionalities. Alternatively, computers may be connected over a wide area network without a connection server, and with or without a VPN. Data transmissions may be managed to minimize bandwidth and may be temporally and/or spatially compressed.

Claims

1. A system for data access among multiple computers connected by a wide area network, comprising:

a local storage device associated with a local computer connectable to the wide area network over a first secure channel through a first firewall element, said first firewall element adapted to protect said local computer from hostile intrusion from the wide area network;

a remote storage device associated with a remote computer connectable to the wide area network over a second secure channel through a second firewall element, said second firewall element adapted to protect said remote computer from hostile intrusion from the wide area network;

an authentication server configured to authenticate said local computer for secure connection with said remote computer, and said remote computer for secure connection with said local computer;

a local cache management application configured to store data from said remote storage device in local cache associated with said local storage device; and

a remote cache management application configured to communicate with said local cache management application, manages file requests from said local cache management application and control a remote reference file for each file accessed by said local cache management application.

2. The system of claim 1, wherein said remote cache management application comprises a remote user module and a remote device module, and wherein said remote device module is configured to manage the file requests from said local cache management application and control the reference file for each file accessed by said local cache management application.

3. The system of claim 2, wherein said remote user module is configured to store data from said local storage device in remote cache associated with said remote storage device, said remote user module configured to request the data from said local storage device through said local cache management application.

4. The system of claim 3, wherein said local cache management application comprises a local user module and a local device module, and wherein said local device module is configured to manage file requests from said remote user module and control a local reference file for each file accessed by said remote cache management application.

5. The system of claim 4, wherein said local user module is configured to request data from said remote device module.

6. The system of claim 5, wherein said local cache management application is configured to control a flow of data from said remote cache management application to said local cache management application.

7. The system of claim 1, wherein said local cache management application is configured to implement standard file system functionalities and interfaces with local applications executed on the local computer.

8. The system of claim 1, wherein said remote cache management application is configured to implement standard file system functionalities and interfaces with applications executed locally on the remote computer.

9. The system of claim 1, wherein said local cache management application is configured to accept a local request for file overhead information, and determine whether said file overhead information is available and valid in said local cache.

10. The system of claim 9, wherein said local cache management application configured to send a request to said remote cache management application for said file overhead information when it is determined that said file overhead information is not available and valid in said local cache.

11. The system of claim 10, wherein said local cache management application is configured to store said file overhead information with a time stamp used to determine an active subscription to said file overhead information by said local computer.

12. The system of claim 11, wherein said remote cache management application is configured to store said time stamp with said file overhead information in said remote cache.

13. The system of claim 11, wherein said remote cache management application is configured to set an active subscription time period.

14. The system of claim 13, wherein said remote cache management application is configured to send any updates of said file overhead information to said local cache management application during said active subscription time period.

15. The system of claim 13, wherein said local cache management application is configured to send any local updates of said file overhead information to said remote cache management application during said active subscription time period.

16. The system of claim 15, wherein said remote cache management application, upon receiving an update of said file overhead information from said local cache management application, is configured to identify other cache management applications associated with computers other than said local computer and said remote computer which are actively subscribed to said file overhead information from said remote cache management application, and to transmit the update of said file overhead information to said other cache management applications.

17. The system of claim 11, wherein said local cache management application is configured to access said file overhead information directly from said local cache in response to any local requests after receiving said file overhead information and for a duration of the active subscription.

18. The system of claim 11, wherein said remote cache management application is configured to listen to file access events from applications associated with said remote computer, and to transmit any changes to said file overhead information affected by said file access events to said local cache management application and any other cache management applications associated with computers other than said local computer and said remote computer which are actively subscribed to said file overhead information from said remote cache management application.

19. The system of claim 11, wherein said local cache management application is configured to accept a local request for file data based on selection from said file overhead information stored in said local cache, and to determine whether said file data is available and valid in said local cache.

20. The system of claim 19, wherein said local cache management application is configured to send a request to said remote cache management application for said file data when it is determined that said file data is not available and valid in said local cache.

21. The system of claim 20, wherein said local cache management application is configured to transmit information to said remote cache management application characterizing one or more versions of said file data stored in said local cache that have been determined to be invalid.

22. The system of claim 20, wherein said remote cache management application is configured to search a remote cache to determine if a reference file for the requested file data exists in said remote cache, and is further configured to create and store said reference file in said remote cache if it is determined that said reference file does not yet exist in said remote cache.

23. The system of claim 22, wherein said remote cache management application is configured to index and store version numbers of file data.

24. The system of claim 23, wherein, upon creating and storing said reference file in said remote cache, said remote cache management application is further configured to identify the created reference file as an initial version.

25. The system of claim 23, wherein said local cache management application is configured to transmit information to said remote cache management application characterizing one or more versions of said file data stored in said local cache that have been determined to be invalid, and, upon receiving said request for said file data and said information characterizing the one or more invalid versions of said file data, said remote cache management application is configured to determine an appropriate transmission to provide said local cache management application with a latest version of said reference file.

26. The system of claim 25, wherein said local cache management application and said remote cache management application are both configured to temporally compress said file data by comparing said file data after changes have been made with file data as it existed prior to said changes and creating a Delta file which contains only said changes.

27. The system of claim 26, wherein said local cache management application and said remote cache management application are both configured to create Inverse Delta files by comparing said file data as it existed prior to said changes with said file data after said changes have been made.

28. The system of claim 27, wherein said information characterizing the one or more invalid versions of said file data is transmitted to said remote cache management application as temporally compressed data.

29. The system of claim 28, wherein said temporally compressed data comprises at least one Delta file or at least one Inverse Delta file.

30. The system of claim 29, wherein said remote cache management application is configured to store and manage said Delta files and said Inverse Delta files.

31. The system of claim 30, wherein, upon receiving said request for said file data and said information characterizing the one or more invalid versions of said file data, said remote cache management application is configured to construct a temporally compressed file, based on a current version of said reference file, any Delta and Inverse Delta files received from said local cache management application, and other Inverse Delta and Delta files stored in said remote cache that pertain to said reference file, and is further configured to transmit the constructed temporally compressed file or said latest version of said reference file to said local cache management application, thereby providing said local cache management application with said latest versions of said reference file or information to construct said latest version of said reference file in said local cache.

32. The system of claim 31, wherein said local cache management application is configured to store said file data or said temporally compressed file with a time stamp to determine an active subscription to said reference file by said local computer.

33. The system of claim 32, wherein said remote cache management application is configured to store said time stamp with said file data or said temporally compressed file in said remote cache.

34. The system of claim 33, wherein said remote cache management application is configured to send any updates of said file data to said local cache management application during an active subscription time period.

35. The system of claim 34, wherein said local cache management application is configured to access said file data directly from said local cache during said active subscription time period.

36. The system of claim 34, wherein said local cache management application is configured to send any local updates of said file data to said remote cache management application during said active subscription time period.

37. The system of claim 36, wherein said remote cache management application, upon receiving an update of said file data from said local cache management application, is configured to identify other cache management applications associated with computers other than said local computer and said remote computer which are actively subscribed to said remote cache management application, and to transmit any changes to said file data resulting from said received update to said other cache management applications.

38. The system of claim 37, wherein said remote cache management application is further configured to transmit said changes to said file data resulting from said received update to said other cache management applications using a point-to-multi-point data transmission process.

39. The system of claim 36, wherein said local updates of said file data are sent as temporally compressed, Delta files.

40. The system of claim 36, wherein said remote cache management application is configured to listen to file access events from applications associated with said remote computer, and to transmit any changes to said file data affected by said file access events to said local cache management application and any other cache management applications associated with computers other than said local computer and said remote computer which are actively subscribed to said file data from said remote cache management application.

41. The system of claim 40, wherein said remote cache management application is further configured to transmit said changes to said file data affected by said file access events to said local cache management application and said other cache management applications using a point-to-multi-point data transmission process.

42. The system of claim 26, wherein said local cache management application and said remote cache management application are both configured to spatially compress said file data, whether or not said file data has been already temporally compressed.

43. The system of claim 27, wherein said remote cache management application is further configured to resolve conflicts presented by two or more versions of said file data, based on said Delta and Inverse Delta files.

44. The system of claim 30, wherein, upon receiving said request for said file data and said information characterizing the one or more invalid versions of said file data, said remote cache management application is configured to instruct said local cache management application to use one of said invalid versions of said file data as a current version of said reference file, when it is determined that said latest version of said reference file contains no changes when compared with said one of said invalid versions of said file data.

45. The system of claim 9, wherein said local cache management application is configured to access said file overhead information directly from said local cache when it is determined that said file information is available and valid.

46. The system of claim 1, wherein said remote cache management application is configured to temporally compress file data by comparing said file data after changes have been made with file data as it existed prior to said changes and creating a Delta file which contains only said changes, and wherein said remote cache management application is configured to transmit said Delta file to said local cache management application.

47. The system of claim 1, wherein said remote cache management application is configured to spatially compress file data.

48. The system of claim 1, further comprising a connection server through which said local cache management application connects with said remote cache management application over the wide area network.

49. The system of claim 48, wherein said connection server is configured to form a first, secure, firewall compliant connection with said local cache management application of the local computer, and a second, secure, firewall compliant connection with said remote cache management application of the remote computer, and is further configured to send communications received from said local cache management application to said remote cache management application while maintaining second firewall compliance, and to send communications received from said remote cache management application to said local cache management application while maintaining first firewall compliance.

50. The system of claim 1, wherein said local and remote cache management applications are connectable over a secure channel.

51. The system of claim 1, wherein said local cache management application is connectable with said remote cache management application over a seamless, firewall-compliant channel.

52. The system of claim 1, wherein said local cache management application and said remote cache management application communicate with one another via encrypted data over the wide area network.