Managed peer-to-peer applications, systems and methods for distributed data access and storage

• US 20050144186A1
• Filed: 11/13/2004
• Published: 06/30/2005
• Est. Priority Date: 12/02/1999
• Status: Active Grant

• Current Assignee: Western Digital Technologies Incorporated (Western Digital Corporation)
• Original Assignee: Western Digital Technologies Incorporated (Western Digital Corporation)
• Inventors: Rizal, Dharmarus, Hesselink, Lambertus, Bjornson, Eric S.
• Granted Patent: US 7,587,467 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: H04L 63/0209 Architectural arrangements, e.g. perimeter networks or demilitarized zones
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

Applications, systems and methods for efficiently accessing and controlling data of devices among multiple computers over a network. Strategic cache management processes are provided to manage the data in cache memory of the storage devices involved. Communication of data over the network may be managed by means of one or more connection servers which may also manage any or all of authentication, authorization, security, encryption and point-to-multipoint communications functionalities. Alternatively, computers may be connected over a wide area network without a connection server, and with or without a VPN. Data transmissions may be managed to minimize bandwidth and may be temporally and/or spatially compressed.

Claims

1. A system for efficient data access and control of devices among multiple computers connected by a wide area network, comprising:

a local storage device associated with a local computer connectable to the wide area network over a first secure channel through a first firewall element, said first firewall element adapted to protect said first computer from hostile intrusion from the wide area network;

a remote storage device associated with a remote computer connectable to the wide area network over a second secure channel through a second firewall element, said second firewall element adapted to protect said second computer from hostile intrusion from the wide area network;

means for authenticating at least one of said first and second computers for secure connection with the other of said first and second computers;

a local cache management application that stores data from said remote storage device in local cache associated with said local storage device; and

a remote cache management application that communicates with said local cache management application, manages file requests from said local cache management application and controls a reference file for each file accessed by said local cache management application.

2. The system of claim 1, wherein said remote cache management application comprises a user module and a device module, and wherein said device module manages file requests from said local cache management application and controls a reference file for each file accessed by said local cache management application.

3. The system of claim 2, wherein said user module stores data from said local storage device in cache associated with said remote storage device, said user module configured to request data from said local storage device through said local cache management application.

4. The system of claim 3, wherein said local cache management application comprises a user module and a device module, and wherein said device module of said local cache management application manages file requests from said user module of said remote cache management application and controls a reference file for each file accessed by said remote cache management application.

5. The system of claim 4, wherein said user module of said local cache management application requests data from said device module of said remote cache management application.

6. The system of claim 1, wherein said local cache management application implements standard file system functionalities and interfaces with local applications operated by the local computer.

7. The system of claim 1, wherein said remote cache management application implements standard file system functionalities and interfaces with applications operated locally by the remote computer.

8. The system of claim 1, wherein said local cache management application includes means for accepting a local request for file overhead information, and means for determining whether said file overhead information is available and valid in said local cache.

9. The system of claim 8, wherein said local cache management application comprises means for sending a request to said remote cache management application for said file overhead information when it is determined by said means for determining that said file information is not available and valid in said local cache.

10. The system of claim 9, wherein said local cache management application stores said file overhead information with a time stamp used to determine an active subscription period to said file overhead information by said local computer.

11. The system of claim 10, wherein said remote cache management also stores said time stamp with said file overhead information in said remote cache.

12. The system of claim 10, further comprising means for setting an active subscription time period.

13. The system of claim 12, wherein said remote cache management application comprises means for sending any updates of said file overhead information to said local cache management application during said active subscription time period.

14. The system of claim 8, wherein said local cache management application accesses said file overhead information directly from said local cache when it is determined that said file information is available and valid.

15. The system of claim 10, wherein said local cache management application accesses any local requests to said file overhead information directly from said local cache after receiving said file overhead information and for a duration of the active subscription.

16. The system of claim 12, wherein said local cache management application comprises means for sending any local updates of said file overhead information to said remote cache management application during said active subscription time period.

17. The system of claim 16, wherein said remote cache management application, upon receiving an update of said file overhead information from said local cache management application, comprises means for identifying other cache management applications associated with computers other than said local computer and said remote computer and which are actively subscribed to said remote cache management application, and transmitting any changes to said file overhead information to said other cache management applications.

18. The system of claim 10, wherein said remote cache management application comprises means for listening to file access events from applications associated with said remote computer, and transmitting any changes to said file overhead information affected by said file access events to said local cache management application and any other cache management applications associated with computers other than said local computer and said remote computer which are actively subscribed to said file overhead information from said remote cache management application.

19. The system of claim 10, wherein said local cache management application further comprises means for accepting a local request for file data based on selection from said available and valid file overhead information stored in said local cache, and means for determining whether said file data is available and valid in said local cache.

20. The system of claim 19, wherein said local cache management application comprises means for sending a request to said remote cache management application for said file data when it is determined by said means for determining that said file data is not available and valid in said local cache.

21. The system of claim 20, wherein said local cache management further comprises means for transmitting information to said remote cache management characterizing one or more versions of said file data stored in said local cache that have been determined to be invalid.

22. The system of claim 20, wherein said remote cache management application further comprises means for searching said remote cache to determine if a reference file for the requested file data.exists in said remote cache;

and means for creating and storing said reference file in said remote cache if it is determined that said reference file does not yet exist in said remote cache.

23. The system of claim 22, wherein said remote cache management comprises means for indexing and storing version numbers of file data.

24. The system of claim 23, wherein, upon creating and storing said reference file in said remote cache, said means for indexing identifies the created reference file as an initial version.

25. The system of claim 1, further comprising means for temporally compressing file data, wherein said means for temporally compressing compares said file data after changes have been made with said file data as it existed prior to said changes and creates a Delta file which contains only said changes, wherein only said Delta file is transmitted between said local and remote cache management systems to update said file data to reduce transmission bandwidth requirements.

26. The system of claim 1, further comprising means for spatially compressing file data.

27. The system of claim 23, wherein said local cache management further comprises means for transmitting information to said remote cache management characterizing one or more versions of said file data stored in said local cache that have been determined to be invalid;

and wherein upon receiving said request and any information characterizing one or more invalid versions of said file data, said remote cache management application determines an appropriate transmission to provide said local cache management application with a latest version of said reference file.

28. The system of claim 27, wherein said local cache management application and said remote cache management application both comprise means for temporally compressing said file data, wherein said means for temporally compressing compares said file data after changes have been made with said file data as it existed prior to said changes and creates a Delta file which contains only said changes.

29. The system of claim 28, wherein said local cache management application and said remote cache management application both comprise means for creating Inverse Delta files, wherein an Inverse Delta file characterizes changes determined by comparing said file data as it existed before changes have been made with said file data as it exists after said changes have been made.

30. The system of claim 29, wherein said information characterizing one or more versions of said file data stored in said local cache that have been determined to be invalid is transmitted to said remote cache management application in the form of temporally compressed data.

31. The system of claim 30, wherein said temporally compressed data consists of at least one Delta file, at least one Inverse Delta file, or a combination of said Delta and Inverse Delta files.

32. The system of claim 31, wherein said remote cache management application comprises means for storing and managing said Delta files and said Inverse Delta files.

33. The system of claim 32, wherein, upon receiving said request and said information characterizing invalid data files, said remote cache management application contains means for constructing a temporally compressed file, based on a current version of said reference file, any Delta and Inverse Delta files received from said local cache management application, and other Inverse Delta and Delta files stored in said remote cache that pertain to said reference file, and means for transmitting the constructed temporally compressed file to said local cache management application or said current version of said reference file, whichever optimizes data transmission, thereby providing said local cache management application with said current versions of said reference file or information to construct said current version of said reference file in said local cache.

34. The system of claim 32, wherein, upon receiving said request and said information characterizing invalid data files, said remote cache management application instructs said local cache management application to use one of said invalid data files as a current version of said reference file, when it is determined that said current reference file contains no changes when compared with said one of said invalid data files.

35. The system of claim 28, wherein said local cache management application and said remote cache management application both comprise means for spatially compressing said file data, whether or not said file data has been already temporally compressed.

36. The system of claim 33, wherein said local cache management application stores said file data or file data information with a time stamp used to determine an active subscription period to said current reference file by said local computer.

37. The system of claim 36, wherein said remote cache management application also stores said time stamp with said file data or file data information in said remote cache.

38. The system of claim 37, wherein said remote cache management application comprises means for sending any updates of said file data to said local cache management application during an active subscription time period.

39. The system of claim 38, wherein said local cache management application accesses said file data directly from said local cache during said active subscription time period.

40. The system of claim 38, wherein said local cache management application comprises means for sending any local updates of said file data to said remote cache management application during said active subscription time period.

41. The system of claim 40, wherein said remote cache management application, upon receiving an update of said file data from said local cache management application, comprises means for identifying other cache management applications associated with computers other than said local computer and said remote computer and which are actively subscribed to said remote cache management application, and transmitting any changes to said file data resultant from said received update, to said other cache management applications.

42. The system of claim 40, wherein said local updates of said file data are sent as temporally compressed, Delta files.

43. The system of claim 40, wherein said remote cache management application comprises means for listening to file access events from applications associated with said remote computer, and transmitting any changes to said file data effected by said file access events to said local cache management application and any other cache management applications associated with computers other than said local computer and said remote computer which are actively subscribed to said-file data from said remote cache management application.

44. The system of claim 5, wherein said local cache management application further comprises means for controlling a flow of data from said remote cache management application to said local cache management application.

45. The system of claim 41, wherein said transmitting any changes to said file data resultant from said received update, to said other cache management applications is conducted using a point-to-multi-point data transmission process.

46. The system of claim 43, wherein said transmitting any changes to said file data effected by said file access events to said local cache management application and any other cache management applications associated with computers other than said local computer and said remote computer which are actively subscribed to said file data from said remote cache management application, is conducted using a point-to-multi-point data transmission process.

47. The system of claim 29, further comprising means for resolving conflicts presented by two or more versions of said data file, based on said Delta and Inverse Delta files.

48. The system of claim 1, further comprising a connection server through which said local cache management application connects with said remote cache management application over the wide area network.

49. The system of claim 1, wherein said local and remote cache management applications are connectable over a secure channel.

50. The system of claim 1, wherein said local cache management application is connectable with said remote cache management application over a seamless, firewall-compliant channel.

51. The system of claim 48, wherein said connection server includes means for forming a first, secure, firewall compliant connection with said local cache management application of the local computer, and means for forming a second, secure, firewall compliant connection with said remote cache management application of the remote computer, means for sending communications received from said local cache management application to said remote cache management application while maintaining second firewall compliance, and means for sending communications received from said remote cache management application to said local cache management application while maintaining first firewall compliance.

52. The system of claim 1, wherein said local cache management application and said remote cache management application communicate with one another via encrypted data over the wide area network.

53. An application for providing efficient data access and control of devices among multiple computers connected by a wide area network, comprising:

a strategic cache management process operatively connected with a local device for managing operations of data in cache memory of the local device; and

a strategic cache management process operatively connected with a remote device for managing operations of data in cache memory of the remote device.

54. The application of claim 53, wherein said strategic cache management process in said local device comprises a user module for accessing remote data on said remote device and said user module manages caching operations of said remote data on said local device.

55. The application of claim 53, wherein said strategic cache management process in said remote device comprises a device module for managing access requests from said strategic cache management process in said local device, and said device module manages caching operations of said remote data on said remote device to optimize data sharing capabilities.

56. The application of claim 54, wherein said strategic cache management process in said local device further comprises a device module for managing access requests from said strategic cache management process in said remote device, and said device module manages caching operations of said local data on said local device to optimize data sharing capabilities.

57. The application of claim 55, wherein said strategic cache management process in said remote device further comprises a user module for accessing data on said local device and said user module manages caching operations of said local data on said remote device.

58. The application of claim 53, wherein said strategic cache management processes interface with applications used locally with said local device and said remote device, respectively, and manage data required by application requests in cache memory of said local device and said remote device, respectively.

59. The application of claim 53, further comprising means for temporally compressing an updated file and transmitting only changes between an updated file and a previous version of the file between said remote device and said local device.

60. The application of claim 56, wherein said device module comprises means for generating and maintaining a reference file, of a file data requested from the local strategic cache management process, in cache memory of the remote device.

61. The application of claim 60, wherein said device module comprises means for assigning version numbers to different versions of said file data, said device module further comprising means for tracking said version numbers of said file data subscribed to by said local strategic cache management process and any other strategic cache management process associated with another device and actively subscribed to said file data.

62. The application of claim 61, further comprising a connection server operatively connecting said local device and said remote device across the wide area network, said connection server providing at least one of authentication, authorization, security, encryption and point-to-multipoint communications functionalities to said application.