Highly Available Clustered Storage Network

• US 20080077635A1
• Filed: 08/16/2007
• Published: 03/27/2008
• Est. Priority Date: 09/22/2006
• Status: Abandoned Application

• Current Assignee: Digital Bazaar
• Original Assignee: Digital Bazaar
• Inventors: Johnson, Michael B., Sporny, Manushantha, Longley, David D.
• Application Number:
  US11/839,904
• Publication Number:
  US 20080077635A1
• Time in Patent Office: Days
• US Class Current: 707/204
• CPC Class Codes: G06F 16/1834 implemented based on peer-to-peer networks, e.g. gnutella p2p communication protocols H04L67/104

Abstract

A computing method and system is presented that allows multiple heterogeneous computing systems containing file storage mechanisms to work together in a peer-to-peer fashion to provide a fault-tolerant decentralized highly available clustered file system. The file system can be used by multiple heterogeneous systems to store and retrieve files. The system automatically ensures fault tolerance by storing files in multiple locations and requires hardly any configuration for a computing device to join the clustered file system. Most importantly, there is no central authority regarding meta-data storage, ensuring no single point of failure.

Claims

1. A computing system comprising:

a plurality of storage servers coupled with long-term storage devices;

a communication network;

a plurality of storage clients;

each storage server adapted to communicate via unicast, broadcast or multicast;

each storage server further adapted to store files on a long term basis and process file system requests by the storage client;

each storage server further adapted to asynchronously join and leave the communication network;

each storage server further adapted to automatically mirror files to at least one other storage server such that at least one complete copy of a file remains when a storage server permanently disconnects from the network;

each storage server further adapted to gracefully degrade the file system and provide availability with up to N-(N−

1) system failures;

and each storage server further adapted to use a distributed meta-data storage system.

2. The system of claim 1, further comprising:

a super-node server.

3. The system of claim 1, further comprising:

a privilege device.

4. The system of claim 1, wherein any data stored on the network is stored on at least two different storage servers.

5. The system of claim 4, wherein if one of the two storage servers leaves the network for an extended period of time, the remaining storage server partners with another storage server on the network to mirror the data.

6. The system of claim 5, wherein when a storage server returns to the network, it synchronizes all of the information, meta-data and data files with the up-to-date storage servers.

7. The system of claim 1, wherein any storage client may perform file storage without needing to perform the operation through a central authority.

8. The system of claim 1, wherein any storage client may perform resource queries via the communication network without needing to perform the operation through a central authority.

9. The system of claim 1, wherein any file retrieval may be performed by retrieving the file from a plurality of locations.

10. The system of claim 1, wherein any information or meta-data query may be performed in a distributed, non-centralized manner.

11. The system of claim 3, wherein the privilege device is used to authenticate connections between storage servers, storage clients and super-node servers.

12. The system of claim 1, wherein any modification to information, meta-data, or a data file requires locking the resource before performing the modification.

13. The system of claim 12, wherein any modification to file data may be written to a file journal to speed synchronization between storage servers.

14. The system of claim 1, wherein a method of voting on cluster-wide resources and issues is provided such that any participant in the network may initiate a vote, decision actions are provided for the vote, and every participant that the vote affects votes to determine the decision of the network as a whole.

15. The system of claim 2 and claim 14, wherein elections of regular participants in the network are made to make their super-node servers take responsibility for access and modification of certain meta-data to lower operational latency on the file system.

16. The system of claim 15, wherein meta-data modified via the super-node is synchronized to permanent storage servers and vice-versa.

17. A method for utilizing a computer system and network for the highly-available, fault-tolerant, storage of file data comprising:

pairing a storage server in the network with a long-term storage device;

designating a method of communication via the network that includes unicast, multicast and broadcast messaging;

designating at least one storage client to access the storage servers to store and retrieve file data;

querying the storage servers for information without relying on a central authority or super-nodes;

immediately mirroring information, meta-data and file data between at least two storage servers in the network when possible;

18. The method of claim 17, further comprising:

synchronizing data between an out-of-date storage server that has previously left the network or fallen out of sync and an up-to-date storage server.

19. The method of claim 17, further comprising:

synchronizing data between an out-of-date storage server this is currently connected to the network and a up-to-date storage server that is going to leave the network.

20. The method of claim 17, further comprising:

performing file storage on the network without the aid of a central authority.

21. The method of claim 17, further comprising:

performing a resource query on the network without the need for a central authority.

22. The method of claim 17, further comprising:

retrieving a resource from the network without the direction of a central authority and downloading the resource from multiple up-to-date sources.

23. The method of claim 17, further comprising:

performing a information or meta-data query on the network without the need for a central authority.

24. The method of claim 17, further comprising:

authorizing connections by using a privilege device to ensure authorized connections.

25. The method of claim 24, further comprising:

utilizing the privilege device to authorize specific file system operations by storage clients.

26. The method of claim 17, further comprising:

modifying information, meta-data or file data without the need for a central authority.

27. The method of claim 17, further comprising:

writing modifications to a file journal to aid in synchronization speed between partnered storage servers.

28. The method of claim 17, further comprising:

voting on cluster-wide resources and issues such that any participant in the network may initiate a vote, provide decision actions for the vote, and ensuring that every participant that the vote affects votes to determine the decision of the network as a whole.

29. The method of claim 28, further comprising:

electing a regular participant in the network to super-node status, which will provide a less decentralized authority for a particular set of resources on the network.

30. The method of claim 29, further comprising:

modifications to resources may be made via a super-node and propagated to the partnered storage node on which they belong.