Distributed network data storage system and method

• US 20020147815A1
• Filed: 07/30/2001
• Published: 10/10/2002
• Est. Priority Date: 04/09/2001
• Status: Active Grant

• Current Assignee: Virtuozzo International GmbH
• Original Assignee: Swsoft Holdings, Ltd.
• Inventors: Beloussov, Sergei, Tormasov, Alexander, Protassov, Stanislav, Pudgorodsky, Yuri
• Granted Patent: US 7,209,973 B2
• Time in Patent Office: Days
• US Class Current: 709/226
• CPC Class Codes: G06F 16/1837 Management specially adapted to peer-to-peer storage networks topology management mechanisms of peer-to-peer networks H04L67/1042
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/10015 Access to distributed or replicated servers, e.g. using brokers
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1021 based on client or server locations
• CPC Class Codes: H04L 67/1029 using data related to the state of servers by a load balancer
• CPC Class Codes: H04L 9/40 Network security protocols

Abstract

The present invention is a system and method for distributed, highly scalable, wide area peer-to-peer network data storage. The functionally equivalent servers in the system are divided into groups. Each server maintains a dynamic list which is polled to determine the availability of the closest neighbor servers. Each server is switched between the groups of servers to optimize network connectivity parameters. Data and directory files are divided into a plurality of pieces which are stored on different servers. Files are uniformly and independently named, utilizing a tree with a common root, logical pathways, and unique file identifiers. When a server receives a client request for file system access, the plurality of file pieces are collected and sent to the client server from the neighbor servers simultaneously in order to optimize bandwidth. Servers with maximum throughput capacity are utilized for highest transmission speed and reduced processing time.

Claims

1. A system for organizing distributed file storage, including:

a highly scalable set of servers;

a unified client-server model;

a standard set of network services; and

a small dynamic list of the closest neighbor servers which is maintained by each member of said set of servers.

2. The system of claim 1, wherein said set of highly scalable servers are connected via a peer-to-peer network.

3. The system of claim 1, wherein said set of highly scalable servers are functionally equivalent.

4. A method for organizing distributed file storage including the steps of:

utilizing a plurality of servers in a highly scalable set of servers;

supporting a standard set of network services by each member of said plurality of servers;

wherein said plurality of servers is divided into more than one group of servers;

wherein each member of said plurality of servers belongs to at least one of said more than one group of servers; and

wherein each member of said plurality of servers maintains only a small dynamic list of the closest neighbor members of said plurality of servers.

5. The method of claim 4, further including the step of:

connecting each member of said plurality of servers via a peer-to-peer network.

6. The method of claim 4, further including the step of:

maintaining functional equivalence among each of said members of said plurality of servers.

7. The method of claim 4, further including the step of:

verifying the availability of said closest neighbor members of said plurality of servers.

8. The method of claim 7, further including the steps of:

polling said dynamic list of the closest neighbor members of said plurality of servers;

adding said each member of the plurality of servers into any of said more than one group of servers; and

switching said each member of the plurality of servers into any of said more than one group of severs which has better network connectivity parameters.

9. The method of claim 8, further including the steps of:

maintaining files in said distributed file storage;

dividing said files into a plurality of pieces; and

storing each of said pieces on a different member of said plurality of servers.

10. A method of client access to a distributed file storage system, including the steps of:

utilizing a plurality of servers in a highly scalable set of servers;

supporting a standard set of network services by each member of said plurality of servers;

wherein said plurality of servers is divided into more than one group of servers;

wherein each member of said plurality of servers belongs to at least one of said more than one group of servers; and

wherein each member of said plurality of servers maintains only a small dynamic list of the closest neighbor members of said plurality of servers.

11. The method of claim 10, further including the step of:

connecting each member of said plurality of servers via a peer-to-peer network.

12. The method of claim 10, further including the step of:

maintaining functional equivalence from said client'"'"'s standpoint among each of said members of said plurality of servers.

13. The method of claim 10, further including the step of:

verifying the availability of said closest neighbor members of said plurality of servers.

14. The method of claim 13, further including the steps of:

polling said dynamic list of the closest neighbor members of said plurality of servers;

connecting said client to any of said each member of the plurality of servers; and

switching said client to any of said each member of the plurality of servers which has better network connectivity parameters and smaller workload to improve level of service.

15. The method of claim 14, wherein said client writes a file, further including the steps of:

dividing said file into a plurality of pieces;

sending said plurality of pieces to the client'"'"'s server; and

distributing said plurality of pieces to the closest neighbor servers in order to achieve an appropriate fault tolerance level.

16. The method of claim 14, further including the steps of:

maintaining files in said distributed file storage;

dividing said files into a plurality of pieces; and

storing each of said pieces on a different member of said plurality of servers.

17. The method of claim 16, further including the steps of:

identifying said client'"'"'s name in name space;

sending a request from the client server to any neighbor server;

collecting a set of said plurality of file pieces to assemble a file;

checking for file pieces in local cache and on said neighbor server;

sending said set of pieces to the client server;

transferring said set of pieces to the client.

18. The method of claim 17, further including the step of:

sending all of said pieces of said files from the neighbor servers to said client server simultaneously in order to optimize bandwidth and minimize overall file access time.

20. The method of claim 19, further including the step of:

connecting each member of said plurality of servers via a peer-to-peer network.

21. The method of claim 19, further including the step of:

maintaining functional equivalence among each of said members of said plurality of servers.

22. The method of claim 19, further including the step of:

verifying the availability of said closest neighbor members of said plurality of servers.

23. The method of claim 22, further including the steps of:

polling said dynamic list of the closest neighbor members of said plurality of servers;

adding said each member of the plurality of servers into any of said more than one group of servers; and

switching said each member of the plurality of servers into any of said more than one group of servers which has better network connectivity parameters.

24. The method of claim 19, further including the steps of:

maintaining a uniform name space as a tree with a common root and logical pathways; and

maintaining data files and directory files.

25. The method of claim 24, further including the steps of:

maintaining the directory files as executable files with their own executable code and data; and

providing translation from said logical pathways inside said uniform name space to a unique file identifier.

19. A method of naming files inside a distributed file storage system, including the steps of:

utilizing a plurality of servers in a highly scalable set of servers;

supporting a standard set of network services by each member of said plurality of servers;

naming said files uniformly within said storage system;

naming said files independently of any member of said plurality of servers;

wherein said plurality of servers is divided into more than one group of servers;

wherein each member of said plurality of servers belongs to at least one of said more than one group of servers; and

wherein each member of said plurality of servers maintains only a small dynamic list of the closest neighbor members of said plurality of servers.