Storage cluster server network

• US 20060015555A1
• Filed: 07/15/2004
• Published: 01/19/2006
• Est. Priority Date: 07/15/2004
• Status: Active Grant

• Current Assignee: YHC Corporation
• Original Assignee: YHC Corporation
• Inventors: Henigin, Edward, Molter, Philip, Swarin, Douglas, Douglass, Michael, Yokubaitis, Jonah
• Granted Patent: US 7,895,264 B2
• Time in Patent Office: Days
• US Class Current: 709/203
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Abstract

A clustered server network for receiving, storing and retrieving usenet articles is provided. The clustered server network includes various sets of clustered servers. One set of clustered servers may be for receiving articles from peer usenet networks, while another set of clustered servers stores only header information for each unique article. A third set of clustered servers may be dedicated to storing, in a redundant fashion, each unique article. The articles are not stored in any particular order, but instead are stored based on a deterministic calculation so that each unique article stored is found in at least one particular server(s). The redundant storage of each unique article eliminates a need for a backend server cluster for storing one copy of every article. An exemplary usenet system provides for continuous operation even if any server in any cluster becomes non-operational. A server rebuild function is provided in the usenet system. Also, an ability to emulate Radius packets and establish LDAP connections with customers is available.

Claims

1. A usenet system comprising:

a feeder cluster comprising a plurality of feeder servers, each said feeder server being in communication with said other feeder servers, said feeder cluster adapted to receive usenet articles from at least one source outside of said usenet system;

a server cluster comprising a plurality of cluster servers, each said cluster server being in communication with said other cluster servers, said cluster servers being adapted to store articles received from said feeder cluster; and

a header cluster comprising a plurality of header servers, each said header servers being in communication with said other header servers, said header servers adapted to store headers associated with said articles stored in said cluster servers, said header server adapted to receive said headers from said feeder cluster.

2. The usenet system of claim 1, wherein said at least one source comprises a usenet peer system.

3. The usenet system of claim 1, wherein said server cluster is adapted to have a data connection with at least one usenet customer.

4. The usenet system of claim 1, wherein said plurality of feeder servers are adapted to read a usenet article'"'"'s message-id and determine a specific cluster server, of said plurality of cluster servers, that is to receive the usenet article associated with said article message-id.

5. The usenet system of claim 4, wherein said plurality of servers perform a hash of said usenet article'"'"'s message-id and use the hash result to deterministically calculate which cluster server(s) is to store said usenet article.

6. The usenet system of claim 1, wherein said feeder cluster instructs said server cluster to redundantly store a particular usenet article.

7. The usenet system of claim 5, wherein said usenet article is redundantly stored on more than one cluster server.

8. The usenet system of claim 1, wherein each usenet article is given an out-of-band article number by said header cluster, said out-of-band article number being a number unrelated to a usenet article'"'"'s message-id number.

9. The usenet system of claim 1, wherein no backend servers are connected to said usenet system.

10. The usenet system of claim 1, wherein said feeder servers share the flow of usenet articles from said source outside said usenet system, said feeder servers adapted to use load balancing techniques to share said flow of usenet articles.

11. The usenet system of claim 1, adapted to be connected to a second usenet system that is configured substantially the same as said usenet system, said usenet system and said second usenet system being able to retrieve usenet articles from each other.

12. The usenet system of claim 1, wherein each one of said feeder servers, cluster servers and header servers are each substantially a commodity server operating substantially the same program that allows said commodity server to be configured to operate as at least one of a feeder server, a cluster server, and a header server.

13. The usenet system of claim 1, wherein each said usenet article comprises at least one of a written information, an audio information, and an video information.

14. A clustered server network for receiving, storing and retrieving data articles, each said data article comprising header information and data, said clustered server network comprising:

a feeder cluster for receiving said data articles from an external data article source;

a header cluster for receiving data article header information provided from said feeder cluster, said header cluster organizing and storing said header information for said data articles; and

a server cluster for storing said data for each data article, said data for said data articles being provided from said feeder cluster.

15. The clustered server network of claim 14, wherein said header information comprises a message-id and wherein said feeder uses said message-id to deterministically calculate where in said server cluster said data associated with said header is to be stored.

16. The clustered server network of claim 15, wherein each said data article is assigned an article number by said header cluster, said article number being unrelated to said message-id of said data article.

17. The clustered server network of claim 14, adapted to allow a customer to communicate with said server cluster such that said customer can select a first data article to view, said customer having a subscription to said cluster server network that defines the types of data articles that said customer can view.

18. The clustered server network of claim 17, wherein said subscription defines a number of data articles that said customer can view.

19. The clustered server network of claim 17, wherein said server cluster uses an authorization service to aid a determination of the subscription.

20. The clustered server network of claim 14, wherein said feeder cluster comprises a plurality of feeder servers that are in communication with one another.

21. The clustered server network of claim 14, wherein said server cluster comprises a plurality of cluster servers, said data from each data article being stored redundantly on more than one cluster server.

22. The clustered server network of claim 21, wherein the number of redundant copies being stored is dependant on the age of each data article.

23. The clustered server network of claim 22, wherein the age of each data article is based on a date each data article was first received by said clustered server network.

24. The clustered server network of claim 21, wherein said feeder cluster deterministically calculates which one of said plurality of cluster servers will be a primary cluster server for storing a first data article, and wherein a redundant first data article is stored redundantly in a second cluster server.

25. The clustered server network of claim 14, wherein said header cluster comprises a plurality of header servers.

26. The clustered server network of claim 25, wherein a predetermined header server is adapted to establish an out-of-band article number for each new data article received by said clustered server.

27. The clustered server network of claim 26, wherein said article number is not related to a data article'"'"'s message-id, said message-id being a unique number assigned to each unique data article and found in said header of each data article.

28. The clustered server network of claim 14, wherein communication protocols are used to transfer data articles, header information and data about said clustered server network, said communication protocol comprises at least one of NNTP, FTP, SMTP, POP3, IMAP, HTTPS and HTTP protocols.

29. The clustered server network of claim 14, wherein said article comprises at least one of a news article, text, fonts, pictures, graphics, music, video clips, software, moving pictures and digital information.

30. A clustered server network comprising:

a header cluster comprising a plurality of header servers, said header servers being for storing header information and storage location information for articles;

a server cluster comprising a plurality of cluster servers, said cluster servers being for storing said articles; and

a feeder cluster for receiving said articles from peer networks, said feeder cluster comprising a plurality of feeder servers, each said feeder server adapted to provide header data to said header cluster and said articles to said server cluster.

31. The clustered server network of claim 30, further comprising an authentication server in communication with said server cluster, said authentication server being for receiving an authentication request from said server cluster and for authenticating a customer as being allowed to use an aspect of said usenet network.

32. The clustered server network of claim 31, wherein said authentication server is further configured to determine a relationship between a customer being authenticated and a client to which a customer is associated with, said authentication server communicating with a client server in order to receive authentication criteria for said customer.

33. The clustered server network of claim 32, wherein said authentication server is further configured to authenticate said customer based on a comparison of said authentication criteria received from said client server and authentication information received from said customer.

34. The clustered server network of claim 32, wherein said client server is at least one of an LDAP server and a Radius server.

35. The clustered server network of claim 30, wherein said aspect of said usenet system comprises predetermined usage allowances including at least one of regulating the amount of data that can be downloaded from said usenet system, regulating which usenet newsgroups can be accessed, and regulating the age of data searched.

36. The clustered server network of claim 31, wherein said article comprises at lease one of a news article, text, fonts, pictures, graphics, music, video clips, software, moving pictures and digital information.

37. The clustered server network of claim 30, wherein said feeder cluster receives articles and provides header data and articles using as least one of a NNTP, FTP, SMTP, POP3 IMAP and HTTP protocol.