Scalable storage system supporting multi-level query resolution

• US 6,829,610 B1
• Filed: 03/11/1999
• Issued: 12/07/2004
• Est. Priority Date: 03/11/1999
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: DeMello, Marco A., Hickman, Robert Chris
• Primary Examiner(s): PARDO, THUY N
• Application Number:
  US09/266,459
• Time in Patent Office: 2,098 Days
• Field of Search: 711/202, 395/187.01, 395/200.49, 395/200.79, 707/1, 707/3, 707/10, 707/201, 707/2, 707/8
• US Class Current: 1/1
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99938 Concurrency, e.g. lock management in shared database
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users

Abstract

A massively scalable architecture for providing a highly reliable storage system that is capable of handling hundreds of millions of users and tens of billions of files. The system includes one or more clusters storing data elements that are received from a plurality of clients. Each cluster comprises a plurality of storage servers. Each storage server maintains one or more indexes for responding to simple read requests such as retrieving a designated element. According to the invention, the storage system includes one or more database clusters each database cluster having one or more database servers. The storage system accesses the storage clusters in order to reply to simple read requests; however, the database clusters are used to resolve more complex queries that are based on attributes of elements stored within the storage system. In one configuration, the storage system includes a database cluster for each class of applications accessing the storage system.

Claims

1. A storage system comprising:

multiple storage servers, each storage server storing data elements from multiple clients without using a databases, the data elements stored in one storage server being replicated in other storage servers; and

one or more database servers for receiving attributes of the data elements and further for resolving requests to access the data elements stored in the multiple storage servers as a function of the attributes.

2. The storage system of claim 1, wherein at least one of the storage servers has an index for retrieving a requested data element.

3. The storage system of claim 1, wherein the multiple storage servers are grouped into one or more storage clusters.

4. The storage system of claim 1, wherein the database servers are grouped into database clusters.

5. The system of claim 1, wherein the clients are Internet users.

6. The system of claim 1, wherein the clients are applications executing on at least one Web server.

7. The system of claim 1, wherein the clients are applications executing on at least one Web server, and further wherein the database servers are grouped into database clusters such that each database cluster corresponds to one of the applications.

8. The system of claim 7, wherein the applications conform to an Internet Server Application Programming Interface (ISAPI).

9. The system of claim 6 and further including a storage access module executing on each of the Web servers, wherein the storage access module communicates the requests to the database servers when the requests require searching for data elements according to the attributes of the data elements.

10. The system of claim 9 and wherein the storage access module communicates the requests to the storage servers when the requests specify a particular data element for retrieval.

11. The system of claim 1, wherein said storage server stores said data elements in partitions associated with said clients.

29. The system of claim 6, wherein each of said partitions is associated with one of said clients.

12. A computing method comprising:

storing data elements within multiple storage servers without using a database, the data elements stored in one storage server being replicated in other storage servers;

directing access requests to the multiple storage servers when the requests specify a particular data element for retrieval; and

directing the access requests to multiple database servers when the requests require searching for data elements according to attributes of the data elements.

13. The method of claim 12, further including receiving the requests from Internet users.

14. The method of claim 12, further including receiving requests from applications executing on at least one Web server.

15. The method of claim 14, wherein the database servers are grouped into database clusters such that each database cluster corresponds to one of the applications, and further wherein directing the requests to the database servers includes directing the requests to the database cluster corresponding to the application issuing the request.

16. A computer-readable medium having computer-executable instructions for performing the method of claim 12.

17. A computer-readable medium having computer-executable instructions for performing the methods of claims 13 through 15.

18. The method of claim 12, wherein said storage server stores said data elements in partitions associated with a plurality of clients.

19. The method of claim 18, wherein each of said partitions is associated with one of said clients.

20. A system comprising:

multiple Web servers receiving client requests to store data elements;

multiple storage clusters storing the data elements received from the clients without using a database, each cluster comprising multiple storage servers, the data elements stored in one storage server being replicated in other storage servers; and

multiple database clusters for receiving attributes of the data elements and further for resolving requests to access the data elements stored in the multiple storage servers as a function of the attributes.

21. The system of claim 20 and further comprising a storage access module executing on each of the Web servers, wherein for each client request the storage access module selectively communicates the request to the storage clusters and the database clusters.

22. The system of claim 21, wherein the storage access module communicates the requests to the database clusters when the requests require searching for data elements according to the attributes of the data elements.

23. The system of claim 21 wherein the storage access module communicates the requests to the storage clusters when the requests specify a particular data element for retrieval.

24. The system of claim 20, wherein the data elements stored within each cluster are replicated across the multiple storage servers within the respective cluster.

25. The system of claim 20, wherein at least one of the multiple storage servers has an index for retrieving a requested data element.

26. The system of claim 20, wherein the clients are Internet users.

27. The system of claim 20, wherein the clients are applications executing on the Web servers, and further wherein each database cluster corresponds to one of the applications.

28. The system of claim 27, wherein the applications conform to an Internet Server Application Programming Interface (ISAPI).

30. The system of claim 20, wherein said storage server stores said data elements in partitions associated with said clients.

31. The system of claim 30, wherein each of said partitions is associated with one of said clients.