Lookup partitioning storage system and method

• US 7,676,551 B1
• Filed: 06/25/2003
• Issued: 03/09/2010
• Est. Priority Date: 06/25/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Harry, Craig Allen, Jiang, Shan, Barrows, Bradley J., Dani, Nishant, Shutt, David R., Dalia, Apurva F., Thompson, Brad Dean
• Primary Examiner(s): Srivastava; Vivek
• Assistant Examiner(s): Swearingen; Jeffrey R
• Application Number:
  US10/606,626
• Time in Patent Office: 2,449 Days
• Field of Search: 709/225
• US Class Current: 709/218
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A method, system, and computer-readable medium for accessing and managing an online storage system is disclosed. Access to a storage resource in a multiple server storage system is provided by sending to a lookup partitioning service server a resource identifier that is associated with a storage resource stored in a particular storage partition of a storage server. The LPS returns the looked-up partition that stores the storage resource associated with the resource identifier. Access to the storage resource is then enabled to the looked-up storage partition.

Claims

1. A computer implemented method of managing access to a storage resource for one of a plurality of network-based applications in a multiple server storage system, the method comprising:

obtaining a resource identifier from a front end server;

utilizing the resource identifier to lookup, in a resource lookup store of a lookup partitioning service server, a partition of a storage server associated with the resource identifier;

in an event the partition of the storage server is associated with the resource identifier, granting access to the storage resource by providing a location of the partition of the storage server to the front end server; and

in an event no partition of the storage server is associated with the resource identifier;

failing to locate a mapping to the storage resource;

determining a load balancing factor for each storage partition of a plurality of storage partitions;

using the load balancing factors to determine a new storage partition in which a new storage resource should be created and creating the new storage resource in the new storage partition;

mapping the resource identifier to the new storage partition in the resource lookup store; and

providing a location of the new storage partition to the front end server,

wherein determining a load balancing factor for each storage partition comprises;

given n storage partitions, determining the number of mapping counts C for each of the n storage partitions; and

calculating a load balancing factor (LBF_m) for each storage partition m using the following;

LBF_m=(1/C_m)/(1/C₁+1/C₂+ . . . +1/C_n).

2. The method of claim 1, wherein the location of the partition of the storage server is on one of a plurality of storage servers.

3. The method of claim 1, wherein the plurality of storage partitions include a primary storage partition and a redundant storage partition each containing the storage resource.

4. The method of claim 3, wherein the primary storage partition and the redundant storage partition are each located on separate storage servers of a plurality of storage servers.

5. The method of claim 4, wherein, if the primary storage partition is unavailable, the storage server location is the redundant storage partition.

6. The method of claim 1, further comprising determining which lookup partitioning service server of a plurality of lookup partitioning service servers will provide wherein the looked-up storage server location in response to the resource identifier.

7. The method of claim 6, wherein determining which lookup partitioning service server will provide the looked-up storage server location comprises processing the resource identifier through a hash function to provide a hashed resource identifier associated with a particular lookup partitioning service server.

8. The method of claim 7, wherein each lookup partitioning service server is associated with a predetermined set of hashed resource identifiers.

9. The method of claim 1, further comprising moving the storage resource from one storage partition to another storage partition and updating the resource lookup store with the another storage partition.

10. The method of claim 1, wherein the load balancing factors are further based on a value selected from the values consisting of:

a mapping number and a manual weighting value.

11. The method of claim 1 further comprising adjusting a manual weighting value to increase a usage of a storage server.

12. The method of claim 1, further comprising adjusting a manual weighting value to decrease a usage of a storage server.

13. The method of claim 1, wherein using the load balancing factors to determine a new storage partition comprises:

generating a random number R, where R is a real number between 0 and 1;

ranking the load balancing factors in ascending order;

locating a particular storage partition, where the sum of all the load balancing factors ranked lower than the load balancing factor corresponding to the particular storage partition is less than or equal to R, and the sum of all the load balancing factors ranked lower than the load balancing factor corresponding to the particular storage partition plus the load balancing factor corresponding to the particular storage partition is greater than R.

14. A computer readable storage medium containing computer-executable instructions for performing a method of managing access to a storage resource for one of a plurality of network-based applications in a multiple server storage system, the computer-executable instructions comprising instructions comprising:

receiving a resource identifier associated with the storage resource from a front end server;

utilizing the resource identifier to lookup, in a resource lookup store of a lookup partitioning service server, a storage partition associated with the resource identifier;

in an event the storage partition is associated with the resource identifier;

locating the storage partition of a storage server; and

sending the location of the storage partition to the front end server to grant access to the storage resource; and

in an event the storage partition is not associated with the resource identifier;

failing to locate a mapping to the storage resource;

determining a load balancing factor for each storage partition of a plurality of storage partitions;

using the load balancing factors to determine a new storage partition in which a new storage resource should be created and creating the new storage resource in the new storage partition;

mapping the resource identifier to the new storage partition in the resource lookup store; and

sending a location of the new storage partition to the front end server,

wherein determining a load balancing factor for each storage partition comprises;

given n storage partitions, determining the number of mapping counts C for each of the n storage partitions; and

calculating a load balancing factor (LBF_m) for each storage partition m using the following;

LBF_m=(1/C_m)/(1/C₁+1/C₂+ . . . +1/C_n.

15. The computer-readable storage medium of claim 14, further comprising determining which lookup partitioning service server of a plurality of lookup partitioning service servers will locate the storage partition in response to the resource identifier.

16. The computer-readable storage medium of claim 14, further comprising relocating the storage resource from one storage partition to a different storage partition and updating the mapping of the resource identifier at the lookup partitioning service server.

17. The computer-readable storage medium of claim 14, wherein the load balancing factors are further based on a value selected from the values consisting of:

a mapping number and a manual weighting value.

18. The computer-readable storage medium of claim 14, wherein using the load balancing factors to determine a new storage partition comprises:

generating a random number R, where R is a real number between 0 and 1;

ranking the load balancing factors in ascending order;

locating a particular storage partition, where the sum of all the load balancing factors ranked lower than the load balancing factor corresponding to the particular storage partition is less than or equal to R, and the sum of all the load balancing factors ranked lower than the load balancing factor corresponding to the particular storage partition plus the load balancing factor corresponding to the particular storage partition is greater than R.

19. A lookup partitioning server comprising:

a processing unit;

at least one primary lookup partition; and

at least two redundant lookup partitions which mirror two respective different primary lookup partitions stored on other look-up partitioning servers;

a memory configured to store computer-executable instructions configured to manage access to a plurality of storage resources at a plurality of storage servers, the computer-executable instructions performing acts comprising;

receiving a resource identifier associated with a storage resource from a front end server;

utilizing the resource identifier to lookup, in a resource lookup store, a storage partition associated with the resource identifier;

in an event a particular storage partition is associated with the resource identifier;

locating the particular storage partition of a storage server; and

sending a location of the particular storage partition of the storage server to the front end server to grant access to the storage resource; and

in an event no particular storage partition is associated with the resource identifier;

failing to locate a mapping to the storage resource;

determining a load balancing factor for each storage partition of a plurality of storage partitions;

using the load balancing factors to determine a new storage partition in which a new storage resource should be created and creating the new storage resource in the new storage partition;

mapping the resource identifier to the new storage partition in the resource lookup store; and

sending a location of the new storage partition to the front end server,

wherein determining a load balancing factor for each storage partition comprises;

given n storage partitions, determining the number of mapping counts C for each of the n storage partitions; and

calculating a load balancing factor (LBF_m) for each storage partition m using the following;

LBF_m=(1/C_m)/(1/C₁+1/C₂+ . . . +1/C_n.

20. The lookup partitioning server of claim 19, wherein using the load balancing factors to determine a new storage partition comprises:

generating a random number R, where R is a real number between 0 and 1;

ranking the load balancing factors in ascending order;

locating another particular storage partition, where the sum of all the load balancing factors ranked lower than the load balancing factor corresponding to the another particular storage partition is less than or equal to R, and the sum of all the load balancing factors ranked lower than the load balancing factor corresponding to the another particular storage partition plus the load balancing factor corresponding to the particular storage partition is greater than R.