EFFICIENTLY MANAGING MODULAR DATA STORAGE SYSTEMS

• US 20100058023A1
• Filed: 08/29/2008
• Published: 03/04/2010
• Est. Priority Date: 08/29/2008
• Status: Active Grant

• Current Assignee: Zhigu Holdings Limited
• Original Assignee: Microsoft Corporation
• Inventors: Tan, Kee Hern
• Granted Patent: US 8,180,838 B2
• Time in Patent Office: Days
• US Class Current: 711/202
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0667 at data level, e.g. file, record or object virtualisation
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

The management of a data storage system. The system may store data objects that are subject to change in container sets. The data storage system uses location maps to map the data objects to a corresponding container. When there has been, or will be, a change in the availability of containers, a new location map is created which maps the data objects to a new potentially overlapping set of containers. New data objects are added to the new set of containers, and a data object is found by searching all location maps. As an alternative or as an addition to this system, data objects may be stored in a manner that they may be efficiently removed when a condition is met. A container is created which stores all data objects to be removed when the condition occurs. When the condition occurs, the container is removed.

Claims

1. A method for efficiently managing a modular data storage system which stores a plurality of data objects, the method comprising:

an act of determining that a first plurality of containers is available for storing a plurality of data objects, wherein the number and identity of the plurality of data objects is subject to change;

an act of defining a previous set of one or more location maps associated with the data storage system, wherein the previous set of one or more location maps deterministically maps the plurality of data objects to a corresponding one or more containers of the first plurality of containers;

an act of using the previous set of one or more location maps to store at least some of the plurality of data objects in respective containers of the first plurality of containers;

an act of determining that there has been or is to be a change in the availability of containers of the first plurality of containers to a subsequent second plurality of containers that is available for storing the plurality of data objects;

an act of defining a second location map associated with the data storage system, wherein the second location map deterministically maps the plurality of data objects to a corresponding one or more containers of the second plurality of containers, wherein the second plurality of containers may, but need not, at least partially overlap the first plurality of containers such that one or more containers may be members of both the first and second plurality of containers;

an act of using the second location map to store at least some of the plurality of data objects in respective containers of the second plurality of containers; and

an act of searching for at least one data object stored in the first or second plurality of containers, comprising;

an act of using at least one of the previous set of one or more location maps to determine whether the at least one data object is stored in the first plurality of containers; and

an act of using the second location map to determine whether the at least one data object is stored in the second plurality of containers.

2. The method in accordance with claim 1, wherein the previous set of one or more location maps and the second location map each comprises at least one hash map.

3. The method in accordance with claim 2, wherein the hash maps store hashes of e-mail addresses corresponding to e-mail messages stored in the data storage system.

4. The method in accordance with claim 1, wherein the plurality of data objects comprises e-mail messages.

5. The method in accordance with claim 4, wherein the e-mail messages comprise spam e-mail messages.

6. The method in accordance with claim 1, further comprising the act of determining when a data object is no longer valid.

7. The method in accordance with claim 6, further comprising the act of removing at least one of the previous set of one or more location maps when it is determined that all of the data objects deterministically mapped by the removed location map are no longer valid.

8. The method in accordance with claim 1, wherein the change in the availability of containers of the first plurality of containers comprises adding one or more containers to the first plurality of containers to create the second plurality of containers, and wherein the second plurality of containers comprises the first plurality of containers together with the added one or more containers.

9. The method in accordance with claim 8, wherein the act of defining the second location map occurs after the one or more containers have been added to the first plurality of containers to create the second plurality of containers.

10. The method in accordance with claim 1, wherein the change in the availability of containers of the first plurality of containers comprises removing one or more containers from the first plurality of containers to create the second plurality of containers, and wherein the second plurality of containers comprises the first plurality of containers minus the removed one or more containers.

11. The method in accordance with claim 10, wherein the act of removing the one or more containers occurs after defining the second location map and after removing all location maps which deterministically map objects to the one or more containers which are being removed.

12. A method for storing a plurality of data objects from a data storage system such that they may be later efficiently removed, the method comprising:

an act of receiving a data object which will be removed when a condition is met;

an act of determining that the data object is of a type which is to be removed according to the condition;

an act of determining whether a container exists for storing a plurality of data objects which are to be removed according to the condition;

if it is determined that the container does not already exist, an act of creating the container for storing the plurality of data objects which are to be removed according to the condition; and

an act of storing the data object in the container that is determined to exist for storing data objects which are to be removed according to the condition or, in the case where such a container is not determined to already exist, the container that was created in the act of creating.

13. A method in accordance with claim 12, further comprising:

an act of determining that the condition for removal of the data objects in the container has been met, or if the condition for removal has changed, an act of determining that the changed condition for removal of the data objects in the container has been met; and

an act of removing the container when it is determined that the condition or the changed condition has been met.

14. The method in accordance with claim 13, wherein the container comprises a volume.

15. The method in accordance with claim 14, wherein the act of removing the container comprises deleting the volume.

16. The method in accordance with claim 13, wherein the container comprises a database table.

17. The method in accordance with claim 16, wherein the act removing the container comprises dropping the database table.

18. The method in accordance with claim 12, wherein the condition is an expiration date.

19. The method in accordance with claim 12, wherein the data object comprises a spam e-mail message.

20. A method for efficiently administering a spam quarantine system which stores a plurality of spam e-mail messages, the method comprising:

an act of determining that a first plurality of containers is available for storing a plurality of spam e-mail messages, wherein the number and identity of the plurality of spam e-mail messages is subject to change;

an act of defining a first hash map associated with the spam quarantine system, wherein the first hash map defines a range of hash values which are distributed across the first plurality of containers, and wherein the first hash map deterministically maps each hash value of the range of hash values to one container of the first plurality of containers;

an act of using the first hash map to store at least some of the plurality of spam e-mail messages in respective containers of the first plurality of containers;

an act of determining that one or more containers has been or will be added to the first plurality of containers to define a subsequent second plurality of containers that is available for storing the plurality of spam e-mail messages;

an act of defining a second hash map associated with the spam quarantine system, wherein the second hash map defines a same range of hash values defined by the first hash map but which are distributed across the second plurality of containers, and wherein the second hash map deterministically maps each hash value of the same range of hash values to one container of the second plurality of containers, and further wherein the second plurality of containers may, but need not, at least partially overlap the first plurality of containers such that one or more containers may be members of both the first and second plurality of containers;

an act of using the second hash map to store at least some of the plurality of spam e-mail messages in respective containers of the second plurality of containers;

an act of searching for at least one of the plurality of spam e-mail messages stored in the first or second plurality of containers, comprising;

an act of using the first hash map to determine whether the at least one of the plurality of spam e-mail messages is stored in the first plurality of containers; and

an act of using the second hash map to determine whether the at least one of the plurality of spam e-mail messages is stored in the second plurality of containers;

an act of determining that the first hash map is no longer valid when all data objects mapped by the first hash map are determined to no longer be valid; and

in response to determining that the first hash map is no longer valid, an act of removing the first hash map.