MODIFYING INFORMATION LIFECYCLE MANAGEMENT RULES IN A DISTRIBUTED SYSTEM

• US 20100185963A1
• Filed: 01/19/2009
• Published: 07/22/2010
• Est. Priority Date: 01/19/2009
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: Bycast Inc. (NetApp, Inc.)
• Inventors: Silvester, Pey, Seiler, Oliver, Slik, David
• Granted Patent: US 8,898,267 B2
• Time in Patent Office: Days
• US Class Current: 715/764
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/1844 Management specifically adapted to replicated file systems
• CPC Class Codes: G06F 16/278 Data partitioning, e.g. horizontal or vertical partitioning

Abstract

A system and method for specifying the placement of and managing the placement of objects on a distributed networked grid. Some embodiments of the invention comprise a user interface for specifying rules for the placement of objects on the grid and an execution module that places objects on the grid based on the rules specified. Rules may comprise a matching criteria specification, which determines whether a rule applies to a particular object, and a placement specification, which directs the locations at which an object will be placed under the rule.

Claims

1. A method of specifying one or more rules that control the placement of data objects, comprising:

determining a system topology of a distributed storage grid;

using the system topology to provide configuration options for one or more rules to a user;

receiving, for each rule, one or more criteria for comparison with an object metadata value, an object duration value, an object status value, or a value contained in the contents of an object;

receiving, for each rule, one or more placement constraints, each placement constraint specifying how an object should be placed within the system topology;

creating a data structure that describes the criteria in a form that can be used to compare against objects;

creating a data structure that describes the placement constraints in a form that can be used to match against the system topology;

incorporating the criteria and placement constraints data structures that comprise the rule with any previously existing rules to form updated rules;

distributing the updated rules across the system;

triggering the processing any existing objects that are affected by the addition of new rules or modification of existing rules.

2. The method of claim 1, wherein the criteria for comparison may be joined by an AND or OR function.

3. The method of claim 1, wherein each of the one or more placement constraints comprises references to one or more storage pools.

4. The method of claim 1, wherein the updated rules comprise at least one default rule, to be used when no other rule matches.

5. The method of claim 1, wherein each of the one or more placement constraints comprises a starting time and an ending time.

6. The method of claim 5, wherein the starting time and ending time are specified relative to the time of object creation, the time of object access, the time of object reference release, or a time in the object'"'"'s metadata.

7. The method of claim 1, wherein the updated rules are ordered.

8. The method of claim 1, further comprising presenting an information display comprising a display of the number of data objects matching one or more rules.

9. The method of claim 8, said information display further comprising a display of the resource requirements resulting from the application of one or more rules.

10. The method of claim 8, said information display further comprising a display of changes to system topology required for the application of one or more rules.

11. The method of claim 1, wherein objects are assigned to an administrative domain, and wherein rules are associated with one or more administrative domains.

12. The method of claim 11, wherein the user is associated with one or more administrative domains, and wherein the administrative domains associated with the one or more rules are limited to the administrative domains with which the user is associated.

13. The method of claim 11, wherein objects are assigned to a partition, wherein each partition is associated with an administrative domain, and wherein rules are associated with one or more partitions.

14. The method of claim 13, wherein the user is associated with one or more partitions, and wherein the partitions associated with the one or more rules are limited to the partitions with which the user is associated.

15. A system to specify and manage the placement of objects on a distributed grid, comprising:

a plurality of storage devices capable of storing objects, wherein the storage devices are networked;

a plurality of rules, wherein each rule comprises at least one matching criteria and at least one placement constraint;

a rule execution module, capable of receiving an object, identifying a rule to apply to the object based on the identified rule'"'"'s matching criteria specification, and placing the object on one or more of the storage devices based on the identified rule'"'"'s placement specification; and

a placement rule specification module, capable of determining a topology of the plurality of storage devices, presenting a user with a user interface, receiving one or more matching criteria from the user, receiving one or more placement constraints from the user, creating a data structure that describes the one or more matching criteria from the user and the one or more placement constraints from the user, incorporating the data structure into the plurality of rules to form updated rules, distributing the updated rules across the system, and triggering the processing of any existing objects that are affected by the addition of new rules or modification of existing rules;

said user interface comprising an interface for specifying a matching criteria for a rule; and

said user interface further comprising an interface to specify a placement constraint for a rule, said interface incorporating the topology of the plurality of storage devices.

16. The system of claim 15, wherein the matching criteria comprises one or more logical tests joined by an AND or OR function.

17. The system of claim 15, wherein the placement constraint comprises references to one or more storage pools.

18. The system of claim 15, wherein the plurality of rules comprises at least one default rule, to be used when no other rule matches.

19. The system of claim 15, wherein the interface that specifies a placement specification for a rule further comprises an input for a starting time and an ending time.

20. The system of claim 19, wherein the starting time and ending time are specified relative to the time of object creation, the time of object access, the time of object reference release, or a time in the object'"'"'s metadata.

21. The system of claim 15, wherein the plurality of rules is ordered.

22. The system of claim 15, said user interface further comprising an information display comprising a display of the number of data objects matching one or more rules.

23. The system of claim 22, said information display further comprising a display of the resource requirements resulting from the application of one or more rules.

24. The system of claim 22, said information display further comprising a display of changes to system topology required for the application of one or more rules.

25. The system of claim 15, further comprising a plurality of administrative domains, wherein objects are assigned to an administrative domain, and wherein the rule execution module, when identifying a rule to apply to an object, further determines whether the rule applies to the object'"'"'s administrative domain.

26. The system of claim 25, wherein the placement rule specification module is capable of limiting a user to affecting only rules associated with a set of administrative domains.

27. The system of claim 25, further comprising a plurality of partitions, wherein each partition belongs to an administrative domain, and wherein objects are assigned to a partition, and wherein the rule execution module, when identifying a rule to apply to an object, further determines whether the rule applies to the object'"'"'s partition.

28. The system of claim 27, wherein the placement rule specification module is capable of limiting a user to affecting only rules associated with a set of partitions.

29. A method of managing the placement of objects on a distributed grid in accordance with a set of placement rules, said method comprising:

receiving a set of user-defined placement rules, wherein the rules are in a specified order;

evaluating the set of user-defined placement rules against an object after a trigger event is received to determine a first match of a user-defined placement rule;

calculating an ideal placement for the object based on the evaluation of the set of user-defined placement rules against the object;

obtaining the topology of the distributed grid;

obtaining a representation of system resources comprising computational resources, storage resources, and network resources;

obtaining a current placement of the object;

calculating an achievable placement for the object based on the current placement, the ideal placement, the topology of the distributed grid, and the representation of system resources;

calculating a set of actions to be performed to transform the current placement into the achievable placement; and

performing one or more of the actions from the set of actions to be performed.

30. The method of claim 29, wherein the trigger event is any of the creation of an instance of an object, access of an instance of an object, destruction of an instance of an object, cessation of existence of an instance of an object, finding an instance of an object, determination that an instance of an object'"'"'s existence is questionable, changing of rules affecting an object, changing of rules possibly affecting an object, request for reevaluation of rules associated with an object, failure of an attempt to change the placement of an object, change in metadata of an object, change in the content of an object, change in the topology of the distributed grid, change in the characteristics of the resources of the distributed grid or firing of a timed trigger.

31. The method of claim 29, wherein the calculation of an ideal placement for the object incorporates at least the metadata associated with the object.

32. The method of claim 29, wherein the calculation of an ideal placement for the object incorporates at least data external to the object or the set of placement rules.

33. The method of claim 29, further comprising producing an alert if the achievable placement is not equivalent to the ideal placement.

34. The method of claim 33, wherein the alert indicates what aspect of the system topology or system resources caused the achievable placement to differ from the ideal placement.

35. The method of claim 29, wherein evaluating the set of user-defined rules against an object comprises placing the object on a queue for object evaluation.

36. The method of claim 35, wherein the object is removed from the queue if the current placement is equivalent to the achievable placement.

37. The method of claim 29, further comprising prioritizing the set of actions to be performed based on the representation of system resources.

38. The method of claim 29, further comprising batching together actions to be performed as a single action, from the set of actions to be performed.

39. The method of claim 29, wherein performing one or more of the actions from the set of actions to be performed comprises identifying parallelizable actions and performing the parallelizable actions in parallel.

40. The method of claim 29, further comprising prioritizing the set of actions to be performed based on a set of safety restrictions to maximize the degree of replication at any given time.

41. The method of claim 29, further comprising adding one or more safety placement locations to the achievable placement to ensure a minimum replication degree or placement.

42. The method of claim 41, wherein the one or more safety placement locations are contained in a placement rule.

43. The method of claim 29, wherein further comprising caching results of calculations and reusing the cached results of the calculations for new calculations when the inputs to the calculations have not changed.

44. The method of claim 29, wherein evaluating the set of user-defined placement rules comprises sending a notification to one or more external systems.

45. A method of efficiently processing objects affected by an addition or change to rules, comprising:

caching with each object a first rule based on a previous rules evaluation,

adding the object to a set of objects associated with the first rule,

receiving a change to a second rule,

determining whether the second rule has priority over the first rule,

retrieving the set of objects associated with the first rule,

retrieving a metadata index of the set of objects associated with the first rule,

constructing a subset of affected objects from the set of objects associated with the first rule, based on the metadata index and the change to the second rule, and

invoking a trigger to cause rules processing for the subset of affected objects.

46. The method of claim 45, wherein objects belong to an administrative domain and partition, and wherein the set of objects associated with the first rule is limited to objects of the same administrative domain and partition.