Modifying information lifecycle management rules in a distributed system

• US 8,898,267 B2
• Filed: 01/19/2009
• Issued: 11/25/2014
• Est. Priority Date: 01/19/2009
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Slik, David, Seiler, Oliver, Silvester, Pey
• Primary Examiner(s): Woolcock, Madhu
• Application Number:
  US12/356,049
• Publication Number:
  US 20100185963A1
• Time in Patent Office: 2,136 Days
• Field of Search: 709220-221, 709223-224, 709/226, 707600-601, 707/609, 707661-663, 707665-666, 707/694, 707/696, 707736-741, 707/758, 707791-793, 707/797, 707802-803, 707/812, 707/821, 707/809
• US Class Current: 709/223
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/1844 Management specifically adapted to replicated file systems
• CPC Class Codes: G06F 16/278 Data partitioning, e.g. horizontal or vertical partitioning

Abstract

A system and method for specifying the placement of and managing the placement of objects on a distributed networked grid. Some embodiments of the invention comprise a user interface for specifying rules for the placement of objects on the grid and an execution module that places objects on the grid based on the rules specified. Rules may comprise a matching criteria specification, which determines whether a rule applies to a particular object, and a placement specification, which directs the locations at which an object will be placed under the rule.

Claims

1. A computer-implemented method comprising:

providing configuration options for one or more rules regarding placement of data objects within a system topology of a distributed storage grid comprising a plurality of storage devices associated with a plurality of isolated administrative domains, each data object being assigned to at least one isolated administrative domain of the plurality of isolated administrative domains, the one or more rules being associated with the at least one isolated administrative domain of the plurality of isolated administrative domains, and the configuration options being based on the system topology, the configuration options comprising one or more user interface elements indicative of data storage capabilities associated with the plurality of storage devices;

receiving, for each of the one or more rules, one or more criteria for comparison with a data object metadata value, an object duration value, a data object status value, or a value contained in contents of a data object;

receiving, for each of the one or more rules, one or more placement constraints, each placement constraint specifying how a particular data object is to be placed within the system topology and a length of time that the particular data object is to be stored, the length of time being specified relative to a time of data object creation, a time of data object access, or a time stored in metadata associated with the particular data object;

creating a criteria data structure that describes the one or more criteria in a form that can be used to compare against data objects;

creating a placement constraints data structure that describes the one or more placement constraints in a form that can be used to match against the system topology;

incorporating the criteria data structure and the placement constraints data structure with one or more previously existing rules to form updated rules;

distributing the updated rules across one or more of the plurality of storage devices;

triggering processing of data objects that are affected by the updated rules;

associating a dirty state with the data objects that are affected by the updated rules;

determining a proposed action to modify a placement of a data object associated with the dirty state by at least matching the data object against the updated rules based at least partially on the criteria data structure incorporated in the updated rules; and

executing the proposed action to modify the placement of the data object repeatedly while the data object is associated with the dirty state indicating that further changes to the placement of the data object are required to place the data object within the system topology based upon the placement constraints data structure.

2. The method of claim 1, wherein the criteria data structure comprises an indication of an AND or OR function.

3. The method of claim 1, wherein each of the one or more placement constraints comprises references to one or more storage pools.

4. The method of claim 1, wherein the updated rules comprise at least one default rule to be used when no other rule matches.

5. The method of claim 1, wherein the updated rules are ordered.

6. The method of claim 1, further comprising displaying a number of data objects matching the one or more rules on an information display.

7. The method of claim 6, further comprising resource requirements resulting from an application of the one or more rules on an information display.

8. The method of claim 6, further comprising displaying changes to the system topology required for an application of the one or more rules on the information display.

9. The method of claim 1, wherein the plurality of isolated administrative domains are associated with a user, and wherein the at least one isolated administrative domain associated with the one or more rules are limited to isolated administrative domains with which the user is associated.

10. The method of claim 1, wherein the data objects are assigned to a partition, wherein each partition is associated with an isolated administrative domain, and wherein the one or more rules are associated with one or more partitions.

11. The method of claim 10, wherein the one or more partitions are associated with a user, and wherein the one or more partitions associated with the one or more rules are limited to one or more partitions with which the user is associated.

12. The method of claim 1, wherein triggering the processing of data objects comprises:

caching with each data object a first rule based on a previous rules evaluation,

adding a data object to a set of data objects associated with the first rule,

receiving a change to a second rule,

determining whether the second rule has priority over the first rule,

retrieving the set of data objects associated with the first rule,

retrieving a metadata index of the set of data objects associated with the first rule,

constructing a subset of affected data objects from the set of data objects associated with the first rule, based on the metadata index and the change to the second rule, and

invoking a trigger to cause rules processing for the subset of affected data objects.

13. The method of claim 12, wherein data objects belong to an isolated administrative domain and a partition, and wherein the set of data objects associated with the first rule is limited to data objects of the same isolated administrative domain and partition.

14. The method of claim 1, wherein the one or more user interface elements indicative of storage capabilities is to further indicate an estimated quantity of data objects to be managed by one or more rules.

15. The method of claim 1, wherein the one or more user interface elements indicative of storage capabilities is to further indicate an estimated quantity of data objects that will match the one or more received criteria for comparison.

16. The method of claim 1, wherein the one or more user interface elements indicative of storage capabilities is to further indicate an estimated available storage capacity associated with storage devices that will match the one or more placement constraints.

17. A system comprising:

a plurality of networked storage devices capable of storing data objects;

a memory storing a plurality of rules, wherein each rule comprises at least one matching criteria and at least one placement constraint;

a processor configured to identify a rule to apply to a received data object based on the identified rule'"'"'s matching criteria specification, and place the received data object on one or more of the networked storage devices based on placement constraints of the identified rule, wherein each data object is assigned to at least one isolated administrative domain of a plurality of isolated administrative domains, the plurality of rules is associated with the at least one isolated administrative domain of the plurality of isolated administrative domains; and

the processor further configured to;

determine a system topology of the plurality of networked storage devices;

generate a user interface based on the determined system topology of the plurality of networked storage devices, the user interface comprising one or more indications of data storage capability parameters associated with at least one of the plurality of networked storage devices;

receive one or more matching criteria and one or more placement constraints, the one or more matching criteria for comparison with a data object metadata value, an object duration value, a data object status value, or a value contained in content of a data object, and the one or more placement constraints specifying how a particular data object is to be placed within the system topology and a length of time that the particular data object is to be stored, the length of time being specified relative to a time of data object creation, a time of data object access, or a time stored in metadata associated with the particular data object;

incorporate the one or more matching criteria and the one or more placement constraints into the plurality of rules to form updated rules; and

trigger processing of any data objects that are affected by the updated rules;

associate a dirty state with the data objects that are affected by the updated rules;

determine a proposed action to modify a placement of a data object associated with the dirty state by at least matching the data object against the updated rules based at least partially on the one or more matching criteria incorporated in the updated rules; and

execute the proposed action to modify the placement of the data object repeatedly while the data object is associated with the dirty state indicating that further changes to the placement of the data object are required to place the data object within the system topology based upon the one or more placement constraints, and the user interface is configured to enable a user to specify a matching criteria for at least one rule of the plurality of rules and a placement constraint for the at least one rule of the plurality of rule.

18. The system of claim 17, wherein the matching criteria for the at least one rule of the plurality of rules comprises one or more logical tests joined by an AND or OR function.

19. The system of claim 17, wherein the placement constraint for the at least one rule of the plurality of rules comprises references to one or more storage pools.

20. The system of claim 17, wherein the plurality of rules comprises at least one default rule to be used when no other rule matches.

21. The system of claim 17, wherein the plurality of rules is ordered.

22. The system of claim 17, the user interface further comprising an information display configured to display a number of data objects matching the plurality of rules.

23. The system of claim 22, the information display further configured to display resource requirements resulting from an application of the plurality of rules.

24. The system of claim 22, the information display further configured to display changes to the system topology required for an application of the plurality of rules.

25. The system of claim 17, wherein the processor is further configured to determine whether a rule of the plurality of rules applies to the received data object'"'"'s isolated administrative domain, when identifying a rule to apply to the received data object.

26. The system of claim 25, wherein the processor is further configured to limit a user to affecting only rules associated with a set of isolated administrative domains.

27. The system of claim 25, further comprising a plurality of partitions, wherein each partition of the plurality of partitions belongs to an isolated administrative domain of the plurality of isolated administrative domains, and wherein data objects are assigned to a partition, and wherein the processor is further configured to determine whether a rule applies to a data object'"'"'s partition, when identifying a rule to apply to the received data object.

28. The system of claim 27, wherein the processor is further configured to limit a user to affecting only rules associated with a set of partitions.

29. The system of claim 17, wherein the one or more indications of data storage capability parameters is to further indicate an estimated quantity of data objects to be managed by the plurality of rules.

30. The system of claim 17, wherein the one or more indications of data storage capability parameters is to further indicate an estimated quantity of data objects that will match the one or more received matching criteria for comparison.

31. The system of claim 17, wherein the one or more indications of data storage capability parameters is to further indicate an estimated available storage capacity associated with storage devices that will match the one or more placement constraints.

32. The system of claim 17, wherein the processor is configured to trigger processing of the data objects by at least repeatedly determining and executing the proposed action to modify the placement of the data object, the proposed action being based at least partially on a current placement of the data object, a current grid topology, and the updated rules, wherein the determination and execution are repeated while the data object is associated with the dirty state indicating that further changes to the placement of the data object are required.