Modifying information lifecycle management rules in a distributed system

• US 9,542,415 B2
• Filed: 10/27/2014
• Issued: 01/10/2017
• Est. Priority Date: 01/19/2009
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Slik, David, Seiler, Oliver, Silvester, Pey
• Primary Examiner(s): Woolcock, Madhu
• Application Number:
  US14/524,033
• Publication Number:
  US 20150169625A1
• Time in Patent Office: 806 Days
• Field of Search: 709/223, 709/221, 707/802, 707/809
• US Class Current: 1/1
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/1844 Management specifically adapted to replicated file systems
• CPC Class Codes: G06F 16/278 Data partitioning, e.g. horizontal or vertical partitioning

Abstract

A system and method for specifying the placement of and managing the placement of objects on a distributed networked grid. Some embodiments of the invention comprise a user interface for specifying rules for the placement of objects on the grid and an execution module that places objects on the grid based on the rules specified. Rules may comprise a matching criteria specification, which determines whether a rule applies to a particular object, and a placement specification, which directs the locations at which an object will be placed under the rule.

Claims

1. A method comprising:

determining a system topology of a distributed storage grid comprising a plurality of storage devices for storing a plurality of objects;

presenting, using the system topology, configuration options for one or more rules;

presenting a first user interface for specifying, for each rule of the one or more rules, one or more matching criteria, wherein the one or more matching criteria is specified as a set of logical functions using logical operators for comparing specified parameters with at least one of an object metadata value, an object duration value, an object status value, and a value contained in contents of an object of the plurality of objects, wherein presenting the first user interface comprises presenting a dynamic estimate representing how each of the one or more rules impact the system topology;

presenting a second user interface for specifying, for each rule of the one or more rules, one or more placement constraints, each placement constraint of the one or more placement constraints specifying a storage device of the plurality of storage devices in which an object should be placed within the system topology;

creating at least one data structure summarizing the one or more rules based on the one or more matching criteria and the one or more placement constraints;

incorporating the one or more rules summarized in the at least one data structure into any previously existing rules to form updated rules;

distributing the updated rules across the system topology;

indexing metadata for an existing set of objects in the distributed storage grid to identify objects within the system topology which are affected by the updated rules; and

triggering reevaluation of the identified objects to move the identified objects to a different location within the system topology based on the updated rules.

2. The method of claim 1, wherein at least one placement constraint of the one or more placement constraints specifies a number of copies of the object to store in the storage device.

3. The method of claim 1, wherein at least one placement constraint of the one or more placement constraints specifies a time period that the object is retained in the storage device.

4. The method of claim 1, wherein at least one rule of the one or more rules applies to an object in a defined object state.

5. The method of claim 1, wherein the existing set of objects are assigned to a partition, wherein each partition is associated with an isolated administrative domain, and wherein the one or more rules are associated with one or more partitions.

6. The method of claim 1, wherein presenting the second user interface for specifying the one or more placement constraints comprises presenting an estimated number of objects matching the rule, an estimated amount of storage space consumed by the objects matching the rule, and an estimated amount of storage space available for new objects matching the rule.

7. The method of claim 1, wherein a storage grade is defined for each of the plurality of storage devices, wherein the storage grade comprises a type and a class, wherein at least one placement constraint of the one or more placement constraints specifies the storage grade of the storage device where the object that matches the rule for the at least one placement constraint are to be stored.

8. One or more non-transitory machine-readable storage media having program code for specifying one or more rules that control storage of a plurality of objects stored therein, the program code comprising instructions to:

determine a system topology of a distributed storage grid comprising a plurality of storage devices;

present, using the system topology, configuration options for one or more rules;

present a first user interface for specifying, for each rule of the one or more rules, one or more matching criteria, wherein the one or more matching criteria is specified as a set of logical functions using logical operators for comparing specified parameters with at least one of an object metadata value, an object duration value, an object status value, and a value contained in contents of an object of the plurality of objects, wherein the program code comprising instructions to present the first user interface comprises program code comprising instructions to present a dynamic estimate representing how each of the one or more rules impact the system topology;

present a second user interface for specifying, for each rule of the one or more rules, one or more placement constraints, each placement constraint of the one or more placement constraints specifying a storage device of the plurality of storage devices in which an object should be placed within the system topology;

create at least one data structure summarizing the one or more rules based on the one or more matching criteria and the one or more placement constraints;

incorporate the one or more rules summarized in the at least one data structure into any previously existing rules to form updated rules;

distribute the updated rules across the system topology;

index metadata for an existing set of objects in the distributed storage grid to identify objects within the system topology which are affected by the updated rules; and

trigger reevaluation of the identified objects to move the identified objects to a different location within the system topology based on the updated rules.

9. The one or more non-transitory machine-readable storage media of claim 8, wherein at least one placement constraint of the one or more placement constraints specifies a number of copies of the object to store in the storage device.

10. The one or more non-transitory machine-readable storage media of claim 8, wherein at least one placement constraint of the one or more placement constraints specifies a time period that the object is retained in the storage device.

11. The one or more non-transitory machine-readable storage media of claim 8, wherein at least one rule of the one or more rules applies to an object in a defined object state.

12. The one or more non-transitory machine-readable storage media of claim 8, wherein the existing set of objects are assigned to a partition, wherein each partition is associated with an isolated administrative domain, and wherein the one or more rules are associated with one or more partitions.

13. The one or more non-transitory machine-readable storage media of claim 8, wherein the program code comprising instructions to present the second user interface for specifying the one or more placement constraints comprises program code comprising instructions to present an estimated number of objects matching the rule, an estimated amount of storage space consumed by the objects matching the rule, and an estimated amount of storage space available for new objects matching the rule.

14. The one or more non-transitory machine-readable storage media of claim 8, wherein a storage grade is defined for each of the plurality of storage devices, wherein the storage grade comprises a type and a class, wherein at least one placement constraint of the one or more placement constraints specifies the storage grade of the storage device where the object that matches the rule for the at least one placement constraint are to be stored.

15. An apparatus comprising:

a processor; and

a non-transitory machine-readable medium having program code executable by the processor to cause the apparatus to,

determine a system topology of a distributed storage grid comprising a plurality of storage devices for storing a plurality of objects;

present, using the system topology, configuration options for one or more rules;

present a first user interface for specifying, for each rule of the one or more rules, one or more matching criteria, wherein the one or more matching criteria is specified as a set of logical functions using logical operators for comparing specified parameters with at least one of an object metadata value, an object duration value, an object status value, and a value contained in contents of an object of the plurality of objects, wherein the program code executable by the processor to cause the apparatus to present the first user interface comprises program code executable by the processor to cause the apparatus to present a dynamic estimate representing how each of the one or more rules impact the system topology;

present a second user interface for specifying, for each rule of the one or more rules, one or more placement constraints, each placement constraint of one or more placement constraints specifying a storage device of the plurality of storage devices in which an object should be placed within the system topology;

create at least one data structure summarizing the one or more rules based on the one or more matching criteria and the one or more placement constraints;

incorporate the one or more rules summarized in the at least one data structure into any previously existing rules to form updated rules;

distribute the updated rules across the system topology;

index metadata for an existing set of objects in the distributed storage grid to identify objects within the system topology which are affected by the updated rules; and

trigger reevaluation of the identified objects to move the identified objects to a different location within the system topology based on the updated rules.

16. The apparatus of claim 15, wherein at least one placement constraint of the one or more placement constraints specifies a number of copies of the object to store in the storage device.

17. The apparatus of claim 15, wherein at least one placement constraint of the one or more placement constraints specifies a time period that the object is retained in the storage device.

18. The apparatus of claim 15, wherein at least one rule of the one or more rules applies to an object in a defined object state.

19. The apparatus of claim 15, wherein the existing set of objects are assigned to a partition, wherein each partition is associated with an isolated administrative domain, and wherein the one or more rules are associated with one or more partitions.

20. The apparatus of claim 15, wherein the program code executable by the processor to cause the apparatus to present the second user interface for specifying the one or more placement constraints comprises program code executable by the processor to cause the apparatus to present an estimated number of objects matching the rule, an estimated amount of storage space consumed by the objects matching the rule, and an estimated amount of storage space available for new objects matching the rule.

21. The apparatus of claim 15, wherein a storage grade is defined for each of the plurality of storage devices, wherein the storage grade comprises a type and a class, wherein at least one placement constraint of the one or more placement constraints specifies the storage grade of the storage device where the object that matches the rule for the at least one placement constraint are to be stored.