Policy-based management of a redundant array of independent nodes

• US 7,155,466 B2
• Filed: 10/27/2004
• Issued: 12/26/2006
• Est. Priority Date: 10/27/2003
• Status: Active Grant

• Current Assignee: Hitachi Vantara, LLC (Hitachi, Ltd.)
• Original Assignee: Archivas, Inc. (Hitachi, Ltd.)
• Inventors: Bernhard, Benjamin K. D., Orenstein, Jack A., Rodriguez, Andres, Shaw, David M.
• Primary Examiner(s): ALAM, SHAHID AL
• Application Number:
  US10/974,443
• Publication Number:
  US 20050120025A1
• Time in Patent Office: 790 Days
• Field of Search: 707/103.R, 707/202, 707/205, 707/2, 707/101, 705/54, 709/244, 709/219, 712/1, 712/16, 712/17, 711/112, 711/203, 711/161, 713/165, 714/6, 714/752
• US Class Current: 707/694
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/162 Delete operations erasing in storage systems G06F3/0652
• CPC Class Codes: G06F 16/181 providing write once read many [WORM] semantics
• CPC Class Codes: G06F 16/1824 implemented using Network-attached Storage [NAS] architecture distributed or networked storage systems G06F3/067; protocols for distributed storage of data in a network H04L67/1097
• CPC Class Codes: G06F 16/185 Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof details of archiving G06F16/11
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99944 Object-oriented database structure
• CPC Class Codes: Y10S 707/99956 File allocation

Abstract

An archive cluster application runs in a distributed manner across a redundant array of independent nodes. Each node preferably runs a complete archive cluster application instance. A given nodes provides a data repository, which stores up to a large amount (e.g., a terabyte) of data, while also acting as a portal that enables access to archive files. Each symmetric node has a set of software processes, e.g., a request manager, a storage manager, a metadata manager, and a policy manager. The request manager manages requests to the node for data (i.e., file data), the storage manager manages data read/write functions from a disk associated with the node, and the metadata manager facilitates metadata transactions and recovery across the distributed database. The policy manager implements one or more policies, which are operations that determine the behavior of an “archive object” within the cluster. The archive cluster application provides object-based storage. Preferably, the application permanently associates metadata and policies with the raw archived data, which together comprise an archive object. Object policies govern the object'"'"'s behavior in the archive. As a result, the archive manages itself independently of client applications, acting automatically to ensure that all object policies are valid.

Claims

1. In a redundant array of independent nodes, wherein each node comprises given hardware on which a given operating system executes, the given hardware including a processor, a data store coupled to the processor, and a local file system, the improvement comprising:

a set of data objects stored in the data store, wherein a given data object is associated with given fixed content data that has been written to the local file system of the node, the given data object encapsulating the given fixed content data together with metadata that includes policy data; and

an application instance stored in the data store and executable by the processor on the given node, comprising;

file system code that provides access to data objects stored in the local file system of the given node;

request manager code that processes requests associated with a given data object;

storage manager code that stores the data objects in the local file system of the node;

metadata manager code that manages the data objects across a set of cooperating nodes including the given node; and

policy manager code that enforces a set of one or more policies against the policy data in the data object'"'"'s metadata during a life cycle of the archive object.

2. The application instance as described in claim 1 wherein the set of one or more policies includes a data protection policy.

3. The application instance as described in claim 1 wherein the set of one or more policies includes an authentication policy.

4. The application instance as described in claim 1 wherein the set of one or more policies includes a retention policy.

5. The application instance as described in claim 1 wherein the set of one or more policies includes a cluster balance policy.

6. The application instance as described in claim 1 wherein the set of one or more policies includes a garbage collection policy.

7. The application instance as described in claim 1 wherein the set of one or more policies includes a scavenging policy.

8. The application instance as described in claim 1 wherein the set of one or more policies includes a duplication elimination policy.

9. The application instance as described in claim 1 wherein the set of one or more policies include one of:

a data protection policy, an authentication policy, a retention policy, a cluster balance policy, a garbage collection policy, a scavenger policy, and a duplicate elimination policy.

10. The application instance as described in claim 9 wherein the policy data associated with the data protection policy identifies a tolerable-points-of-failure (TPOF) value.

11. The application instance as described in claim 9 wherein the policy data associated with the authentication policy identifies a given cryptographic function for use in verifying authenticity of at least a portion of the given fixed content data in the data object.

12. The application instance as described in claim 9 wherein the policy data associated with the authentication policy identifies a run frequency value, and a given cryptographic function for use in periodically verifying authenticity of at least a portion of the given fixed content in the data object according to the run frequency value.

13. The application instance as described in claim 9 wherein the policy data associated with the retention policy identifies a retention period for the given fixed content data.

14. The application instance as described in claim 9 wherein the policy data associated with the cluster balance policy retention policy comprises storage capacity utilization associated with a given node.

15. The application instance as described in claim 9 wherein the policy data associated with the garbage collection policy identifies a run frequency value.

16. The application instance as described in claim 9 wherein the policy data associated with the scavenger policy identifies a run frequency value.

17. A system comprising a redundant array of independent nodes, wherein each node comprises given hardware on which a given operating system executes, the given hardware including a processor, a data store coupled to the processor, and a local file system, comprising:

a set of data objects stored in the data store, wherein a given data object is associated with given fixed content data that has been written to the local file system of the node, the given data object encapsulating the given fixed content data together with metadata that includes policy data; and

an application instance stored in the data store and executable by the processor on the given node, comprising;

file system code that provides access to data objects stored in the local file system of the given node;

request manager code that processes requests associated with a given data object;

storage manager code that stores the data objects in the local file system of the node;

metadata manager code that manages the data objects across a set of cooperating nodes including the given node; and

policy manager code that enforces a set of one or more policies against the policy data in the data object'"'"'s metadata during a life cycle of the data object.

18. The system as described in claim 17 wherein the given hardware on first and second nodes is heterogeneous.

19. The system as described in claim 17 wherein the application instance further includes an administration interface through an entity manages the given node.

20. The system as described in claim 17 wherein the set of one or more policies includes a data protection policy.

21. The system as described in claim 17 wherein the set of one or more policies includes an authentication policy.

22. The system as described in claim 17 wherein the set of one or more policies includes a retention policy.

23. The system as described in claim 17 wherein the set of one or more policies includes a cluster balance policy.

24. The system as described in claim 17 wherein the set of one or more policies includes a garbage collection policy.

25. The system as described in claim 17 wherein the set of one or more policies includes a scavenging policy.

26. The system as described in claim 17 wherein the set of one or more policies includes a duplication elimination policy.

27. The system as described in claim 17 wherein the application instance executing on any given node is the same application instance executing on all other nodes, and wherein given hardware on first and second of the given nodes is heterogeneous.

28. The system as described in claim 17 wherein the file system code provides access to the given fixed content data over an Internet Protocol (IP)-based protocol, being one of:

HTTP, FTP, NFS, AFS, SMB, and a Web service.

29. The system as described in claim 17 wherein the file system code emulates a native file system.