Content repository implemented in a network storage server system

• US 8,180,813 B1
• Filed: 12/08/2009
• Issued: 05/15/2012
• Est. Priority Date: 12/08/2009
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Goodson, Garth, Kavuri, Ravi, Voruganti, Kaladhar, Srinivasan, Sudhir, Pasupathy, Shankar, Niranjan, Thirumale
• Primary Examiner(s): Hasan, S H
• Application Number:
  US12/633,718
• Time in Patent Office: 889 Days
• Field of Search: None
• US Class Current: 707/827
• CPC Class Codes: G06F 16/907 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually

Abstract

A network storage server system includes a distributed object store, a presentation layer, a metadata subsystem, and a content management subsystem. The object store has no namespace and provides location-independent addressing of data objects. The presentation layer provides multiple interfaces for accessing data stored in the object store, including a NAS interface and a Web Service interface, and provides at least one namespace for accessing data via the NAS interface or the Web Service interface. The Web Service interface allows access to stored data via the namespace or without using the namespace (“raw object” access). The metadata subsystem stores user-specified and/or system-generated metadata relating to data objects and allows data objects to be identified and retrieved by searching on the metadata. The content management subsystem autonomously manages lifecycles of data objects according to user-specified policies, based on metadata associated with the data objects and tracked by the metadata subsystem.

Claims

1. A network storage system comprising:

a distributed object store to store a set of data objects, the distributed object store implemented in one or more network nodes and configured to provide location-independent addressing of data objects;

a presentation module configured to generate a presentation layer functionally over the distributed object store and configured to provide a NAS interface and a Web Service interface, to allow access to data objects stored in the distributed object store;

a metadata subsystem configured to store metadata relating to data objects stored in the distributed object store, including user-defined tags, and to allow data objects to be identified and retrieved by searching the metadata including the user-defined tags in response to user-specified search queries;

a policy-based content management subsystem configured to manage data objects stored in the distributed object store according to user-specified policies, based on stored metadata associated with the data objects; and

an object location subsystem configured to locate data objects stored in the network storage system in response to requests from storage clients, by using a mapping structure that represents mappings between location-independent object identifiers of the data objects and corresponding location identifiers of the plurality of data objects.

2. A network storage system as recited in claim 1, wherein the distributed object store has no namespace.

3. A network storage system as recited in claim 2, wherein the presentation layer provides a namespace for use in accessing data via the NAS interface and in accessing data via the Web Service interface.

4. A network storage system as recited in claim 3, wherein the Web Service interface is configured to provide access to data stored in the distributed object store via the namespace and is also configured to provide access to data stored in the distributed object store without using the namespace.

5. A network storage system as recited in claim 1, wherein the distributed object store comprises a plurality of single-node object stores.

6. A network storage system as recited in claim 1, wherein the distributed object store has an object address space and is configurable to span the object address space across a plurality of storage systems spread over a geographic region.

7. A network storage system as recited in claim 1, wherein the distributed object store is such that objects can be moved among storage locations without a change in their addressing.

8. A network storage system as recited in claim 1, wherein the metadata subsystem is configured to store metadata specified by users and metadata generated by the network storage system.

9. A network storage system as recited in claim 1, wherein the policy-based content management subsystem is configured to execute policies relating to system performance, data protection and data security.

10. A network storage system as recited in claim 1, wherein the policy-based content management subsystem is configured to autonomously manage lifecycles of data objects by autonomously determining and executing data migration and deletion actions based on the user-specified policies.

11. A network storage system as recited in claim 1, wherein the policy-based content management subsystem is configured to use metadata tracked by the metadata subsystem to identify data objects upon which to act.

12. A storage server node configured for use in a storage server cluster that includes a plurality of storage server nodes, the storage server node comprising:

a D-module configured to provide backend storage and retrieval of data objects in a nonvolatile mass storage facility, the D-module configured to implement a portion of a distributed object store that is distributed across a plurality of interconnected storage server nodes, wherein the distributed object store includes a plurality of single-node object stores, and wherein the distributed object store provides location-independent addressing of data objects;

an N-module operatively coupled to the D-module and configured to provide network communication services to communicate with a storage client over a network, the N-module configured to communicate with a D-module in each of the plurality of storage server nodes in the cluster, the N-module configured to provide a presentation layer that operates logically over the distributed object store and which presents a plurality of separate interfaces for accessing a set of stored data objects;

an M-host module operatively coupled to the N-module and including a metadata subsystem which stores metadata relating to data objects, the metadata including user-defined tags, and which allows data objects to be identified and retrieved by searching the metadata including the user-defined tags in response to user-specified search queries, the host module further including a policy-based content management subsystem which autonomously manages data objects stored in the distributed object store, according to user-specified policies and based on metadata associated with the data objects; and

an object location subsystem configured to locate data objects stored in the network storage system in response to requests from storage clients, by using a mapping structure that represents mappings between location-independent object identifiers of the data objects and corresponding location identifiers of the plurality of data objects.

13. A storage server node as recited in claim 12, wherein the distributed object store has no namespace.

14. A storage server node as recited in claim 12, wherein the plurality of separate interfaces presented by the presentation layer include a NAS interface and a Web Service interface.

15. A storage server node as recited in claim 14, wherein the presentation layer includes a namespace for use in accessing data via the NAS interface and in accessing data via the Web Service interface, and wherein the Web Service interface is configured to provide access to data stored in the distributed object store via the namespace and is also configured to provide access to data stored in the distributed object store without using the namespace.

16. A storage server node as recited in claim 12, wherein the distributed object store has an object address space and has an ability to span the object address space across a plurality of storage systems spread over a geographic region.

17. A storage server node as recited in claim 12, wherein the distributed object store is such that objects can be moved among storage locations without a change in their addressing.

18. A storage server node as recited in claim 12, wherein the metadata subsystem is configured to store metadata specified by users and metadata generated by the network storage system.

19. A storage server node as recited in claim 12, wherein the policy-based content management subsystem is configured to execute policies relating to system performance, data protection and data security.

20. A storage server node as recited in claim 12, wherein the policy-based content management subsystem is configured to autonomously manage lifecycles of data objects by autonomously determining and executing data migration and deletion actions based on the user-specified policies.

21. A storage server node as recited in claim 12, wherein the policy-based content management subsystem is configured to use metadata tracked by the metadata subsystem to identify data objects upon which to act.

22. A network storage server system comprising:

a processor;

a network interface through which to communicate with a plurality of storage clients over a network;

a storage interface through which to communicate with a nonvolatile mass storage subsystem; and

a memory storing code which, when executed by the processor, causes the processor to perform a plurality of operations, including

implementing at least a portion of a distributed object store to store data objects, the object store distributed across one or more storage server nodes and including a plurality of single-node object stores, including providing location-independent addressing of data objects in the distributed object store such that objects can be moved among storage locations without a change in their addressing, wherein the distributed object store has no namespace;

providing a presentation layer functionally layered over the distributed object store, the presentation layer accessible by users via each of a plurality of different interfaces, including a NAS interface and a Web Service interface;

storing metadata including user-defined tags relating to data objects in the distributed object store and allowing data objects to be identified and retrieved by searching on the metadata including the user-defined tags in response to user-specified search queries;

managing data objects stored in the distributed object store according to user-specified policies, based on metadata associated with the data objects; and

providing an object location service to locate data objects stored in the network storage system in response to requests from storage clients, by using a mapping structure that represents mappings between location-independent object identifiers of the data objects and corresponding location identifiers of the plurality of data objects.

23. A network storage server system as recited in claim 22, wherein the distributed object store has an object address space and has an ability to span the object address space across a plurality of storage systems spread over a geographic region.

24. A network storage server system as recited in claim 22, wherein the presentation layer provides a namespace for use in accessing data via the NAS interface and in accessing data via the Web Service interface, and wherein the Web Service interface is configured to provide access to data stored in the distributed object store via the namespace and is also configured to provide access to data stored in the distributed object store without using the namespace.

25. A network storage server system as recited in claim 22, wherein the policy-based content management subsystem is configured to execute policies relating to system performance, data protection and data security.

26. A network storage server system as recited in claim 22, wherein managing lifecycles of data objects comprises autonomously determining and executing data migration and deletion actions based on the user-specified policies.

27. A network storage server system as recited in claim 26, wherein managing lifecycles of data objects comprises using the metadata to identify data objects upon which to act.

28. A method of operating a network storage system, the method comprising:

implementing at least a portion of an object store to store data objects in the network storage system, the object store distributed across a plurality of interconnected storage server nodes, including providing location-independent addressing of data objects in the object store such that objects can be moved among storage locations without a change in their addressing;

providing a presentation layer functionally over the object store, the presentation layer accessible by users via each of a plurality of different interfaces, including a NAS interface and a Web Service interface;

storing metadata including user-defined tags relating to data objects in the object store and allowing data objects to be identified and retrieved by searching on the metadata including the user-defined tags in response to user-specified search queries;

managing lifecycles of data objects stored in the object store according to user-specified policies, based on the metadata; and

providing an object location service to locate data objects stored in the network storage system in response to requests from storage clients, by using a mapping structure that represents mappings between location-independent object identifiers of the data objects and corresponding location identifiers of the plurality of data objects.

29. A method as recited in claim 28, wherein the object store has an object address space and has an ability to span the object address space across other storage systems spread over a geographic region.

30. A method as recited in claim 29, wherein providing the presentation layer comprises providing a namespace for use in accessing data via the NAS interface and in accessing data via the Web Service interface, and wherein the Web Service interface is configured to provide access to data stored in the object store via the namespace and is also configured to provide access to data stored in the object store without using the namespace.

31. A method as recited in claim 28, wherein the object store has no namespace.

32. A method as recited in claim 28, wherein managing lifecycles of data objects comprises autonomously determining and executing data migration and deletion actions based on the user-specified policies.

33. A method as recited in claim 32, wherein managing lifecycles of data objects comprises using the metadata to identify data objects upon which to act.