Object location service for network-based content repository

• US 8,832,154 B1
• Filed: 12/08/2009
• Issued: 09/09/2014
• Est. Priority Date: 12/08/2009
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Srinivasan, Sudhir, Pasupathy, Shankar, Shao, Minglong, Goodson, Garth
• Primary Examiner(s): Madamba, Glenford
• Application Number:
  US12/633,745
• Time in Patent Office: 1,736 Days
• Field of Search: 709/219
• US Class Current: 707/796
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A distributed object store in a network storage system uses location-independent global object identifiers (IDs) for stored data objects. The global object ID enables a data object to be seamlessly moved from one location to another without affecting clients of the storage system, i.e., “transparent migration”. The global object ID can be part of a multilevel object handle, which also can include a location ID indicating the specific location at which the data object is stored, and a policy ID identifying a set of data management policies associated with the data object. The policy ID may be associated with the data object by a client of the storage system, for example when the client creates the object, thus allowing “inline” policy management. An object location subsystem (OLS) can be used to locate an object when a client request does not contain a valid location ID for the object.

Claims

1. A method comprising:

in a network storage server system, storing a plurality of data objects which are sub-volume level data objects;

in the network storage server system, providing a layer of indirection between storage locations of the data objects and at least one of

path names of the data objects, or

object identifiers of the data objects, so that the path names and/or object identifiers of the data objects are independent of the storage locations of the data objects, wherein providing the layer of indirection includes;

implementing, in the network storage system, a first mapping structure that represents mappings between location-independent object identifiers of the plurality of data objects and corresponding location identifiers of the plurality of data objects; and

implementing, in the network storage server system, a path namespace associated with the plurality of data objects, wherein the path namespace includes a second mapping structure that represents a mapping between the path names of the plurality of data objects and corresponding location-independent object identifiers of the plurality of data objects;

wherein the path namespace in conjunction with the first mapping structure provides a multilevel mapping between path names of the plurality of data objects and corresponding location identifiers of the plurality of data objects;

in the network storage server system,

receiving from one of a plurality of storage clients a request that includes a path name of a particular data object of the plurality of data objects;

using the path name to look up an entry in the path namespace; and

retrieving from said entry in the path namespace the location-independent object identifier of the particular data object;

locating the particular data object by using the location-independent object identifier of the particular data object to look up a location identifier of the particular data object in the first mapping structure, wherein the first mapping structure is a multilevel hierarchical structure, and wherein using the location-independent object identifier of the particular data object to look up a location identifier of the particular data object in the multilevel hierarchical structure comprises;

using a first portion of the location-independent object identifier of the data object to identify a first entry in a first level of the multilevel hierarchical structure;

using a second portion of the location-independent object identifier of the data object to identify a second entry in a second level of the multilevel hierarchical structure, the second entry being descendent from the first entry in the multilevel hierarchical structure; and

retrieving the location identifier of the particular data object from the second entry.

2. A method as recited in claim 1, further comprising:

using the path namespace in conjunction with the mapping structure to associate a plurality of different path names with a particular data object of the plurality of data objects.

3. A method comprising:

in a network storage server system, storing a plurality of data objects in a distributed object store which includes a plurality of individual object stores;

implementing in the network storage system a first mapping structure that represents mappings between location-independent object identifiers of the plurality of data objects and corresponding location identifiers of the plurality of data objects;

receiving from one of a plurality of storage clients a request that includes a path name of a particular data object of the plurality of data objects;

using the path name to look up an entry in the path namespace; and

retrieving from said entry in the path namespace a location-independent object identifier of the particular data object;

locating the particular data object by using the location-independent object identifier of the particular data object to look up a location identifier of the particular data object in the first mapping structure,

wherein the first mapping structure is a multilevel hierarchical structure, wherein using the location-independent object identifier of the particular data object to look up a location identifier of the particular data object in the multilevel hierarchical structure comprises;

using a first portion of the location-independent object identifier of the data object to identify a first entry in a first level of the multilevel hierarchical structure;

using a second portion of the location-independent object identifier of the data object to identify a second entry in a second level of the multilevel hierarchical structure, the second entry being descendent from the first entry in the multilevel hierarchical structure; and

retrieving the location identifier of the particular data object from the second entry.

4. A method as recited in claim 3, wherein the data objects are sub-volume level data objects.

5. A method as recited in claim 3, further comprising:

implementing in the network storage server system a path namespace associated with the plurality of data objects, wherein path names in the path namespace are independent of the first mapping structure, the path namespace having a second mapping structure that represents a mapping between path names of the plurality of data objects and corresponding location-independent object identifiers of the plurality of data objects, wherein the path namespace in conjunction with the first mapping structure provides a multilevel mapping between path names of the plurality of data objects and corresponding location identifiers of the plurality of data objects.

6. A method as recited in claim 5, further comprising:

using the path namespace in conjunction with the mapping structure to associate a plurality of different path names with a particular data object of the plurality of data objects.

7. A storage server node configured for use in a storage server cluster that includes a plurality of storage server nodes, the storage server node comprising:

a D-module configured to provide backend storage and retrieval of a plurality data objects in a nonvolatile mass storage facility; and

an N-module configured to provide network communication services to communicate with a storage client over a network, the N-module configured to communicate with the D-module through an interconnect;

the storage server node further configured with the ability to provide a layer of indirection between storage locations of the data objects and at least one of;

path names of the data objects, or

object identifiers of the data objects,

so that the path names and/or object identifiers of the data objects are independent of the storage locations of the data objects,

wherein the storage server node is configured to provide the layer of indirection by;

implementing in the network storage system a mapping structure that represents mappings between location-independent object identifiers of the plurality of data objects and corresponding location identifiers of the plurality of data objects; and

implementing in the network storage server system a path namespace associated with the plurality of data objects, wherein the path namespace includes a second mapping structure that represents a mapping between the path names of the plurality of data objects and corresponding location-independent object identifiers of the plurality of data objects,

wherein the path namespace in conjunction with the first mapping structure provides a multilevel mapping between path names of the plurality of data objects and corresponding location identifiers of the plurality of data objects,

wherein the storage server node is further configured with the ability to

receive from one of a plurality of storage clients a request that includes a path name of a particular data object of the plurality of data objects,

use the path name to look up an entry in the path namespace;

retrieve from said entry in the path namespace the location-independent object identifier of the particular data object; and

further

locate the particular data object by using the location-independent object identifier of the particular data object to look up a location identifier of the particular data object in the mapping structure,

wherein the mapping structure is a multilevel hierarchical structure, and wherein using the location-independent object identifier of the particular data object to look up a location identifier of the particular data object in the multilevel hierarchical structure comprises;

using a first portion of the location-independent object identifier of the data object to identify a first entry in a first level of the multilevel hierarchical structure;

using a second portion of the location-independent object identifier of the data object to identify a second entry in a second level of the multilevel hierarchical structure, the second entry being descendent from the first entry in the multilevel hierarchical structure; and

retrieving the location identifier of the particular data object from the second entry.

8. A storage server node as recited in claim 7, wherein the data objects are sub-volume level data objects.

9. A storage server node as recited in claim 8, wherein the storage server node is further configured with the ability to use the path namespace in conjunction with the mapping structure to associate a plurality of different path names with a particular data object of the plurality of data objects.

10. A network storage server system comprising:

a processor;

a network interface through which to communicate with a plurality of storage clients over a network;

a storage interface through which to communicate with a nonvolatile mass storage subsystem; and

a memory storing code which, when executed by the processor, causes the processor to perform a plurality of operations, including

storing a plurality of data objects which are sub-volume level data objects;

implementing a mapping structure that represents mappings between location-independent object identifiers of the plurality of data objects and corresponding location identifiers of the plurality of data objects;

providing a layer of indirection, in response to receiving from one of a plurality of storage clients a request that includes a path name of a particular data object of the plurality of data objects, by

using the path name to look up an entry in the path namespace;

retrieving from said entry in the path namespace the location-independent object identifier of the particular data object; and

further by;

locating the particular data object by using the location-independent object identifier of the particular data object to look up a location identifier of the particular data object in the first mapping structure, wherein the first mapping structure is a multilevel hierarchical structure, wherein using the location-independent object identifier of the particular data object to look up a location identifier of the particular data object in the multilevel hierarchical structure comprises;

using a first portion of the location-independent object identifier of the data object to identify a first entry in a first level of the multilevel hierarchical structure;

using a second portion of the location-independent object identifier of the data object to identify a second entry in a second level of the multilevel hierarchical structure, the second entry being a descendent of the first entry in the multilevel hierarchical structure; and

retrieving the location identifier of the particular data object from the second entry.

11. A network storage server system as recited in claim 10, further comprising:

implementing a path namespace associated with the plurality of data objects, wherein the path namespace includes a second mapping structure that represents a mapping between the path names of the plurality of data objects and corresponding location-independent object identifiers of the plurality of data objects, wherein the path namespace in conjunction with the first mapping structure provides a multilevel mapping between path names of the plurality of data objects and corresponding location identifiers of the plurality of data objects.

12. A network storage server system as recited in claim 10, further comprising:

using the path namespace in conjunction with the mapping structure to associate a plurality of different path names with a particular data object of the plurality of data objects.

13. A method comprising:

in a network storage server system, storing a plurality of data objects that are sub-volume level data objects;

in the network storage server system, providing a layer of indirection between storage locations of the data objects and object identifiers of the data objects, such that object identifiers of the data objects are independent of the storage locations of the data objects, and

wherein providing the layer of indirection includes implementing a multilevel hierarchical mapping structure that represents a mapping between the object identifiers of the data objects and corresponding location identifiers of the data objects,

in the network storage server system, further comprising

receiving from one of a plurality of storage clients a request that includes a path name of a particular data object of the plurality of data objects;

using the path name to look up an entry in the path namespace; and

retrieving from said entry in the path namespace the object identifier of the particular data object;

locating the particular data object by using an object identifier of the particular data object to look up a location identifier of the particular data object in the multilevel hierarchical mapping structure,

wherein using the object identifier of the particular data object to look up the location identifier of the particular data object in the multilevel hierarchical mapping structure includes;

using a first portion of the object identifier of the data object to identify a first entry in a first level of the multilevel hierarchical mapping structure;

using a second portion of the object identifier of the data object to identify a second entry in a second level of the multilevel hierarchical mapping structure, the second entry being descendent from the first entry in the multilevel hierarchical mapping structure; and

retrieving the location identifier of the particular data object from the second entry.

14. The method as recited in claim 13, further comprising:

in the network storage server system, providing a second layer of indirection between storage locations of the data objects and path names of the data objects, so that path names of the data objects are independent of the storage locations of the data objects, wherein providing the second layer of indirection includes;

implementing, in the network storage server system, a path namespace associated with the data objects, wherein the path namespace includes a mapping structure that represents a mapping between the path names of the plurality of data objects and corresponding object identifiers of the plurality of data objects;

wherein the path namespace in conjunction with the multilevel hierarchical mapping structure provides a multilevel mapping between path names of the plurality of data objects and corresponding location identifiers of the plurality of data objects.

15. The method as recited in claim 14, further comprising:

using the path namespace in conjunction with the multilevel hierarchical mapping structure to associate a plurality of different path names with a particular data object of the plurality of data objects.