Consistent ring namespaces facilitating data storage and organization in network infrastructures

• US 10,496,627 B2
• Filed: 04/14/2017
• Issued: 12/03/2019
• Est. Priority Date: 04/06/2012
• Status: Active Grant

• Current Assignee: Exablox Corp. (StorageCraft Technology Corporation)
• Original Assignee: StorageCraft Technology Corporation
• Inventors: Hardin, Charles, Subramaniam, Sridhar, Hunt, Tad, Barrus, Frank E.
• Primary Examiner(s): Bates, Kevin T
• Assistant Examiner(s): Scott, Mark A
• Application Number:
  US15/488,303
• Publication Number:
  US 20170220614A1
• Time in Patent Office: 963 Days
• CPC Class Codes: G06F 16/2255 Hash tables
• CPC Class Codes: G06F 16/2282 Tablespace storage structures; Management thereof
• CPC Class Codes: H04L 61/457 containing identifiers of data entities on a computer, e.g. file names
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Computer-implemented methods and systems for managing data objects within a computer network infrastructure that facilitate more efficient and reliable data storage and access are provided. An exemplary method may comprise establishing a physical identifier for each storage resource. A plurality of unique virtual identifiers for each storage resource is generated based on the physical identifier. The plurality of unique virtual identifiers is stored in a consistent ring namespace accessible to every node. An object identifier associated with a location of a data object to be stored in a storage resource is generated. The object identifier is mapped to the consistent ring namespace. The method may also include enabling traversing the consistent ring namespace from any node in the computer network to locate and access the data object.

Claims

1. A method for managing data objects within a computer network infrastructure, the method comprising:

identifying a computer network infrastructure comprising a plurality of nodes, each node comprising one or more tangible storage resources;

establishing a unique physical identifier for each of the tangible storage resources;

generating multiple unique virtual identifiers for each of the tangible storage resources;

storing the multiple unique virtual identifiers in a consistent ring namespace accessible for every one of the nodes;

generating an object identifier associated with a location of a data object stored in one of the tangible storage resources;

mapping the object identifier to the consistent ring namespace in a position in the consistent ring namespace between, in a ring-walk order, the unique virtual identifier of the one of the tangible storage resources and another unique virtual identifier of the multiple unique virtual identifiers stored in the consistent ring namespace; and

enabling traversing of the consistent ring namespace from any of the nodes in the computer network infrastructure in order to locate and access the data object.

2. The method of claim 1, wherein the unique physical identifier is a hash value generated from:

a universally unique storage resource identifier;

or

a combination of the storage resource manufacturing model and a serial number.

3. The method of claim 1, wherein:

the unique virtual identifiers are generated to achieve a specific distribution either manually or programmatically; and

the generating of the multiple unique virtual identifiers comprises applying hash chaining to the physical identifier associated with the tangible storage resource to generate an arbitrary number of unique virtual identifiers associated with an individual tangible storage resource.

4. The method of claim 1, wherein the generating of the multiple unique virtual identifiers is performed in such a way that an arbitrary number of unique virtual identifiers is mapped over the consistent ring namespace.

5. The method of claim 1, wherein the number of generated unique virtual identifiers is the same or greater than the number of the tangible storage resources in the computer network infrastructure.

6. The method of claim 1, wherein the object identifier is a hash value generated from the data object.

7. The method of claim 1, wherein the tangible storage resource of a data object replica is written based upon the location of the mapped virtual identifiers in the consistent ring namespace.

8. The method of claim 7, further comprising generating multiple data object replicas and storing them in different ones of the tangible storage resources by walking the ring namespace to find different virtual identifiers.

9. The method of claim 8, wherein the tangible storage resources are selected based upon a preference to use different nodes in the computer network infrastructure for the data object replicas.

10. The method of claim 8, wherein the tangible storage resources are selected based upon a preference to use different tangible storage resources in the same node for the data object replicas.

11. The method of claim 8, wherein the tangible storage resources are skipped in the order of the ring walk when a data object replica cannot successfully be stored.

12. The method of claim 7, further comprising generating a plurality of data object replicas and storing them in different tangible storage resources.

13. The method of claim 12, wherein a number of data object replicas corresponds to a replication factor based on the number of tangible storage resources or policy requirements of the computer network infrastructure.

14. The method of claim 1, wherein the enabling of traversing of the consistent ring namespace comprises navigating, through the consistent ring namespace, to storage resource identifiers in order to determine a tangible storage resource to retrieve the data object.

15. The method of claim 1, further comprising re-mapping unique virtual identifiers within the consistent ring namespace when a tangible storage resource is added to the computer network infrastructure.

16. The method of claim 1, further comprising re-mapping unique virtual identifiers within the consistent ring namespace when available capacity of any of the tangible storage resources is changed.

17. The method of claim 1, further comprising re-distributing data objects to different ones of the tangible storage resources after the consistent ring namespace has been changed in view of addition or removal of at least one node or at least one tangible storage resource.

18. The method of claim 17, wherein when one of the tangible storage resources is removed from the computer network infrastructure, the data objects are copied in ring walk order to subsequent tangible storage resources still available in the computer network infrastructure.

19. The method of claim 17, wherein re-distribution of data objects can exceed a replication factor to use spare capacity of tangible storage resources in the computer network infrastructure for performance and reconstruction improvements.

20. A non-transitory processor-readable medium having instructions stored thereon, which when executed by one or more processors, causes the one or more processors to perform the method of claim 1.