Enabling placement control for consistent hashing-based object stores

• US 10,248,678 B2
• Filed: 08/25/2015
• Issued: 04/02/2019
• Est. Priority Date: 08/25/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Hildebrand, Dean, Owen, William W., Rupprecht, Lukas L., Zhang, Rui
• Primary Examiner(s): Gebresenbet, Dinku W
• Application Number:
  US14/835,204
• Publication Number:
  US 20170060865A1
• Time in Patent Office: 1,316 Days
• CPC Class Codes: G06F 16/2255 Hash tables

Abstract

Techniques are disclosed herein for controlling object placement in object storage. A placement component of a storage application receives a request to store a first object in an object store having multiple nodes. The object store determines a placement of the first object to one of the nodes based on an object namespace including a numerical namespace and a lexicographical namespace. Each node is assigned a corresponding subspace of the object namespace for storing objects. The first object includes a numerical namespace value and a lexicographical namespace value. A second object (a replica of the first object) is generated. The first object is stored to a first node based on the lexicographical namespace value. The second object is stored to a second node based on the numerical namespace value.

Claims

1. A method for controlling object placement, comprising:

receiving a first request to store a first object in an object store which includes a plurality of nodes, wherein the object store determines a placement of the first object in one of the plurality of nodes based on an object namespace, wherein the object namespace includes a numerical namespace and a lexicographical namespace, wherein each node is assigned a corresponding subspace of the object namespace for storing objects;

upon determining that the first object specifies both a first numerical namespace value and a first lexicographical namespace value;

generating a second object, wherein the second object is a replica of the first object;

storing the first object in a first node of the plurality of nodes based on the first lexicographical namespace value, wherein the first lexicographical namespace value specifies the first node; and

storing the second object in a second node of the plurality of nodes based on the first numerical namespace value;

receiving a second request to store a third object in the object store; and

upon determining that the third object specifies a second numerical namespace value and does not specify a lexicographical namespace value, storing the third object in a third node of the plurality of nodes based on the second numerical namespace value.

2. The method of claim 1, further comprising:

receiving a third request to retrieve the first object;

identifying a node storing the first object based on the first lexicographical namespace value; and

retrieving the first object from the identified node.

3. The method of claim 2, wherein identifying the node storing the first object is further based on a result of a query of (i) a metadata container associated with the first object and (ii) a name of at least a replica of the first object.

4. The method of claim 1, further comprising:

receiving a third request to delete the first object;

identifying a node storing the first object based on the first lexicographical namespace value; and

removing the first object from the identified node.

5. The method of claim 1, further comprising, prior to generating the second object:

validating the first request using role-based authentication.

6. The method of claim 1, wherein storing the first object in the first node based on the first lexicographical namespace value comprises:

upon determining that a metadata container associated with the first object includes metadata associated with a fourth object having a numerical namespace value that is identical to the first numerical namespace value of the first object, replacing the fourth object with the first object.

7. The method of claim 1, wherein the first numerical namespace value is generated by performing a hash function on the object.

8. A computer program product, comprising:

a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code configured to perform an operation for controlling object placement, comprising;

receiving a first request to store a first object in an object store which includes a plurality of nodes, wherein the object store determines a placement of the first object in one of the plurality of nodes based on an object namespace, wherein the object namespace includes a numerical namespace and a lexicographical namespace, wherein each node is assigned a corresponding subspace of the object namespace for storing objects;

upon determining that the first object specifies both a first numerical namespace value and a first lexicographical namespace value;

generating a second object, wherein the second object is a replica of the first object,

storing the first object in the first node of the plurality of nodes based on the first lexicographical namespace value, wherein the first lexicographical namespace value specifies the first node, and

storing the second object in a second node of the plurality of nodes based on the first numerical namespace value,

receiving a second request to store a third object in the object store; and

upon determining that the third object specifies a second numerical namespace value and does not specify a lexicographical namespace value, storing the third object in a third node of the plurality of nodes based on the second numerical namespace value.

9. The computer program product of claim 8, wherein the operation further comprises:

receiving a third request to retrieve the first object;

identifying a node storing the first object based on the first lexicographical namespace value; and

retrieving the first object from the identified node.

10. The computer program product of claim 9, wherein identifying the node storing the first object is further based on a result of a query of (i) a metadata container associated with the first object and (ii) a name of at least a replica of the first object.

11. The computer program product of claim 8, wherein the operation further comprises:

receiving a third request to delete the first object;

identifying a node storing the first object based on the first lexicographical namespace value; and

removing the first object from the identified node.

12. The computer program product of claim 8, wherein the operation further comprises, prior to generating the second object:

validating the first request using role-based authentication.

13. The computer program product of claim 8, wherein storing the first object in the first node based on the first lexicographical namespace value comprises:

upon determining that a metadata container associated with the first object includes metadata associated with a fourth object having a numerical namespace value that is identical to the first numerical namespace value of the first object, replacing the fourth object with the first object.

14. The computer program product of claim 8, wherein the first numerical namespace value is generated by performing a hash function on the object.

15. A system, comprising:

a processor; and

a memory storing a program, which, when executed on the processor, performs an operation for controlling object placement, the operation comprising;

receiving a first request to store a first object in an object store which includes a plurality of nodes, wherein the object store determines a placement of the first object in one of the plurality of nodes based on an object namespace, wherein the object namespace includes a numerical namespace and a lexicographical namespace, wherein each node is assigned a corresponding subspace of the object namespace for storing objects;

upon determining that the first object specifies both a first numerical namespace value and a first lexicographical namespace value;

generating a second object, wherein the second object is a replica of the first object,

storing the first object in a first node of the plurality of nodes based on the first lexicographical namespace value, wherein the first lexicographical namespace value specifies the first node, and

storing the second object in a second node of the plurality of nodes based on the first numerical namespace value,

receiving a second request to store a third object in the object store; and

upon determining that the third object specifies a second numerical namespace value and does not specify a lexicographical namespace value, storing the third object in a third node of the plurality of nodes based on the second numerical namespace value.

16. The system of claim 15, wherein the operation further comprises:

receiving a third request to retrieve the first object;

identifying a node storing the first object based on the first lexicographical namespace value; and

retrieving the first object from the identified node.

17. The system of claim 16, wherein identifying the node storing the first object is further based on a result of a query of (i) a metadata container associated with the first object and (ii) a name of at least a replica of the first object.

18. The system of claim 15, wherein the operation further comprises:

receiving a third request to delete the first object;

identifying a node storing the first object based on the first lexicographical namespace value; and

removing the first object from the identified node.

19. The system of claim 15, wherein the operation further comprises, prior to generating the second object:

validating the first request using role-based authentication.

20. The system of claim 15, wherein storing the first object in the first node based on the first lexicographical namespace value comprises:

upon determining that a metadata container associated with the first object includes metadata associated with a fourth object having a numerical namespace value that is identical to the first numerical namespace value of the first object, replacing the fourth object with the first object.