Method for generating universal objects identifiers in distributed multi-purpose storage systems

• US 9,311,135 B2
• Filed: 01/18/2011
• Issued: 04/12/2016
• Est. Priority Date: 01/18/2011
• Status: Active Grant

• Current Assignee: Scality, S.A. (Scality, Inc.)
• Original Assignee: Scality, S.A. (Scality, Inc.)
• Inventors: Regni, Giorgio, Rancurel, Vianney, Tauch, Alain, Artuso, Benoit, Gramain, Jonathan, Demiddelaer, Bertrand
• Primary Examiner(s): Christensen, Scott B
• Application Number:
  US13/008,390
• Publication Number:
  US 20120185555A1
• Time in Patent Office: 1,911 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/137 Hash-based content-based indexing of textual data G06F16/31
• CPC Class Codes: G06F 2209/463 Naming
• CPC Class Codes: G06F 9/465 Distributed object oriented systems remote method invocation [RMI] G06F9/548
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A computer implemented method and system for generating secure universal object identifiers on a multipurpose storage system is disclosed. According to one embodiment, a system comprises a client system in communication with a network. An application server is in communication with the network. A storage cluster is in communication with the network. The storage cluster has a plurality of storage nodes. The client system stores a data object via the application server. The application server generates an object identifier assigned to the data object. The application server stores the data object on a storage node of the plurality of storage nodes. The data object is moved to another application server without moving contents of the data object in the storage cluster.

Claims

1. An apparatus, comprising:

an application server having a processor and a memory, said application server having an interface to couple to a distributed storage system that stores an object and replicas of the object, said application server to generate a key for the object or a replica of the object and pass the key through the interface, said application server comprising program code stored on a computer readable storage medium, said program code to cause the following method to be performed when executed by a processing unit of the application server;

structure the key to include a payload comprising an application object ID, a namespace, an application type and application specific information;

generate for inclusion into the key a dispersion field calculated as;

for a first instance of the object;

a dispersion constant calculated as a hash of the payload and R where R is a revision value of the object that increments with each new revision of the object;

for the first and subsequent replicas of the object;

a modulo operation that divides two terms, a first of the terms including a summation of the dispersion constant keyspace size term, said keyspace size term normalized by a term that increases with the number of replicas, a second of the terms being said keyspace size term;

generate for inclusion into the key a field identifying the number of replicas for the object;

generate for inclusion into the key a replica number specific to the key'"'"'s associated object;

perform a hash of a secret key, the payload and R and incorporate the hash with the key;

wherein, upon the application server passing the key through the interface to the distributed data storage system, the distributed storage system proceeds to;

route the key through the distributed data storage system based on the field that identifies the number of replicas for the object and the replica number that is specific to the key'"'"'s associated object;

access a keyspace location based on the dispersion field;

prevent unwanted access to the object by an entity that does not know the secret key.

2. The apparatus of claim 1 wherein said application server is also to generate a checksum for inclusion into said key.

3. The apparatus of claim 1 wherein said payload can be mapped to any of the following types:

file system;

block service;

email application;

database application.

4. The apparatus of claim 1 wherein said network is a Chord network.

5. An apparatus, comprising:

a distributed storage system; and

,

an application server having a processor and a memory, the application server having an interface to couple to the distributed storage system that stores an object and replicas of the object, said application server to generate a key for the object or a replica of the object and pass the key through the interface, said application server comprising program code stored on a computer readable storage medium, said program code to cause the following method to be performed when executed by a processing unit of the application server;

structure the key to include a payload comprising an application object ID, a namespace, an application type and application specific information;

generate for inclusion into the key a dispersion field calculated as;

for a first instance of the object;

a dispersion constant calculated as a hash of the payload and R where R is a revision value of the object that increments with each new revision of the object;

for the first and subsequent replicas of the object;

a modulo operation that divides two terms, a first of the terms including a summation of the dispersion constant keyspace size term, said keyspace size term normalized by a term that increases with the number of replicas, a second of the terms being said keyspace size term;

generate for inclusion into the key a field identifying the number of replicas for the object;

generate for inclusion into the key a replica number specific to the key'"'"'s associated object;

perform a hash of a secret key, the payload and R and incorporate the hash with the key;

wherein, upon the application server passing the key through the interface to the distributed data storage system, the distributed storage system proceeds to;

route the key through the distributed data storage system based on the field that identifies the number of replicas for the object and the replica number that is specific to the key'"'"'s object;

access a keyspace location based on the dispersion field;

prevent unwanted access to the object by an entity that does not know the secret key.

6. The apparatus of claim 5 wherein said application server is also to generate a checksum for inclusion into said key.

7. The apparatus of claim 5 wherein said payload can be mapped to any of the following types:

file system;

block service;

email application;

database application.

8. The apparatus of claim 5 wherein said network is a Chord network.

9. A non-transitory machine readable storage medium having stored thereon program code that when processed by a processing unit of a computing system causes the computing system to perform a method, comprising:

structuring the key to include a payload comprising an application object ID, a namespace, an application type and application specific information, the key for accessing an object from a distributed object storage system that stores the object and replicas of the object;

generating for inclusion into the key a dispersion field calculated as;

for a first instance of the object;

a dispersion constant calculated as a hash of the payload and R where R is a revision value of the object that increments with each new revision of the object;

for the first and subsequent replicas of the object;

a modulo operation that divides two terms, a first of the terms including a summation of the dispersion constant keyspace size term, said keyspace size term normalized by a term that increases with the number of replicas, a second of the terms being said keyspace size term;

generating for inclusion into the key a field identifying the number of replicas for the object;

generating for inclusion into the key a replica number specific to the key'"'"'s associated object;

performing a hash of a secret key, the payload and R and incorporate the hash with the key;

forwarding the key toward an interface to the distributed data storage system wherein unwanted access to the object is prevented by an entity that does not know the secret key.

10. The non-transitory machine readable storage medium of claim 9 wherein said application server is also to generate a checksum for inclusion into said key.

11. The non-transitory machine readable storage medium of claim 9 wherein said payload can be mapped to any of the following types:

file system;

block service;

email application;

database application.

12. The non-transitory machine readable storage medium of claim 9 wherein said network is a Chord network.

14. The method of claim 9 wherein said application server is also to generate a checksum for inclusion into said key.

15. The method of claim 9 wherein said payload can be mapped to any of the following types:

file system;

block service;

email application;

database application.

16. The method of claim 9 wherein said network is a Chord network.

13. A method, comprising:

structuring a key to include a payload comprising an application object ID, a namespace, an application type and application specific information, the key for accessing an object or a replica of the object from a distributed object storage system that stores the object and replicas of the object;

generating for inclusion into the key a dispersion field calculated as;

for a first instance of the object;

a dispersion constant calculated as a hash of the payload and R where R is a revision value of the object that increments with each new revision of the object;

for the first and subsequent replicas of the object;

a modulo operation that divides two terms, a first of the terms including a summation of the dispersion constant and a keyspace size term, said keyspace size term normalized by a term that increases with the number of replicas, a second of the terms being said keyspace size term;

generating for inclusion into the key a field identifying the number of replicas for the object;

generating for inclusion into the key a replica number specific to the key'"'"'s associated object;

performing a hash of a secret key, the payload and R and incorporate the hash with the key; and

,

forwarding the key toward an interface to the distributed data storage system wherein unwanted access to the object is prevented by an entity that does not know the secret key.