Custom object paths for object storage management

• US 10,324,919 B2
• Filed: 10/05/2015
• Issued: 06/18/2019
• Est. Priority Date: 10/05/2015
• Status: Active Grant

• Current Assignee: Red Hat, Inc. (International Business Machines Corporation)
• Original Assignee: Red Hat, Inc. (International Business Machines Corporation)
• Inventors: Chen, Huamin
• Primary Examiner(s): Ho, Binh V
• Application Number:
  US14/875,047
• Publication Number:
  US 20170097952A1
• Time in Patent Office: 1,352 Days
• Field of Search: 707692
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2291 User-Defined Types; Storage management thereof
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/285 Clustering or classification

Abstract

A processing device receives a request that identifies an object to store in a storage system. The request includes a custom path for the object that represents content of the object. The processing device selects one or more other objects in the storage system that has a respective custom path that is similar to the custom path in the request. The one or more other objects that have the respective custom path that is similar to the custom path in the request includes content that is similar to the content of the object that is identified in the request. The processing device determines which data store in the storage system stores the one or more other objects that have the respective custom path that is similar to the custom path in the request, and routes the object identified in the request to a storage node corresponding to the data store to store the object in the data store with the one or more other objects.

Claims

1. A method comprising:

receiving a request that identifies an object to store in a storage system, wherein the request comprises a custom path for the object that represents content of the object, wherein the custom path for the object that is identified in the request comprises a hash value resulting from a hash algorithm;

selecting, by a processing device, one or more other objects in the storage system that have a respective custom path that is similar to the custom path in the request, wherein the one or more other objects having the respective custom path that is similar to the custom path in the request comprises content that is similar to the content of the object that is identified in the request;

determining which data store in the storage system stores the one or more other objects having the respective custom path that is similar to the custom path in the request; and

routing the object identified in the request to a storage node corresponding to the data store to store the object in the data store with the one or more other objects.

2. The method of claim 1, wherein the custom path for the object that is identified in the request comprises the hash value resulting from the hash algorithm being applied to content of the object that is identified in the request.

3. The method of claim 2, wherein the request comprises metadata that identifies the hash algorithm.

4. The method of claim 1, wherein selecting the one or more other objects in the storage system comprises:

extracting a hash value in the custom path of the object that is identified in the request;

searching an index table for one or more custom paths that comprise a similar hash value;

selecting one or more custom paths that comprise a similar hash value; and

identifying the one or more other objects that are associated with the one or more custom paths that are selected.

5. The method of claim 4, wherein determining which data store in the storage system stores the one or more other objects comprises:

determining, from the index table, a storage node identifier associated with the one or more other objects.

6. The method of claim 1, further comprising:

receiving a request to perform deduplication operations on content of a plurality of objects in the storage system;

selecting a data store storing content of a subset of the plurality of objects, wherein the subset of the plurality of objects have similar custom paths; and

performing the deduplication operations on the content of the subset of the plurality of objects, wherein the performing of the deduplication operations comprises analyzing the content of the subset of the plurality of objects that are stored on the data store without analyzing content of other objects of the plurality of objects that are stored in the storage system.

7. The method of claim 1, further comprising:

receiving a request that identifies an object to retrieve from the storage system, wherein the request comprises a custom path for the object to be retrieved, the custom path representing content of the object;

determining which data store in the storage system stores the object in view of the custom path, wherein the data store stores one or more other objects comprising content that is similar to content of the object that is to be retrieved; and

retrieving the object identified in the request from the data store.

8. A non-transitory computer-readable storage medium including instructions that, when executed by a processing device, cause the processing device to:

receive a request that identifies an operation to perform on an object in a storage system, wherein the request comprises a custom path for the object that represents content of the object, wherein the custom path for the object that is identified in the request comprises a hash value resulting from a hash algorithm;

select one or more other objects in the storage system that have a respective custom path that is similar to the custom path in the request;

determine which data store in the storage system stores the one or more other objects having the respective custom path that is similar to the custom path in the request; and

route the object identified in the request to a storage node corresponding to the data store to perform the operation on the object using the data store.

9. The non-transitory computer-readable storage medium of claim 8, wherein the custom path for the object that is identified in the request comprises the hash value resulting from the hash algorithm being applied to content of the object that is identified in the request.

10. The non-transitory computer-readable storage medium of claim 9,

wherein the one or more other objects having the respective custom path that is similar to the custom path in the request comprises content that is similar to the content of the object that is identified in the request.

11. The non-transitory computer-readable storage medium of claim 10, wherein to select the one or more other objects in the storage system, the processing device is to:

extract a hash value in the custom path of the object that is identified in the request;

search an index table for one or more custom paths that comprise a similar hash value;

select the one or more custom paths that comprise a most similar hash value; and

identify the one or more other objects that are associated with the one or more custom paths that are selected.

12. The non-transitory computer-readable storage medium of claim 8, wherein the processing device is further to:

receive a request to perform deduplication operations on content of a plurality of objects in the storage system;

select a data store storing content of a subset of the plurality of objects, wherein the subset of the plurality of objects have similar custom paths; and

perform the deduplication operations on the content of the subset of the plurality of objects, wherein the performing of the deduplication operations comprises analyzing the content of the subset of the plurality of objects that are stored on the data store without analyzing content of other objects of the plurality of objects that are stored in the storage system.

13. The non-transitory computer-readable storage medium of claim 8, wherein the processing device is further to:

receive a request that identifies an object to retrieve from the storage system, wherein the request comprises a custom path for the object to be retrieved, the custom path representing content of the object;

determine which data store in the storage system stores the object in view of the custom path, wherein the data store stores other objects comprising content that is similar to content of the object that is to be retrieved; and

retrieve the object identified in the request from the data store.

14. A system comprising:

a memory;

a processing device, operatively coupled to the memory, to;

receive a request that identifies an object to store in a storage system, wherein the request comprises a custom path for the object that represents content of the object, wherein the custom path for the object that is identified in the request comprises a hash value resulting from a hash algorithm;

select one or more other objects in the storage system that have a respective custom path that is similar to the custom path in the request, wherein the one or more other objects having the respective custom path that is similar to the custom path in the request comprise content that is similar to the content of the object that is identified in the request;

determine which data store in the storage system stores the one or more other objects having the respective custom path that is similar to the custom path in the request; and

route the object identified in the request to a storage node corresponding to the data store to store the object in the data store with the one or more other objects.

15. The system of claim 14, wherein the custom path for the object that is identified in the request comprises the hash value resulting from the hash algorithm being applied to content of the object that is identified in the request.

16. The system of claim 15, wherein the request comprises metadata that identifies the hash algorithm.

17. The system of claim 14, wherein to select the one or more other objects in the storage system, the processing device is to:

extract a hash value in the custom path of the object that is identified in the request;

search an index table for one or more custom paths that comprise a similar hash value;

select the one or more custom paths that comprise a most similar hash value; and

identify the one or more other objects that are associated with the one or more custom paths that are selected.

18. The system of claim 17, wherein to determine which data store in the storage system stores the other object, the processing device is to:

determine, from the index table, a storage node identifier associated with the one or more other objects.

19. The system of claim 14, wherein the processing device is further to:

receive a request to perform deduplication operations on content of a plurality of objects in the storage system;

select a data store storing content of a subset of the plurality of objects, wherein the subset of the plurality of objects have similar custom paths; and

perform the deduplication operations on the content of the subset of the plurality of objects, wherein the performing of the deduplication operations comprises analyzing the content of the subset of the plurality of objects that are stored on the data store without analyzing content of other objects of the plurality of objects that are stored in the storage system.

20. The system of claim 14, wherein the processing device is further to:

receive a request that identifies an object to retrieve from the storage system, wherein the request comprises a custom path for the object to be retrieved, the custom path representing content of the object;

determine which data store in the storage system stores the object in view of the custom path, wherein the data store stores other objects comprising content that is similar to content of the object that is to be retrieved; and

retrieve the object identified in the request from the data store.