SCALABLE VIRTUAL NAMESPACE AND SEMANTIC POLICY MANAGEMENT FOR OBJECT STORAGE

• US 20170124138A1
• Filed: 11/03/2015
• Published: 05/04/2017
• Est. Priority Date: 11/03/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Mandagere, Nagapramod S., Saxena, Mohit
• Granted Patent: US 10,127,266 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/2291 User-Defined Types; Storage management thereof
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

In one embodiment, a computer-implemented method for object-granular policy assignment in an object storage environment includes: automatically assigning a plurality of objects stored within the object storage environment to one or more policies; and locating one or more of the objects among a plurality of policy rings within the object storage environment based on semantics of the one or more policies. A virtual namespace maps each object to at least one of the policy rings based on the one or more policies assigned to the object; and the virtual namespace is scalable with a number of the plurality of objects stored within the object storage environment. Corresponding systems and computer program products are also disclosed.

Claims

1. A computer-implemented method for object-granular policy assignment in an object storage environment, the method comprising:

automatically assigning a plurality of objects stored within the object storage environment to one or more policies; and

locating one or more of the objects among a plurality of policy rings within the object storage environment based on semantics of the one or more policies,

wherein a virtual namespace maps each object to at least one of the policy rings based on the one or more policies assigned to the object; and

wherein the virtual namespace is scalable with a number of the plurality of objects stored within the object storage environment.

2. The computer implemented method of claim 1, comprising assigning a different of the one or more policies to one or more of the objects without modifying an externally client-visible namespace of the object.

3. The computer implemented method of claim 1, wherein the locating uses one or more probabilistic data structures.

4. The computer implemented method of claim 3, wherein the one or more probabilistic data structures are characterized by a 100% recall rate.

5. The computer implemented method of claim 3, wherein the one or more probabilistic data structures comprise one or more Bloom filters.

6. The computer implemented method of claim 3, comprising:

using a semantic policy manager to define a scan order for the one or more probabilistic data structures based on semantics of the one or more policies of the object storage environment, the semantics comprising one or more of availability characteristics and performance characteristics.

7. The computer implemented method of claim 3, comprising using a semantic policy manager to define a size of each probabilistic data structure based on a policy for which the probabilistic data structure is used within the object storage environment.

8. The computer implemented method of claim 3, comprising using a semantic policy manager to:

optimize a false positive rate for each of the probabilistic data structures, each of the probabilistic data structures being optimized to have a different false positive rate; and

define a scan order for the one or more probabilistic data structures based on the different false positive rates.

9. The computer implemented method of claim 1, comprising using a semantic policy manager to:

determine one or more of availability characteristics and performance characteristics of a plurality of physical storage devices within the object storage environment upon which the plurality of objects are stored;

determine at least one differential between the availability characteristics and/or the performance characteristics for the plurality of physical storage devices; and

define a plurality of probabilistic data structures having different false positive rates based on the at least one differential, each probabilistic data structure corresponding to at least one of the policies.

10. A computer program product for object-granular policy assignment in an object storage environment, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instructions executable by an object storage system to cause the object storage system to perform a method comprising:

automatically assigning a plurality of objects stored within the object storage environment to one or more policies; and

locating one or more of the objects among a plurality of policy rings within the object storage environment based on semantics of the one or more policies;

wherein a virtual namespace maps each object to at least one of the policy rings based on the one or more policies assigned to the object; and

wherein the virtual namespace is scalable with a number of the plurality of objects stored within the object storage environment.

11. The computer program product of claim 10, wherein the locating uses one or more probabilistic data structures.

12. The computer program product of claim 11, wherein the one or more probabilistic data structures are characterized by a 100% recall rate.

13. The computer program product of claim 11, wherein the one or more probabilistic data structures comprise one or more Bloom filters.

14. The computer program product of claim 11, comprising program instructions executable by an object storage system to cause a semantic policy manager of the object storage system to define a scan order for the one or more probabilistic data structures based on semantics of the one or more policies of the object storage environment, the semantics comprising one or more of availability characteristics and performance characteristics.

15. The computer program product of claim 11, comprising program instructions executable by an object storage system to cause a semantic policy manager of the object storage system to:

define a size of each probabilistic data structure based on a policy for which the probabilistic data structure is used within the object storage environment.

16. The computer program product of claim 11, comprising program instructions executable by an object storage system to cause a semantic policy manager of the object storage system to:

optimize a false positive rate for each of the probabilistic data structures using the semantic policy manager, each of the probabilistic data structures being optimized to have a different false positive rate; and

define a scan order for the one or more probabilistic data structures based on the different false positive rates.

17. The computer program product of claim 10, comprising program instructions executable by the object storage environment to cause a semantic policy manager of the object storage environment to:

assign a different of the one or more policies to one or more of the objects without modifying a physical storage location of the object.

18. The computer program product of claim 10, comprising program instructions executable by the object storage environment to cause a semantic policy manager of the object storage environment to:

determine one or more of availability characteristics and performance characteristics of a plurality of physical storage devices within the object storage environment upon which the plurality of objects are stored;

determine at least one differential between the availability characteristics and/or the performance characteristics for the plurality of physical storage devices; and

define a plurality of probabilistic data structures having different false positive rates based on the at least one differential, each probabilistic data structure corresponding to at least one of the policies.

19. A system for object-granular policy assignment in an object storage environment, comprising:

a processor and logic integrated with and/or executable by the processor, the logic being configured to;

automatically assign a plurality of objects stored within the object storage environment to one or more policies utilizing a semantic policy manager; and

locate one or more of the objects among a plurality of policy rings within the object storage environment based on semantics of the one or more policies,

wherein a virtual namespace maps each object to at least one of the policy rings based on the one or more policies assigned to the object; and

wherein the virtual namespace is scalable with a number of the plurality of objects stored within the object storage environment.

20. The system of claim 19, comprising logic configured to cause a semantic policy manager to:

determine availability characteristics and performance characteristics of a plurality of physical storage devices within the object storage environment upon which the plurality of objects are stored;

determine a plurality of differentials between the availability characteristics and the performance characteristics for each pair of the plurality of physical storage devices;

define a plurality of Bloom filters having different false positive rates based on the plurality of differentials, each Bloom filter corresponding to at least one of the policies;

define a size of each of the Bloom filters based on a policy for which the Bloom Filter is used within the object storage environment;

optimize a false positive rate for each of the Bloom filters using the semantic policy manager, each of the Bloom filters being optimized to have a different false positive rate; and

define a scan order for the plurality of Bloom filters based on the different false positive rates.