GROUPING OF OBJECTS INTO CLUSTERS IN AN OBJECT-BASED STORAGE ENVIRONMENT

• US 20200134082A1
• Filed: 10/25/2018
• Published: 04/30/2020
• Est. Priority Date: 10/25/2018
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Harrington, Shaun E., Rawlins, Benjamin K., Hartz, Ezra E., Barajas Gonzalez, Emmanuel
• Granted Patent: US 11,188,566 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/289 Object oriented databases

Abstract

In one general embodiment, a computer-implemented method includes determining attributes of a plurality of objects in an object-based storage environment, grouping the objects into clusters based on similarities of the attributes using a clustering algorithm, storing indication of which objects are in which clusters, detecting occurrence of a trigger event associated with an object, determining to which cluster the object belongs using the clustering algorithm in response to detecting the trigger event, and performing a deduplication process for the object against other objects in the cluster to which the object belongs. In another general embodiment, a computer program product includes a computer readable storage medium having program instructions embodied therewith. The computer readable storage medium is not a transitory signal per se. The program instructions are readable and/or executable by a computer to cause the computer to perform the foregoing method.

Claims

1. A computer-implemented method, comprising:

determining attributes of a plurality of objects in an object-based storage environment;

grouping the objects into clusters based on similarities of the attributes using a clustering algorithm;

storing indication of which objects are in which clusters;

detecting occurrence of a trigger event associated with an object;

in response to detecting the trigger event, determining to which cluster the object belongs using the clustering algorithm; and

performing a deduplication process for the object against other objects in the cluster to which the object belongs.

2. The computer-implemented method of claim 1, wherein the attributes include characteristics selected from the group consisting of:

file type, file name, file size, object type, object size, and metadata.

3. The computer-implemented method of claim 1, wherein the attributes include relationships selected from the group consisting of:

parent/child relationships, flat relationships, and consistency relationships.

4. The computer-implemented method of claim 1, wherein the trigger event includes detecting modification of the attributes of an object that is already in one of the clusters.

5. The computer-implemented method of claim 4, comprising:

in response to determining that the object belongs to a different cluster, grouping the object into the different cluster; and

performing a deduplication process for the object against other objects in the different cluster.

6. The computer-implemented method of claim 4, comprising:

in response to determining that the object belongs in the same cluster, performing a deduplication process for the object against other objects in the same cluster.

7. The computer-implemented method of claim 1, wherein the trigger event includes receiving a new object.

8. The computer-implemented method of claim 1, wherein the clustering algorithm is a K-means algorithm.

9. A computer program product, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instructions readable and/or executable by a computer to cause the computer to perform a method comprising:

determining, by the computer, attributes of a plurality of objects in an object-based storage environment;

grouping, by the computer, the objects into clusters based on similarities of the attributes using a clustering algorithm;

storing, by the computer, indication of which objects are in which clusters;

detecting, by the computer, occurrence of a trigger event associated with an object;

in response to detecting the trigger event, determining, by the computer, to which cluster the object belongs using the clustering algorithm; and

performing, by the computer, a deduplication process for the object against other objects in the cluster to which the object belongs.

10. The computer program product of claim 9, wherein the attributes include characteristics selected from the group consisting of:

file type, file name, file size, object type, object size, and metadata.

11. The computer program product of claim 9, wherein the attributes include relationships selected from the group consisting of:

parent/child relationships, flat relationships, and consistency relationships.

12. The computer program product of claim 9, wherein the trigger event includes detecting modification of the attributes of an object that is already in one of the clusters.

13. The computer program product of claim 12, the program instructions readable and/or executable by the computer to cause the computer to perform the method comprising:

in response to determining that the object belongs to a different cluster, grouping the object into the different cluster; and

performing a deduplication process for the object against other objects in the different cluster.

14. The computer program product of claim 12, the program instructions readable and/or executable by the computer to cause the computer to perform the method comprising:

in response to determining that the object belongs in the same cluster, performing a deduplication process for the object against other objects in the same cluster.

15. The computer program product of claim 9, wherein the trigger event includes receiving a new object.

16. The computer program product of claim 9, wherein the clustering algorithm is a K-means algorithm.

17. A system, comprising:

a processor; and

logic integrated with the processor, executable by the processor, or integrated with and executable by the processor, the logic being configured to;

determine attributes of a plurality of objects in an object-based storage environment;

group the objects into clusters based on similarities of the attributes using a clustering algorithm;

store indication of which objects are in which clusters;

detect occurrence of a trigger event associated with an object;

in response to detecting the trigger event, determine to which cluster the object belongs using the clustering algorithm; and

perform a deduplication process for the object against other objects in the cluster to which the object belongs.

18. The system of claim 17, wherein the attributes include characteristics selected from the group consisting of:

file type, file name, file size, object type, object size, and metadata.

19. The system of claim 17, wherein the attributes include relationships selected from the group consisting of:

parent/child relationships, flat relationships, and consistency relationships.

20. The system of claim 17, wherein the trigger event includes detecting modification of the attributes of an object that is already in one of the clusters.