IMPORT, EXPORT, AND COPY MANAGEMENT FOR TIERED OBJECT STORAGE

• US 20180373745A1
• Filed: 06/26/2017
• Published: 12/27/2018
• Est. Priority Date: 06/26/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bhagwat, Deepavali M., Bhosale, Nilesh P., Dain, Joseph W., Hewitt, James, Sawdon, Wayne A.
• Granted Patent: US 10,642,809 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/2272 Management thereof
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]

Abstract

In one embodiment, a method includes receiving, at a centralized index operating in a cloud environment via a messaging bus, first events spawned by a first processor managing an object store accessible via the cloud environment. The method also includes receiving, at the centralized index, second events spawned by a second processor managing an archival store accessible via the cloud environment. Also, the method includes normalizing metadata in the first and second events. Moreover, the method includes indexing and storing the metadata in the centralized index. The object store and the archival store operate in different object subsystems of the cloud environment. Other systems, methods, and computer program products are described for managing import, export, and copy of data within a public tiered object storage environment in accordance with more embodiments.

Claims

1. A method, comprising:

receiving, at a centralized index operating in a cloud environment via a messaging bus, first events spawned by a first processor managing an object store accessible via the cloud environment;

receiving, at the centralized index, second events spawned by a second processor managing an archival store accessible via the cloud environment;

normalizing metadata in the first and second events; and

indexing and storing the metadata in the centralized index,

wherein the object store and the archival store operate in different object subsystems of the cloud environment.

2. The method as recited in claim 1, wherein the object store comprises a plurality of non-volatile memory storage devices, wherein the archival store comprises a plurality of higher latency media, and wherein the object store and the archival store operate in a heterogeneous storage environment.

3. The method as recited in claim 1, wherein the first and/or second events comprise indication of an action performed on data associated with the first and/or second events.

4. The method as recited in claim 1, wherein the metadata comprises a storage location for data associated with the first and/or second events.

5. The method as recited in claim 1, further comprising:

receiving a search request for first data;

searching the centralized index for metadata relating to a storage location of the first data; and

responding to the search request with the storage location for the first data,

wherein the storage location for the first data is reported on the object store in response to a determination that at least one copy of the first data exists on the object store, and wherein the storage location for the first data is reported on the archival store in response to a determination that at least one copy of the first data exists on the archival store and not on the object store.

6. The method as recited in claim 1, further comprising:

receiving an import/export request to move second data stored on the object store to the archival store;

searching the centralized index for metadata relating to a storage location of the second data on the object store;

responding to the import/export request with the storage location for the second data on the object store; and

causing a data mover to move the second data from the object store to the archival store using the storage location for the second data on the object store.

7. The method as recited in claim 6, further comprising:

setting a timer for maintaining the second data on the object store; and

deleting the second data from the object store in response to expiration of the timer.

8. The method as recited in claim 6, further comprising:

receiving, at the centralized index, a second event spawned by the second processor indicating storage of the second data to the archival store; and

updating the centralized index to reflect a current storage location for the second data on the archival store.

9. The method as recited in claim 1, further comprising:

receiving a copy request to duplicate third data stored on the archival store onto the object store;

searching the centralized index for metadata relating to a storage location of the third data on the archival store;

responding to the copy request with the storage location for the third data on the archival store; and

causing a data mover to duplicate the third data stored on the archival store onto the object store using the storage location for the third data on the archival store.

10. The method as recited in claim 9, further comprising:

receiving, at the centralized index, a first event spawned by the first processor indicating copying of the third data to the object store; and

updating the centralized index to reflect an additional storage location for the third data on the object store.

11. A computer program product, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the embodied program instructions being executable by a processing circuit to cause the processing circuit to:

receive, by the processing circuit at a centralized index operating in a cloud environment via a messaging bus, first events spawned by a first processor managing an object store accessible via the cloud environment;

receive, by the processing circuit at the centralized index, second events spawned by a second processor managing an archival store accessible via the cloud environment;

normalize, by the processing circuit, metadata in the first and second events; and

index and store, by the processing circuit, the metadata in the centralized index,

wherein the object store and the archival store operate in different object subsystems of the cloud environment.

12. The computer program product as recited in claim 11, wherein the object store comprises a plurality of non-volatile memory storage devices, wherein the archival store comprises higher latency media, and wherein the object store and the archival store operate in a heterogeneous storage environment.

13. The computer program product as recited in claim 11, wherein the first and/or second events comprise indication of an action performed on data associated with the first and/or second events, and wherein the metadata comprises a storage location for data associated with the first and/or second events.

14. The computer program product as recited in claim 11, wherein the embodied program instructions are further executable by the processing circuit to cause the processing circuit to:

receive, by the processing circuit, a search request for first data;

search, by the processing circuit, the centralized index for metadata relating to a storage location of the first data; and

respond, by the processing circuit, to the search request with the storage location for the first data,

wherein the storage location for the first data is reported on the object store in response to a determination that at least one copy of the first data exists on the object store, and wherein the storage location for the first data is reported on the archival store in response to a determination that at least one copy of the first data exists on the archival store and not on the object store.

15. The computer program product as recited in claim 11, wherein the embodied program instructions are further executable by the processing circuit to cause the processing circuit to:

receive, by the processing circuit, an import/export request to move second data stored on the object store to the archival store;

search, by the processing circuit, the centralized index for metadata relating to a storage location of the second data on the object store;

respond, by the processing circuit, to the import/export request with the storage location for the second data on the object store; and

cause, by the processing circuit, a data mover to move the second data from the object store to the archival store using the storage location for the second data on the object store.

16. The computer program product as recited in claim 15, wherein the embodied program instructions are further executable by the processing circuit to cause the processing circuit to:

set, by the processing circuit, a timer for maintaining the second data on the object store; and

delete, by the processing circuit, the second data from the object store in response to expiration of the timer.

17. The computer program product as recited in claim 15, wherein the embodied program instructions are further executable by the processing circuit to cause the processing circuit to:

receive, by the processing circuit at the centralized index, a second event spawned by the second processor indicating storage of the second data to the archival store; and

update, by the processing circuit, the centralized index to reflect a current storage location for the second data on the archival store.

18. The computer program product as recited in claim 11, wherein the embodied program instructions are further executable by the processing circuit to cause the processing circuit to:

receive, by the processing circuit, a copy request to duplicate third data stored on the archival store onto the object store;

search, by the processing circuit, the centralized index for metadata relating to a storage location of the third data on the archival store;

respond, by the processing circuit, to the copy request with the storage location for the third data on the archival store; and

cause, by the processing circuit, a data mover to duplicate the third data stored on the archival store onto the object store using the storage location for the third data on the archival store.

19. The computer program product as recited in claim 18, wherein the embodied program instructions are further executable by the processing circuit to cause the processing circuit to:

receive, by the processing circuit at the centralized index, a first event spawned by the first processor indicating copying of the third data to the object store; and

update, by the processing circuit, the centralized index to reflect an additional storage location for the third data on the object store.

20. A system, comprising:

a processing circuit;

a memory; and

logic stored to the memory, that when executed by the processing circuit causes the processing circuit to;

receive, by the processing circuit at a centralized index operating in a cloud environment via a messaging bus, first events spawned by a first processor managing an object store accessible via the cloud environment;

receive, by the processing circuit at the centralized index, second events spawned by a second processor managing an archival store accessible via the cloud environment;

normalize, by the processing circuit, metadata in the first and second events; and

index and store, by the processing circuit, the metadata in the centralized index,

wherein the object store and the archival store operate in different object subsystems of the cloud environment, and wherein the cloud environment comprises a heterogeneous storage environment that includes the object store and the archival store.