Data storage and archiving spanning multiple data storage systems

• US 8,745,010 B2
• Filed: 04/12/2012
• Issued: 06/03/2014
• Est. Priority Date: 04/12/2012
• Status: Active Grant

• Current Assignee: Micro Focus LLC (Open Text Corporation)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Chan, Steve, Huang, Wei
• Primary Examiner(s): LE, HUNG D
• Application Number:
  US13/445,725
• Publication Number:
  US 20130275385A1
• Time in Patent Office: 782 Days
• Field of Search: None
• US Class Current: 707/670
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 3/065 Replication mechanisms

Abstract

Mutable portions of data are stored in a first storage system and immutable portions of the data are stored in a second storage system. Immutable portions and mutable portions of the data associated with the same time period are initially archived. After a period of time since the initial archiving, the mutable portions of the data from the first storage system that have been modified are supplemental archived. Integrity verifications of successfully performing the initial archiving and supplemental archiving are stored.

Claims

1. A method of archiving data stored in a plurality of data storage systems, the method comprising:

storing mutable portions of data in a first storage system;

storing immutable portions of the data in a second storage system;

performing, by a computer, an initial archiving of immutable portions of the data associated with a time period and of mutable portions of the data associated with the time period, wherein the immutable portions associated with the time period are stored in the second storage system and the mutable portions associated with the time period are stored in the first storage system;

storing an initial integrity verification of successfully archiving the mutable and immutable portions associated with the time period in response to determining the initial archiving is successful;

performing, after a period of time since the initial archiving was performed, a supplemental archiving of the mutable portions of the data from the first storage system that are associated with the time period; and

storing a supplemental integrity verification of successfully archiving the mutable portions associated with the time period in response to determining the supplemental archiving is successful.

2. The method of claim 1, comprising:

storing the initially archived mutable and immutable portions associated with the time period in an archive file in an archiving system and in the first storage system; and

storing the supplemental archived mutable portions with the archived file and in the first storage system.

3. The method of claim 2, wherein storing an initial integrity verification comprises using a hash function to calculate the initial integrity verification from the mutable and immutable portions associated with the time period;

and storing the initial integrity verification with the archive file in the archiving system and with the archived data in the first storage system; and

wherein storing a supplemental integrity verification comprises using the hash function to calculate, after the period of time since the initial archiving, the supplemental integrity verification from the mutable portions stored in the first storage system that are associated with the time period and have been modified after being stored; and

storing the supplemental integrity verification with the archive file in the archiving system and with the archived data in the first storage system.

4. The method of claim 3, comprising:

restoring the initially archived mutable portions and immutable portions associated with the time period and the supplemental archived mutable portions associated with the time period from the archive system based on comparing the initial integrity verification stored in the archive system and the first storage system and based on comparing the supplemental integrity verification stored in the archive system and the first storage system.

5. The method of claim 4, wherein the restoring comprises:

restoring the mutable portions, including the initially archived mutable portions and the supplemental archived mutable portions, and the immutable portions of data associated with the time period from the archive system if the initial integrity verification stored in the archive system matches the initial integrity verification stored in the first storage system and if the supplemental integrity verification stored in the archive system matches the supplemental integrity verification stored in the first storage system.

6. The method of claim 1, wherein performing supplemental archiving comprises:

determining whether any of the mutable portions of data from the first storage system storage system have been modified after the lapse of the period of time since the initial archiving; and

performing the supplemental archiving only on the mutable portions of data from the first storage system storage system that have been modified.

7. The method of claim 1, wherein the first storage system and the second storage system are different types of storage systems storing data in different types of data structures.

8. The method of claim 7, wherein the first storage system is a database storage system and the second storage system is a flat file storage system.

9. The method of claim 1, wherein the data comprises event data including a plurality of transactions performed each day, and each transaction is associated with an action that was performed once.

10. A system to store and archive data comprising:

a processor to

receive data and send the data to a first storage system and a second storage system, wherein the first storage system stores mutable portions of the data and the second storage system stores immutable portions of the data, and the first storage system is a database storage system and the second storage system is a flat file storage system,

invoke an initial archiving of immutable portions of the data associated with a time period and of mutable portions of the data associated with the time period;

store an initial integrity verification of successfully archiving the mutable and immutable portions associated with the time period in response to determining the initial archiving is successful;

invoke, after a period of time since the initial archiving was performed, a supplemental archiving of the mutable portions of the data from the first storage system that are associated with the time period; and

store a supplemental integrity verification of successfully archiving the mutable portions associated with the time period in response to determining the supplemental archiving is successful.

11. The system of claim 10 wherein the initially archived mutable and immutable portions associated with the time period are stored in an archive file in an archiving system and in the first storage system;

and the supplemental archived mutable portions are stored with the archived file and in the first storage system.

12. The system of claim 11, wherein a hash function is used to calculate the initial integrity verification from the mutable and immutable portions associated with the time period, and the initial integrity verification is stored with the archive file in the archiving system and with the archived data in the first storage system;

and

the hash function is used to calculate, after the period of time since the initial archiving, the supplemental integrity verification from the mutable portions stored in the second storage system that are associated with the time period and that have been modified since being stored, and the supplemental integrity verification is stored with the archive file in the archiving system and with the archived data in the first storage system.

13. The system of claim 12, wherein the processor is to invoke restoring the initially archived mutable portions and immutable portions associated with the time period and the supplemental archived mutable portions associated with the time period from the archive system based on comparing the initial integrity verification stored in the archive system and the first storage system and based on comparing the supplemental integrity verification stored in the archive system and the first storage system.

14. An information and event management system comprising:

a system to receive real-time event data from a plurality sources and to store immutable portions of the event data in a second storage system and store mutable portions of the event data in a first storage system, wherein the first storage system is a database storage system and the second storage system is a flat file storage system, and the event data describes events associated with actions performed in a computer network; and

a correlation engine to receive the event data from the system and correlate the events in accordance with rules to identify network security threats, wherein the system comprises at least one processor to

invoke an initial archiving of immutable portions of the event data associated with a time period and of mutable portions of the event data associated with the time period;

store an initial integrity verification of successfully archiving the mutable and immutable portions associated with the time period in response to determining the initial archiving is successful;

invoke, after a period of time since the initial archiving was performed, a supplemental archiving of the mutable portions of the data from the first storage system that are associated with the time period and have been modified since being initially stored in the first storage system; and

store a supplemental integrity verification of successfully archiving the mutable portions associated with the time period in response to determining the supplemental archiving is successful.

15. The information and event management system of claim 14, wherein the at least one processor is to:

store the initially archived mutable and immutable portions associated with the time period in an archive file in an archiving system and in the first storage system; and

store the supplemental archived mutable portions with the archived file and in the first storage system.

16. The information and event management system of claim 15, wherein to store the initial integrity verification the at least one processor is to:

use a hash function to calculate the initial integrity verification from the mutable and immutable portions associated with the time period; and

storing the initial integrity verification with the archive file in the archiving system and with the archived data in the first storage system; and

to store the supplemental integrity verification the at least one processor is to use the hash function to calculate, after the period of time since the initial archiving, the supplemental integrity verification from the mutable portions stored in the first storage system that are associated with the time period and have been modified after being stored; and

store the supplemental integrity verification with the archive file in the archiving system and with the archived data in the first storage system.

17. The information and event management system of claim 16, wherein the at least one processor is to:

restore the initially archived mutable portions and immutable portions associated with the time period and the supplemental archived mutable portions associated with the time period from the archive system based on comparing the initial integrity verification stored in the archive system and the first storage system and based on comparing the supplemental integrity verification stored in the archive system and the first storage system.

18. The information and event management system of claim 17, wherein to restore the initially archived mutable portions and immutable portions, the at least one processor is to:

restore the mutable portions, including the initially archived mutable portions and the supplemental archived mutable portions, and the immutable portions of data associated with the time period from the archive system if the initial integrity verification stored in the archive system matches the initial integrity verification stored in the first storage system and if the supplemental integrity verification stored in the archive system matches the supplemental integrity verification stored in the first storage system.

19. The information and event management system of claim 14, wherein the at least one processor is to:

determine whether any of the mutable portions of data from the first storage system storage system have been modified after the lapse of the period of time since the initial archiving; and

perform the supplemental archiving only on the mutable portions of data from the first storage system storage system that have been modified.