Synthetic full copies of data and dynamic bulk-to-brick transformation

• US 8,918,366 B2
• Filed: 10/02/2006
• Issued: 12/23/2014
• Est. Priority Date: 02/07/2005
• Status: Expired due to Fees

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Mimosa Systems Incorporated (Iron Mountain Incorporated)
• Inventors: D'Souza, Roy P., Ravi, Thirumalai Muppur
• Primary Examiner(s): SYED, FARHAN M
• Application Number:
  US11/541,996
• Publication Number:
  US 20070156793A1
• Time in Patent Office: 3,004 Days
• US Class Current: 707/656
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/2028 eliminating a faulty processor or activating a spare
• CPC Class Codes: G06F 11/203 using migration
• CPC Class Codes: G06F 11/2041 with more than one idle spare processing component
• CPC Class Codes: G06F 11/2046 where the redundant components share persistent storage G06F11/2043 takes precedence
• CPC Class Codes: G06F 11/2056 by mirroring
• CPC Class Codes: G06F 11/2071 using a plurality of controllers
• CPC Class Codes: G06F 11/2082 Data synchronisation
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups

Abstract

Multi-dimensional surrogation systems and methods are provided that generate at least one up-to-date data surrogate using information of data and numerous data changes received from at least one data source, including a production system. Embodiments described further perform bulk-to-brick transformation of the up-to-date data surrogate. Brick-level data, or item data, is further indexed and stored in an application-aware manner. The stored item data is easily accessible by many data management applications for integrated item search and recovery functions, audit functions, monitoring and supervision functions, legal discovery functions, compliance functions, archival functions, backup functions, disaster recovery functions, and more. Bulk-to-brick transformation and access of the stored item data occur off of the production system, thus contributing no performance degradation to the production system.

Claims

1. A method comprising:

receiving a copy of original data and delta data that includes data of at least one of an incremental and differential difference between the original data at a plurality of instances;

generating an updated version of the copy by applying the delta data to the copy as the delta data is received, the updated version including a first data structure corresponding to a raw data structure generated by reading information from a shadow database;

controlling the applying of the delta data using modified information that includes information of a state of the copy, wherein the original data is stored at a first server and the copy is stored at a second server, wherein the first server includes a near line server;

dynamically transforming the updated version from the first data structure by organizing items of data into a second data structure as part of extracting items and metadata from the raw data structure that preserves a hierarchy of the first data structure;

comparing modification information of each of the items with modification information of a previous instance of the item as part of providing the second data structure; and

parsing the items using information of the first data structure, the parsing including reading one or more of tables and items, the tables include one or more of a mailbox table, a folder table, a message table, and an attachment table, the items include one or more of a folder, a mailbox, an electronic mail body, an electronic mail header, and an electronic mail attachment,

wherein the receiving, generating, controlling, transforming, comparing, and parsing are performed by a computer including at least one processor.

2. The method of claim 1, wherein the generating is asynchronous with the receiving, and the reading uses a raw read method that includes referencing a number of tables.

3. The method of claim 1, wherein the applying is according to an interval, wherein the interval is based on one or more of time and events.

4. The method of claim 1, wherein the second server includes one or more of local servers, remote servers, database servers, messaging servers, electronic mail servers, instant messaging servers, voice-over Internet Protocol servers, collaboration servers, messaging and collaboration servers, Exchange Servers, portals, customer relationship management (CRM) servers, enterprise resource planning (ERP) servers, business-to-business servers, and content management servers.

5. The method of claim 1, wherein the modified information is of a component, and wherein the component includes structural metadata of the copy.

6. The method of claim 1, wherein the modified information is of a component, and wherein the component includes a log file of the delta data.

7. The method of claim 1, wherein the modified information is of a component, the method further comprising modifying the component.

8. The method of claim 7, wherein the component includes a structural metadata of the copy.

9. The method of claim 7, wherein the modifying comprises detecting a first state of the copy, wherein the first state indicates the delta data has been applied to the copy.

10. The method of claim 9, wherein the modifying comprises changing the first state to a second state, wherein the second state is a state from which another updated version can be generated by applying additional delta data to the updated version.

11. The method of claim 10, wherein changing the first state to the second state includes modifying the structural metadata of the copy.

12. The method of claim 7, wherein the delta data includes a plurality of log files, wherein the component is a log file of the plurality of log files.

13. The method of claim 12, wherein the applying includes invoking an engine of the first server and the controlling includes stalling the engine.

14. The method of claim 1, further comprising determining the first data structure, wherein the first data structure is a proprietary structure.

15. The method of claim 1, wherein the organizing includes indexing the items into the second data structure.

16. The method of claim 15, wherein the organizing includes traversing the first data structure and locating information of the items.

17. The method of claim 16, wherein the information of the items includes one or more of type and content of the items.

18. The method of claim 16, wherein the indexing comprises:

generating indices corresponding to the items; and

associating the items with the indices using the information of the items.

19. The method of claim 16, further comprising ceasing the traversing at a current node of the first data structure when the modification information shows the item is unmodified since the previous instance.

20. The method of claim 16, further comprising:

determining whether an item is a previously-encountered item that has previously been stored in the second data structure;

generating a pointer to the previously-encountered item.

21. The method of claim 16, further comprising

indexing difference information of a given item when the modification information shows the given item is modified since the previous instance.

22. The method of claim 21, wherein the difference information includes information of changes between a current version and at least one previously-modified version of the given item.

23. The method of claim 16, wherein the organizing comprises hashing information of each instance of a given item by generating a checksum.

24. The method of claim 23, comprising determining if the given item has been altered since the organizing by comparing checksums of different instances of the given item in the second data structure, the determining performed in response to a request to retrieve the given item from the second data structure.

25. The method of claim 23, further comprising transferring the checksum to a hash table.

26. The method of claim 16, further comprising decomposing a given item into at least one component.

27. The method of claim 15, wherein the indexing includes transferring the items into an indexed object repository.

28. The method of claim 15, wherein the indexing includes transferring metadata of the items into a metadata repository.

29. The method of claim 1, further comprising providing access to at least a portion of the items in the second data structure via a plurality of applications.

30. The method of claim 29, wherein providing the access includes providing the access using a second protocol that is different from a first protocol used to access data of the first data structure.

31. The method of claim 1, further comprising transferring the updated version to an indexed object repository.

32. The method of claim 1, wherein the generating is in near real-time and maintains complete integrity and consistency of the original data.

33. The method of claim 1, wherein the original data includes one or more of application data, databases, storage groups, mailbox data, and server data.

34. The method of claim 1, further comprising maintaining the updated version, the maintaining including generating another updated version by applying at least one set of log files to the updated version, the at least one set of log files received later in time than the plurality of log files.

35. A method comprising:

receiving a plurality of delta data at the first server, the delta data including data of at least one of an incremental and differential difference between original data of the second server at a plurality of instances;

dynamically generating an updated version of the copy of the original data at the first server by applying at least one of the plurality of delta data to the copy;

controlling the applying of the at least one delta data using modified information that includes information of a state of the copy, the updated version including a first data structure corresponding to a raw data structure generated by reading information from a shadow database, wherein the original data is stored at the second server and the copy is stored at the first server, wherein the second server includes a near line server;

dynamically transforming the updated version from the first data structure by organizing items of data into a second data structure as part of extracting items and metadata from the raw data structure that preserves a hierarchy of the first data structure;

comparing modification information of each of the items with modification information of a previous instance of the item as part of providing the second data structure; and

parsing the items using information of the first data structure, the parsing including reading one or more of tables and items, the tables include one or more of a mailbox table, a folder table, a message table, and an attachment table, the items include one or more of a folder, a mailbox, an electronic mail body, an electronic mail header, and an electronic mail attachment,

wherein the receiving, generating, controlling, transforming, comparing, and parsing are performed by a computer including at least one processor.

36. A non-transitory computer readable storage medium including:

executable instructions which, when executed in a processing system, cause the processing system to;

receive a copy of original data and delta data that includes data of at least one of an incremental and differential difference between the original data at a plurality of instances;

generate an updated version of the copy by applying the delta data to the copy as the delta data is received, the updated version including a first data structure corresponding to a raw data structure generated by reading information from a shadow database;

control the applying of the delta data using modified information that includes information of a state of the copy, wherein the original data is stored at a first server and the copy is stored at a second server, wherein the first server includes a near line server;

dynamically transform the updated version from the first data structure by organizing items of data into a second data structure as part of extracting items and metadata from the raw data structure that preserves a hierarchy of the first data structure;

compare modification information of each of the items with modification information of a previous instance of the item as part of providing the second data structure; and

parse the items using information of the first data structure, the parsing including reading one or more of tables and items, the tables include one or more of a mailbox table, a folder table, a message table, and an attachment table, the items include one or more of a folder, a mailbox, an electronic mail body, an electronic mail header, and an electronic mail attachment.