Dynamic bulk-to-brick transformation of data

• US 8,799,206 B2
• Filed: 10/02/2006
• Issued: 08/05/2014
• Est. Priority Date: 02/07/2005
• Status: Expired due to Fees

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Mimosa Systems Incorporated (Iron Mountain Incorporated)
• Inventors: D'Souza, Roy P., Ravi, Thirumalai Muppur
• Primary Examiner(s): Bhatia, Ajay
• Assistant Examiner(s): MINA, FATIMA P
• Application Number:
  US11/541,858
• Publication Number:
  US 20070150499A1
• Time in Patent Office: 2,864 Days
• Field of Search: 707/611, 707/613, 707/615, 707/618, 707/655
• US Class Current: 707/602
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/2028 eliminating a faulty processor or activating a spare
• CPC Class Codes: G06F 11/203 using migration
• CPC Class Codes: G06F 11/2041 with more than one idle spare processing component
• CPC Class Codes: G06F 11/2046 where the redundant components share persistent storage G06F11/2043 takes precedence
• CPC Class Codes: G06F 11/2056 by mirroring
• CPC Class Codes: G06F 11/2071 using a plurality of controllers
• CPC Class Codes: G06F 11/2082 Data synchronisation
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups

Abstract

Multi-dimensional surrogation systems and methods are provided that generate at least one up-to-date data surrogate using information of data and numerous data changes received from at least one data source, including a production system. Embodiments described further perform bulk-to-brick transformation of the up-to-date data surrogate. Brick-level data, or item data, is further indexed and stored in an application-aware manner. The stored item data is easily accessible by many data management applications for integrated item search and recovery functions, audit functions, monitoring and supervision functions, legal discovery functions, compliance functions, archival functions, backup functions, disaster recovery functions, and more. Bulk-to-brick transformation and access of the stored item data occur off of the production system, thus contributing no performance degradation to the production system.

Claims

1. A data management system comprising:

a utility system including at least one processor and configured to communicate with a first server that stores original data, wherein the utility system is configured to receive a plurality of delta data, wherein each delta data describes at least one of an incremental and differential change to the original data of the first server, the utility system configured to apply the plurality of delta data to a copy of the original data stored at the utility system,

a transformation system including at least one processor and coupled to the utility system and configured to;

determine a first structure of the copy of the original data, wherein the first structure is a B+ tree structure,

parse items of the copy of the original data using information of the first structure, wherein the parsing decomposes the items, and

dynamically transform the copy of the original data by organizing the items into a second, different structure that preserves a hierarchy of the first structure, wherein the items according to the second structure are accessible to an application, and wherein the second structure is a directory structure, and

wherein the utility system is configured to provide access to at least a portion of the items in the second structure using a second protocol that is different from a first protocol used to access the copy of the original data of the first structure.

2. The data management system of claim 1, wherein the first protocol includes one or more of a Messaging Application Programming Interface (MAPI) protocol, a Post Office Protocol (POP), and an Internet Message Access Protocol (IMAP).

3. The data management system of claim 1, wherein the second structure is stored on the utility system.

4. The data management system of claim 1, wherein the transformation system is configured to parse at predetermined intervals.

5. The data management system of claim 1, wherein the transformation system is configured to parse according to availability of the copy of the original data.

6. The data management system of claim 1, wherein the transformation system is configured to simultaneously parse a plurality of the items.

7. The data management system of claim 1, wherein the transformation system is configured to parse by systematically reading information of the copy of the original data, the information including one or more of tables and the items.

8. The data management system of claim 7, wherein the tables include one or more of a mailbox table, a folder table, a message table, and an attachment table.

9. The data management system of claim 7, wherein the items include one or more of a folder, a mailbox, an electronic mail body, an electronic mail header, and an electronic mail attachment.

10. The data management system of claim 1, wherein the transformation system is configured to decompress one or more of the items.

11. The data management system of claim 1, wherein the transformation system is configured to perform the organizing by transferring a first portion of information of the items into a first file and transferring a second portion of information of the items into a second file.

12. The data management system of claim 1, wherein the transformation system is configured to perform the organizing by indexing the items into the second structure.

13. The data management system of claim 12, wherein the transformation system is configured to perform the organizing by traversing the first structure and locating information of the items.

14. The data management system of claim 13, wherein the information of the items includes one or more of type and content of the items.

15. The data management system of claim 13, wherein the transformation system is configured to index by:

generating indices corresponding to the items; and

associating the items with the indices using the information of the items.

16. The data management system of claim 13, wherein the transformation system is configured to:

compare modification information of each item with modification information of a previous instance of the item; and

cease the traversing at a current node of the first structure when the modification information shows the item is unmodified since the previous instance.

17. The data management system of claim 16, wherein the modification information includes one or more of modification date and modification time.

18. The data management system of claim 13, wherein the transformation system is configured to:

determine whether an item is a previously-encountered item that has previously been stored in the second structure;

generate a pointer to the previously-encountered item.

19. The data management system of claim 13, wherein the transformation system is configured to:

compare modification information of a given item with modification information of a previous instance of the given item; and

index difference information of the given item when the modification information shows the given item is modified since the previous instance.

20. The data management system of claim 19, wherein the difference information includes information of changes between a current version and at least one previously-modified version of the given item.

21. The data management system of claim 13, wherein the transformation system is configured to perform the organizing by hashing information of each instance of a given item by generating a checksum.

22. The data management system of claim 21, wherein the transformation system is configured to determine if the given item has been altered since the organizing by comparing checksums of different instances of the given item in the second structure, the determining performed in response to a request to retrieve the given item from the second structure.

23. The data management system of claim 21, wherein the transformation system is configured to transfer the checksum to a hash table.

24. The data management system of claim 13, wherein the transformation system is configured to decompose a given item into at least one component.

25. The data management system of claim 24, wherein the given item is an electronic mail message, wherein the at least one component includes one or more of a message header, a message body, and a message attachment.

26. The data management system of claim 12, wherein the transformation system is configured to index by transferring the items into an indexed object repository.

27. The data management system of claim 12, wherein the transformation system is configured to index by transferring metadata of the items into a metadata repository.

28. The data management system of claim 1, configured to:

generate an updated version of the copy of the original data at the utility system by applying the delta data; and

controlling the applying using modified information of a component of the utility system.

29. The data management system of claim 28, wherein the component includes structural metadata of the copy of the original data.

30. The data management system of claim 28, wherein the component includes a log file of the delta data.

31. The data management system of claim 28, configured to modify the component.

32. The data management system of claim 31, wherein the component is structural metadata of the copy.

33. The data management system of claim 31, configured to modify by detecting a first state of the copy, wherein the first state indicates the delta data has been applied to the copy.

34. The data management system of claim 33, configured to modify by changing the first state to a second state, wherein the second state is a state from which another updated version can be generated by applying additional delta data to the updated version.

35. The data management system of claim 34, wherein changing the first state to the second state includes modifying structural metadata of the copy.

36. The data management system of claim 33, wherein an additional log file is received after generating the updated version.

37. The data management system of claim 28, configured to update structural metadata of the copy to reference a first unapplied delta data, wherein the first unapplied delta data is a first log file unapplied to the copy.

38. The data management system of claim 31, wherein the component is a log file of the delta data.

39. The data management system of claim 28, configured to receive at the utility system the copy of the original data from the first server.

40. The data management system of claim 39, wherein the copy is one or more of a full copy and an incremental copy.

41. The data management system of claim 28, configured to transfer the updated version to an indexed object repository.

42. The data management system of claim 28, configured to perform the generating in response to at least one of an automatic trigger, a timer notification, an event notification, a poll, and a request.

43. The data management system of claim 42, wherein the automatic trigger includes a trigger automatically initiated in response to at least one pre-specified parameter.

44. The data management system of claim 43, wherein the automatic trigger includes content of the updated version.

45. The data management system of claim 42, wherein the timer notification includes notifications corresponding to scheduled events including at least one of maintenance operations, user activities, server activities, and data population operations.

46. The data management system of claim 42, wherein the event notification includes notifications corresponding to changes to the items of the data.

47. The data management system of claim 42, wherein the request includes at least one of access attempts and configuration attempts to the data by one or more of the users, servers and applications.

48. The data management system of claim 28, configured to perform the generating in near real-time and maintain complete integrity and consistency of the copy of the original data.

49. The data management system of claim 28, configured to maintain the updated version by generating another updated version by applying at least one set of log files to the updated version.

50. The data management system of claim 1, wherein the first server includes one or more of local servers, remote servers, data servers, messaging servers, electronic mail servers, instant messaging servers, voice-over Internet Protocol servers, collaboration servers, messaging and collaboration servers, Exchange servers, portals, customer relationship management (CRM) servers, enterprise resource planning (ERP) servers, business-to-business servers, and content management servers.

51. The data management system of claim 1, wherein the hierarchy is preserved using metadata.

52. The data management system of claim 1, wherein the hierarchy is preserved using a file structure.

53. A method comprising:

receiving, by a utility system including at least one processor and coupled to a first server that stores original data, a plurality of delta data, where each delta data describes at least one of an incremental and differential change to the original data of the first server;

applying, by the utility system, the plurality of delta data to a copy of the original data stored at the utility system;

determining, by a transformation system including at least one processor, a first structure of the copy of the original data, wherein the first structure is a B+ tree structure;

parsing, by the transformation system, items of the copy of the original data using information of the first structure, wherein the parsing decomposes the items;

dynamically transforming, by the transformation system, the copy of the original data by organizing the items into a second, different structure that preserves a hierarchy of the first structure, wherein the items according to the second structure are accessible to an application, and wherein the second structure is a directory structure; and

providing, by the utility system, access to at least a portion of the items in the second structure using a second protocol that is different from a first protocol used to access the copy of the original data of the first structure.