Systems and methods for transformation of logical data objects for storage

• US 9,262,427 B2
• Filed: 04/13/2012
• Issued: 02/16/2016
• Est. Priority Date: 05/31/2006
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Koifman, Chaim, Kedem, Nadav, Zohar, Avi
• Primary Examiner(s): Perveen, Rehana
• Assistant Examiner(s): Wong, Huen
• Application Number:
  US13/446,688
• Publication Number:
  US 20120198275A1
• Time in Patent Office: 1,404 Days
• Field of Search: 707/687, 707/693
• US Class Current: 1/1
• CPC Class Codes: G06F 12/023 Free address space management
• CPC Class Codes: G06F 12/1408 by using cryptography for digital transmission H04L9/00
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/1727 Details of free space management performed by the file system saving storage space on storage systems G06F3/0608; management of blocks in storage devices G06F3/064
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 21/62 Protecting access to data via a platform, e.g. using keys or access control rules
• CPC Class Codes: G06F 21/6209 to a single file or object, e.g. in a secure envelope, encrypted and accessed using a key, or with access control rules appended to the object itself
• CPC Class Codes: G06F 21/78 to assure secure storage of data address-based protection against unauthorised use of memory G06F12/14; record carriers for use with machines and with at least a part designed to carry digital markings G06K19/00
• CPC Class Codes: G06F 2212/1044 Space efficiency improvement
• CPC Class Codes: G06F 2212/152 Virtualized environment, e.g. logically partitioned system
• CPC Class Codes: G06F 2212/401 Compressed data
• CPC Class Codes: G06F 2212/402 Encrypted data
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0623 in relation to content
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0638 Organizing or formatting or addressing of data
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0661 : Format or protocol conversion arrangements
• CPC Class Codes: G06F 3/0665 : at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 : Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0673 : Single storage device
• CPC Class Codes: H03M 7/30 : Compression speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N; Expansion; Suppression of unnecessary data, e.g. redundancy reduction

Abstract

Systems and methods for transforming a logical data object for storage in a storage device operable with at least one storage protocol, creating, reading, writing, optimization and restoring thereof. Transforming the logical data object comprises creating in the storage device a transformed logical data object comprising one or more allocated storage sections with a predefined size; transforming one or more sequentially obtained chunks of obtained data corresponding to the transforming logical data object; and sequentially storing the processed data chunks into said storage sections in accordance with a receive order of said chunks, wherein said storage sections serve as atomic elements of transformation/de-transformation operations during input/output transactions on the logical data object. The processing may comprise two or more data transformation techniques coordinated in time, concurrently executing autonomous sets of instructions, and provided in a manner preserving the sequence of processing and storing the processed data chunks.

Claims

1. A method for recovery of a transformed logical data object including one or more transformed storage sections associated with one or more index sections, said method comprising:

recognizing, by a processor, a recovery status when opening the transformed logical data object;

initiating a recovery process in response to recognizing said recovery status;

searching the transformed logical object to determine if the transformed logical object comprises one or more non-indexed transformed storage sections, the one or more non-indexed transformed storage sections comprising at least one log that does not include a corresponding log record in any of the one or more index sections;

de-transforming each transformed data chunk within the one or more non-indexed transformed storage sections associated with the at least one log to generate a de-transformed data chunk for each transformed data chunk, each transformed data chunk including a respective size and a respective data content;

re-transforming each de-transformed data chunk to generate a re-transformed data chunk for each de-transformed data chunk;

generating an entry in an index section of the one or more index sections for each re-transformed data chunk such that each transformed data chunk in the one or more non-indexed transformed storage sections is indexed in the index section; and

storing each re-transformed data chunk in a transformed storage section on the one or more storage sections, wherein;

each re-transformed data chunk subsequent to the re-transforming process maintains the same respective size as its corresponding transformed data chunk prior to the de-transforming process, and

each re-transformed data chunk subsequent to the re-transforming process maintains the same respective data content as its corresponding transformed data chunk prior to the de-transforming process.

2. The method of claim 1, wherein de-transforming each transformed data chunk comprises sequentially de-transforming in reverse order data included within each transformed data chunk.

3. The method of claim 2, wherein the reverse order comprises de-transforming the data beginning from a last portion of the data until a beginning portion of the data.

4. The method of claim 1, wherein storing each re-transformed data chunk comprises treating each re-transformed data chunk similar to a newly received data chunk for storage in the transformed logical data object.

5. The method of claim 4, wherein treating each re-transformed data chunk similar to a newly received data chunk comprises re-transforming the data in each re-transformed data chunk in an order the data was originally received to generate a newly transformed data chunk for each re-transformed data chunk.

6. The method of claim 5, further comprising:

transmitting each newly transformed data chunk to the transformed logical data object; and

storing each newly transformed data chunk in one of the transformed storage sections.

7. The method of claim 6, further comprising providing an indication of a successful recovery of the one or more non-indexed transformed storage sections.

8. A system for recovery of a transformed logical data object including one or more transformed storage sections associated with one or more index sections, said system comprising:

a processor; and

a storage device coupled to said processor, wherein the storage device is configured to store computer code that, when executed by the processor, causes the processor to;

recognize a recovery status when opening the transformed logical data object,

initiate a recovery process in response to recognizing said recovery status,

searching the transformed logical object to determine if the transformed logical object comprises one or more non-indexed transformed storage sections, the one or more non-indexed transformed storage sections comprising at least one log that does not include a corresponding log record in any of the one or more index sections,

de-transform each transformed data chunk within the one or more non-indexed transformed storage sections associated with the at least one log to generate a de-transformed data chunk for each transformed data chunk, each transformed data chunk including a respective size and a respective data content,

re-transform each de-transformed data chunk to generate a re-transformed data chunk for each de-transformed data chunk,

generate an entry in an index section of the one or more index sections for each re-transformed data chunk such that each transformed data chunk in the one or more non-indexed transformed storage sections is indexed in the index section, and

store each re-transformed data chunk in a transformed storage section on the one or more storage sections, wherein;

each re-transformed data chunk subsequent to the re-transforming process maintains the same respective size as its corresponding transformed data chunk prior to the de-transforming process, and

each re-transformed data chunk subsequent to the re-transforming process maintains the same respective data content as its corresponding transformed data chunk prior to the de-transforming process.

9. The system of claim 8, wherein, when de-transforming each transformed data chunk, the processor is configured to sequentially de-transform in reverse order data included within each transformed data chunk.

10. The system of claim 9, wherein the reverse order comprises de-transforming the data beginning from a last portion of the data until a beginning portion of the data.

11. The system of claim 8, wherein, when storing each re-transformed data chunk, the processor is configured to treat each re-transformed data chunk similar to a newly received data chunk for storage in the transformed logical data object.

12. The system of claim 11, wherein, when treating each re-transformed data chunk similar to a newly received data chunk, the processor is configured to transform the data in each re-transformed data chunk in an order the data was originally received to generate a newly transformed data chunk for each re-transformed data chunk.

13. The system of claim 12, wherein the processor is further configured to:

transmit each newly transformed data chunk to the transformed logical data object; and

store each newly transformed data chunk in one of the transformed storage sections.

14. The system of claim 13, wherein the processor is further configured to provide an indication of a successful recovery of the one or more non-indexed transformed storage sections.

15. A computer program product comprising a computer readable storage medium including a computer readable program code embodied therein for recovery of a transformed logical data object including one or more transformed storage sections associated with one or more index sections, said computer program product comprising:

computer code for recognizing, by a processor, a recovery status when opening the transformed logical data object;

computer code for initiating a recovery process in response to recognizing said recovery status;

computer code for searching the transformed logical object to determine if the transformed logical object comprises one or more non-indexed transformed storage sections, the one or more non-indexed transformed storage sections comprising at least one log that does not include a corresponding log record in any of the one or more index sections;

computer code for de-transforming each transformed data chunk within the one or more non-indexed transformed storage sections associated with the at least one log to generate a de-transformed data chunk for each transformed data chunk, each transformed data chunk including a respective size and a respective data content;

computer code for re-transforming each de-transformed data chunk to generate a re-transformed data chunk for each de-transformed data chunk;

computer code for generating an entry in an index section of the one or more index sections for each re-transformed data chunk such that each transformed data chunk in the one or more non-indexed transformed storage sections is indexed in the index section; and

computer code for storing each re-transformed data chunk in a transformed storage section on the one or more storage sections, wherein;

each re-transformed data chunk subsequent to the re-transforming process maintains the same respective size as its corresponding transformed data chunk prior to the de-transforming process, and

each re-transformed data chunk subsequent to the re-transforming process maintains the same respective data content as its corresponding transformed data chunk prior to the de-transforming process.

16. The computer program product of claim 15, wherein the computer code for de-transforming each transformed data chunk comprises computer code for sequentially de-transforming in reverse order data included within each transformed data chunk.

17. The computer program product of claim 16, wherein the reverse order computer code for comprises computer code for de-transforming the data beginning from a last portion of the data until a beginning portion of the data.

18. The computer program product of claim 15, wherein the computer code for storing each re-transformed data chunk comprises computer code for treating each re-transformed data chunk similar to a newly received data chunk for storage in the transformed logical data object.

19. The computer program product of claim 18, wherein the computer code for treating each re-transformed data chunk similar to a newly received data chunk comprises computer code for transforming the data in each re-transformed data chunk in an order the data was originally received to generate a newly transformed data chunk for each re-transformed data chunk.

20. The computer program product of claim 19, further comprising:

computer code for transmitting each newly transformed data chunk to the transformed logical data object; and

computer code for storing each newly transformed data chunk in one of the transformed storage sections.

21. The computer program product of claim 20, further comprising computer code for providing an indication of a successful recovery of the one or more non-indexed transformed storage sections.