Systems and methods for transformation of logical data objects for storage

• US 9,311,320 B2
• Filed: 07/21/2011
• Issued: 04/12/2016
• Est. Priority Date: 05/31/2006
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Koifman, Chaim, Kedem, Nadav, Zohar, Avi, Amit, Jonathan
• Primary Examiner(s): Perveen, Rehana
• Assistant Examiner(s): Wong, Huen
• Application Number:
  US13/188,438
• Publication Number:
  US 20110276548A1
• Time in Patent Office: 1,727 Days
• Field of Search: 707/687, 707/693
• US Class Current: 1/1
• CPC Class Codes: G06F 12/023 Free address space management
• CPC Class Codes: G06F 12/1408 by using cryptography for digital transmission H04L9/00
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/1727 Details of free space management performed by the file system saving storage space on storage systems G06F3/0608; management of blocks in storage devices G06F3/064
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 21/62 Protecting access to data via a platform, e.g. using keys or access control rules
• CPC Class Codes: G06F 21/6209 to a single file or object, e.g. in a secure envelope, encrypted and accessed using a key, or with access control rules appended to the object itself
• CPC Class Codes: G06F 21/78 to assure secure storage of data address-based protection against unauthorised use of memory G06F12/14; record carriers for use with machines and with at least a part designed to carry digital markings G06K19/00
• CPC Class Codes: G06F 2212/1044 Space efficiency improvement
• CPC Class Codes: G06F 2212/152 Virtualized environment, e.g. logically partitioned system
• CPC Class Codes: G06F 2212/401 Compressed data
• CPC Class Codes: G06F 2212/402 Encrypted data
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0623 in relation to content
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0638 Organizing or formatting or addressing of data
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0661 : Format or protocol conversion arrangements
• CPC Class Codes: G06F 3/0665 : at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 : Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0673 : Single storage device
• CPC Class Codes: H03M 7/30 : Compression speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N; Expansion; Suppression of unnecessary data, e.g. redundancy reduction

Abstract

Systems and methods for compressing a raw logical data object (201) for storage in a storage device operable with at least one storage protocol, creating, reading, writing, optimizatic in and restoring thereof. Compressing the raw logical data object (201) comprises creating in the storage device a compressed logical data object (203) comprising a header (204) and one or more allocated compressed sections with predefined size (205-1-205-2); compressing one or more sequentially obtained chunks of raw data (202-1-202-6) corresponding to the raw logical data object (201) thus giving rise to the compressed data chunks (207-1-207-6); and sequentially accommodating the processed data chunks into: said compressed sections (205-1-205-2) in accordance with an order said chunks received, wherein said compressed sections serve as atomic elements of compression/decompression operations during input/output transactions on the logical data object.

Claims

1. A method for recovery of a compressed logical data object including one or more compressed storage sections associated with one or more index sections, said method comprising:

recognizing, by a processor, a recovery status when opening the compressed logical data object;

initiating a recovery process in response to recognizing said recovery status;

searching the compressed logical object to discover one or more non-indexed compressed data chunks comprising at least one log that does not include a corresponding log record in any of the one or more index sections;

decompressing each non-indexed compressed data chunk associated with the at least one log to generate an uncompressed data chunk for each non-indexed compressed data chunk, each non-indexed compressed data chunk including a respective size and a respective data content;

re-compressing each uncompressed data chunk to generate a re-compressed data chunk for each uncompressed data chunk;

generating an entry in an index section of the one or more index sections for each re-compressed data chunk such that each non-indexed compressed section is indexed in the index section; and

storing each re-compressed data chunk in a compressed storage section of the one or more compressed storage sections, wherein;

each re-compressed data chunk subsequent to the re-compressing process maintains the same respective size as its corresponding compressed data chunk prior to the decompressing process, and

each re-compressed data chunk subsequent to the compressing process maintains the same respective data content as its corresponding compressed data chunk prior to the decompressing process.

2. The method of claim 1, wherein decompressing each compressed data chunk comprises sequentially decompressing in reverse order data included within each compressed data chunk.

3. The method of claim 2, wherein the reverse order comprises decompressing the data beginning from a last portion of the data until a beginning portion of the data.

4. The method of claim 1, wherein re-compressing each uncompressed data chunk comprises treating each uncompressed data chunk similar to a newly received data chunk for storage in the compressed storage section.

5. The method of claim 4, wherein treating each uncompressed data chunk similar to a newly received data chunk comprises re-compressing the data in each uncompressed data chunk in an order the data was originally received.

6. The method of claim 5, further comprising:

transmitting each re-compressed data chunk to the compressed storage section.

7. The method of claim 6, further comprising providing an indication of a successful recovery of each compressed data chunk.

8. A system for recovery of a compressed logical data object including one or more compressed storage sections associated with one or more index sections, said system comprising:

a processor; and

memory coupled to said processor, wherein the processor is configured to store computer code that, when executed by the processor, causes the processor to;

recognize a recovery status when opening the compressed logical data object,

initiate a recovery process in response to recognizing said recovery status,

search the compressed logical object to discover one or more non-indexed compressed data chunks comprising at least one log that does not include a corresponding log record in any of the one or more index sections,

decompress each non-indexed compressed data chunk associated with the at least one log to generate an uncompressed data chunk for each non-indexed compressed data chunk, each non-indexed compressed data chunk including a respective size and a respective data content,

re-compress each uncompressed data chunk to generate a re-compressed data chunk for each uncompressed data chunk,

generate an entry in an index section of the one or more index sections for each re-compressed data chunk such that each non-indexed compressed section is indexed in the index section, and

storing each re-compressed data chunk in a compressed storage section of the one or more compressed storage sections wherein;

each re-compressed data chunk subsequent to the re-compressing process maintains the same respective size as its corresponding compressed data chunk prior to the decompressing process, and

each re-compressed data chunk subsequent to the re-compressing process maintains the same respective data content as its corresponding compressed data chunk prior to the decompressing process.

9. The system of claim 8, wherein, when decompressing each compressed data chunk, the processor is configured to sequentially decompress in reverse order data included within each compressed data chunk.

10. The system of claim 9, wherein the reverse order comprises decompressing the data beginning from a last portion of the data until a beginning portion of the data.

11. The system of claim 8, wherein, when re-compressing each uncompressed data chunk, the processor is configured to treat each uncompressed data chunk similar to a newly received data chunk for storage in the compressed storage section.

12. The system of claim 11, wherein, when treating each uncompressed data chunk similar to a newly received data chunk, the processor is configured to re-compress the data in each uncompressed data chunk in an order the data was originally received.

13. The system of claim 12, wherein the processor is further configured to transmit each re-compressed data chunk to the compressed storage section.

14. The system of claim 13, wherein the processor is further configured to provide an indication of a successful recovery of each compressed data chunk.

15. A computer program product comprising a computer readable storage medium including computer readable program code embodied therein for recovery of a compressed logical data object including one or more compressed storage sections associated with one or more index sections, said computer program product comprising:

computer code for recognizing, by a processor, a recovery status when opening the compressed logical data object;

computer code for initiating a recovery process in response to recognizing said recovery status;

computer code for searching the compressed logical object to discover one or more non-indexed compressed data chunks comprising at least one log that does not include a corresponding log record in any of the one or more index sections;

computer code for decompressing each non-indexed compressed data chunk associated with the at least one log to generate an uncompressed data chunk for each non-indexed compressed data chunk, each non-indexed compressed data chunk including a respective size and a respective data content;

computer code for re-compressing each uncompressed data chunk to generate a re-compressed data chunk for each uncompressed data chunk;

computer code for generating an entry in an index section of the one or more index sections for each re-compressed data chunk such that each non-indexed compressed section is indexed in the index section; and

computer code for storing each re-compressed data chunk in a compressed storage section of the one or more compressed storage sections wherein;

each re-compressed data chunk subsequent to the compressing process maintains the same respective size as its corresponding compressed data chunk prior to the decompressing process, and

each re-compressed data chunk subsequent to the compressing process maintains the same respective data content as its corresponding compressed data chunk prior to the decompressing process.

16. The computer program product of claim 15, wherein the computer code for decompressing each compressed data chunk comprises computer code for sequentially decompressing in reverse order data included within each compressed data chunk.

17. The computer program product of claim 16, wherein the reverse order computer code for comprises computer code for decompressing the data beginning from a last portion of the data until a beginning portion of the data.

18. The computer program product of claim 15, wherein the computer code for searching the uncompressed data chunk comprises computer code for treating each uncompressed data chunk similar to a newly received data chunk for storage in the compressed storage section.

19. The computer program product of claim 18, wherein the computer code for treating each uncompressed data chunk similar to a newly received data chunk comprises computer code for re-compressing the data in each uncompressed data chunk in an order the data was originally received.

20. The computer program product of claim 19, further comprising:

computer code for transmitting each re-compressed data chunk to the compressed storage section.

21. The computer program product of claim 20, further comprising computer code for providing an indication of a successful recovery of each compressed data chunk.