Systems and methods for transformation of logical data objects for storage

• US 9,251,158 B2
• Filed: 07/21/2011
• Issued: 02/02/2016
• Est. Priority Date: 05/31/2006
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Koifman, Chaim, Kedem, Nadav, Zohar, Avi, Amit, Jonathan
• Primary Examiner(s): Perveen, Rehana
• Assistant Examiner(s): Wong, Huen
• Application Number:
  US13/188,435
• Publication Number:
  US 20110276546A1
• Time in Patent Office: 1,657 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 12/023 Free address space management
• CPC Class Codes: G06F 12/1408 by using cryptography for digital transmission H04L9/00
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/1727 Details of free space management performed by the file system saving storage space on storage systems G06F3/0608; management of blocks in storage devices G06F3/064
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 21/62 Protecting access to data via a platform, e.g. using keys or access control rules
• CPC Class Codes: G06F 21/6209 to a single file or object, e.g. in a secure envelope, encrypted and accessed using a key, or with access control rules appended to the object itself
• CPC Class Codes: G06F 21/78 to assure secure storage of data address-based protection against unauthorised use of memory G06F12/14; record carriers for use with machines and with at least a part designed to carry digital markings G06K19/00
• CPC Class Codes: G06F 2212/1044 Space efficiency improvement
• CPC Class Codes: G06F 2212/152 Virtualized environment, e.g. logically partitioned system
• CPC Class Codes: G06F 2212/401 Compressed data
• CPC Class Codes: G06F 2212/402 Encrypted data
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0623 in relation to content
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0638 Organizing or formatting or addressing of data
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0661 : Format or protocol conversion arrangements
• CPC Class Codes: G06F 3/0665 : at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 : Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0673 : Single storage device
• CPC Class Codes: H03M 7/30 : Compression speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N; Expansion; Suppression of unnecessary data, e.g. redundancy reduction

Abstract

Systems and methods for compressing a raw logical data object (201) for storage in a storage device operable with at least one storage protocol, creating, reading, writing, optimizatic in and restoring thereof. Compressing the raw logical data object (201) comprises creating in the storage device a compressed logical data object (203) comprising a header (204) and one or more allocated compressed sections with predefined size (205-1-205-2); compressing one or more sequentially obtained chunks of raw data (202-1-202-6) corresponding to the raw logical data object (201) thus giving rise to the compressed data chunks (207-1-207-6); and sequentially accommodating the processed data chunks into: said compressed sections (205-1-205-2) in accordance with an order said chunks received, wherein said compressed sections serve as atomic elements of compression/decompression operations during input/output transactions on the logical data object.

Claims

1. A method for reading compressed data from a logical object comprising a plurality of accommodation sections, said method comprising:

comparing a characteristic of a received plurality of data chunks to a predefined time-related criterion comprising an estimated or actual time of compression of the plurality of data chunks, the predefined time-related criterion including an estimated time necessary to compress data which, being compressed, would substantially amount to a compressed section size;

if the characteristic matches the predefined time-related criterion;

compressing the plurality of data chunks to create a plurality of compressed data chunks, and

accumulating the plurality of compressed data chunks in a first accommodation section until the first accommodation section is filled to a predefined size;

if the characteristic does not match the predefined time-related criterion;

identifying the plurality of data chunks as being in a non-compressed form or a partly compressed form, and

accumulating the non-compressed or partly compressed plurality of data chunks in a second accommodation section until the second accommodation section is filled to the predefined size;

receiving, by a processor, a request to read a first range of accumulated data stored in the logical object;

identifying the first range of data in the logical object as being first compressed data accumulated in the first accommodation section, first non-compressed data accumulated in the second accommodation section, or first partly compressed data accumulated in the second accommodation section;

performing one of;

decompressing the first compressed data,

decompressing the first partly compressed data, and

retrieving the first non-compressed data; and

transmitting the first range of accumulated data comprised of the decompressed first compressed data, the decompressed first partly compressed data, or the first non-compressed data to a requestor.

2. The method of claim 1, further comprising arranging the first range of accumulated data in an order in which the first range of accumulated data was accumulated prior to transmission to the requestor.

3. The method of claim 2, wherein transmitting the first range of accumulated data comprises transmitting the first range of accumulated data in the order in which the first range of accumulated data was accumulated.

4. The method of claim 1, further comprising:

receiving, by the processor, a request to read a second range of accumulated data stored in the logical object; and

identifying the second range of accumulated data as being second compressed data accumulated in the first accommodation section, second non-compressed data accumulated in the second accommodation section, or second partly compressed data accumulated in the second accommodation section.

5. The method of claim 4, further comprising:

performing at least one of;

decompressing the second compressed data,

decompressing the second partly compressed data, and

retrieving the second non-compressed data; and

transmitting the second range of accumulated data comprised of the decompressed second compressed data, the decompressed second partly compressed data, or the second non-compressed data to the requestor.

6. The method of claim 5, further comprising arranging the second range of accumulated data in an order in which the second range of accumulated data was accumulated prior to transmission to the requestor.

7. The method of claim 6, wherein transmitting the second range of accumulated data comprises transmitting the second range of accumulated data in the order in which the second range of accumulated data was accumulated.

8. A system for reading compressed data from a logical object comprising a plurality of accommodation sections, said system comprising:

a processor; and

memory coupled to said processor, wherein the processor is configured to store computer code that, when executed by the processor, causes the processor to;

compare a characteristic of a received plurality of data chunks to a predefined time-related criterion comprising an estimated or actual time of compression of the plurality of data chunks, the predefined time-related criterion including an estimated time necessary to compress data which, being compressed, would substantially amount to a compressed section size,

if the characteristic matches the predefined time-related criterion;

compress the plurality of data chunks to create a plurality of compressed data chunks, and

accumulate the plurality of compressed data chunks in a first accommodation section until the first accommodation section is filled to a predefined size,

if the characteristic does not match the predefined time-related criterion;

identify the plurality of data chunks as being in a non-compressed form or a partly compressed form, and

accumulate the non-compressed or partly compressed plurality of data chunks in a second accommodation section until the second accommodation section is filled to the predefined size,

receive a request to read a first range of accumulated data stored in the logical object,

identify the first range of data in the logical object as being first compressed data accumulated in the first accommodation section, first non-compressed data accumulated in the second accommodation section, or first partly compressed data accumulated in the second accommodation section,

perform at least one of;

decompressing the first compressed data,

decompressing the first partly compressed data, and

retrieving the first non-compressed data, and

transmit the first range of accumulated data comprised of the decompressed first compressed data, the decompressed first partly compressed data, or the first non-compressed data to a requestor.

9. The system of claim 8, wherein the processor is further configured to arrange the first range of accumulated data in an order in which the first range of accumulated data was accumulated prior to transmission to the requestor.

10. The system of claim 9, wherein, when transmitting the first range of accumulated data, the processor is configured to transmit the first range of accumulated data in the order in which the first range of accumulated data was accumulated.

11. The system of claim 8, wherein the processor is further configured to:

receive a request to read a second range of accumulated data stored in the logical object; and

identify the second range of accumulated data as being second compressed data accumulated in the first accommodation section, second non-compressed data accumulated in the second accommodation section, or second partly compressed data accumulated in the second accommodation section.

12. The system of claim 11, wherein the processor is further configured to:

performing at least one of;

decompressing the second compressed data,

decompressing the second partly compressed data, and

retrieving the second non-compressed data; and

transmit the second range of accumulated data comprised of the decompressed second compressed data, the decompressed second partly compressed data, or the second non-compressed data to the requestor.

13. The system of claim 12, wherein the processor is further configured to arrange the second range of accumulated data in an order in which the second range of accumulated data was accumulated prior to transmission to the requestor.

14. The system of claim 13, wherein, when transmitting the second range of accumulated data in the second range, the processor is configured to transmit the second range of accumulated data in the order in which the second range of accumulated data was accumulated.

15. A computer program product comprising a computer readable storage medium including a computer readable program code embodied therein for reading compressed data from a logical object comprising a plurality of accommodation sections, said computer program product comprising:

computer code for comparing a characteristic of a received plurality of data chunks to a predefined time-related criterion comprising an estimated or actual time of compression of the plurality of data chunks, the predefined criterion including an estimated time necessary to compress data which, being compressed, would substantially amount to a compressed section size;

if the characteristic matches the predefined time-related criterion;

computer code for compressing the plurality of data chunks to create a plurality of compressed data chunks, and

computer code for accumulating the plurality of compressed data chunks in a first accommodation section until the first accommodation section is filled to a predefined size;

if the characteristic does not match the predefined time-related criterion;

computer code for identifying the plurality of data chunks as being in a non-compressed form or a partly compressed form, and

computer code for accumulating the non-compressed or partly compressed plurality of data chunks in a second accommodation section until the second accommodation section is filled to the predefined size;

computer code for receiving, by a processor, a request to read a first range of accumulated data stored in the logical object;

computer code for identifying the first range of data within the logical object as being first compressed data accumulated in the first accommodation section, first non-compressed data accumulated in the second accommodation section, or first partly compressed data accumulated in the second accommodation section;

computer code for performing at least one of;

decompressing the first compressed data,

decompressing the first partly compressed data, and

retrieving the first non-compressed data; and

computer code for transmitting the first range of accumulated data comprised of the decompressed first compressed data, the decompressed first partly compressed data, or the first non-compressed data to a requestor.

16. The computer program product of claim 15, further comprising computer code for arranging the first range of accumulated data in an order in which the first range of accumulated data was accumulated prior to transmission to the requestor.

17. The computer program product of claim 16, wherein the computer code for transmitting the first range of accumulated data comprises computer code for transmitting the first range of accumulated data in the order in which the first range of accumulated data was accumulated.

18. The computer program product of claim 15, further comprising:

computer code for receiving a request to read a second range of accumulated data stored in the logical object; and

computer code for identifying the second range of accumulated data as being second compressed data accumulated in the first accommodation section, second non-compressed data accumulated in the second accommodation section, or second partly compressed data accumulated in the second accommodation section.

19. The computer program product of claim 18, further comprising:

computer code for performing at least one of;

decompressing the second compressed data,

decompressing the second partly compressed data, and

retrieving the second non-compressed data; and

computer code for transmitting the second range of accumulated data comprised of the decompressed second compressed data, the decompressed second partly compressed data, or the second non-compressed data to the requestor.

20. The computer program product of claim 19, further comprising computer code for arranging the second range of accumulated data in an order in which the second range of accumulated data was accumulated prior to transmission to the requestor.

21. The computer program product of claim 20, wherein the computer code for transmitting the second range of accumulated data comprises computer code for transmitting the second range of accumulated data in the order in which the second range of accumulated data was accumulated.