Systems and methods for transformation of logical data objects for storage

• US 9,317,516 B2
• Filed: 07/21/2011
• Issued: 04/19/2016
• Est. Priority Date: 05/31/2006
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Koifman, Chaim, Kedem, Nadav, Zohar, Avi, Amit, Jonathan
• Primary Examiner(s): Perveen, Rehana
• Assistant Examiner(s): Wong, Huen
• Application Number:
  US13/188,437
• Publication Number:
  US 20110276547A1
• Time in Patent Office: 1,734 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 12/023 Free address space management
• CPC Class Codes: G06F 12/1408 by using cryptography for digital transmission H04L9/00
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/1727 Details of free space management performed by the file system saving storage space on storage systems G06F3/0608; management of blocks in storage devices G06F3/064
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 21/62 Protecting access to data via a platform, e.g. using keys or access control rules
• CPC Class Codes: G06F 21/6209 to a single file or object, e.g. in a secure envelope, encrypted and accessed using a key, or with access control rules appended to the object itself
• CPC Class Codes: G06F 21/78 to assure secure storage of data address-based protection against unauthorised use of memory G06F12/14; record carriers for use with machines and with at least a part designed to carry digital markings G06K19/00
• CPC Class Codes: G06F 2212/1044 Space efficiency improvement
• CPC Class Codes: G06F 2212/152 Virtualized environment, e.g. logically partitioned system
• CPC Class Codes: G06F 2212/401 Compressed data
• CPC Class Codes: G06F 2212/402 Encrypted data
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0623 in relation to content
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0638 Organizing or formatting or addressing of data
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0661 : Format or protocol conversion arrangements
• CPC Class Codes: G06F 3/0665 : at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 : Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0673 : Single storage device
• CPC Class Codes: H03M 7/30 : Compression speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N; Expansion; Suppression of unnecessary data, e.g. redundancy reduction

Abstract

Systems and methods for compressing a raw logical data object (201) for storage in a storage device operable with at least one storage protocol, creating, reading, writing, optimizatic in and restoring thereof. Compressing the raw logical data object (201) comprises creating in the storage device a compressed logical data object (203) comprising a header (204) and one or more allocated compressed sections with predefined size (205-1-205-2); compressing one or more sequentially obtained chunks of raw data (202-1-202-6) corresponding to the raw logical data object (201) thus giving rise to the compressed data chunks (207-1-207-6); and sequentially accommodating the processed data chunks into: said compressed sections (205-1-205-2) in accordance with an order said chunks received, wherein said compressed sections serve as atomic elements of compression/decompression operations during input/output transactions on the logical data object.

Claims

1. A method for optimization of a compressed logical data object including a plurality of storage sections storing compressed data, said method comprising:

receiving, by a processor, a request to store data in the compressed logical data object;

searching the plurality of storage sections to identify a storage section comprising a mixture of outdated compressed data and live compressed data, and including greater than a predetermined percentage of the outdated compressed data, wherein the outdated compressed data includes a first timestamp prior to a predefined time and the live compressed data includes a second timestamp subsequent to the predefined time;

decompressing all of the outdated compressed data and all of the live compressed data in the storage section to generate a first portion of outdated data and a second portion of live data;

deleting the first portion of outdated data;

recompressing the second portion of live data;

storing the recompressed second portion of live data in a different storage section, wherein the recompressed second portion of live data keeps the second timestamp subsequent to recompression and storage of the second portion of live data; and

releasing the storage section, wherein physical space corresponding to the storage section is available for re-allocation to a new storage section subsequent to releasing the storage section.

2. The method of claim 1, further comprising:

receiving the data for storage in the compressed logical data object;

compressing the data to generate compressed data; and

storing the compressed data in a space in an identified storage section vacated by said releasing.

3. The method of claim 2, further comprising sequentially storing the compressed data in the space in an order the data was received.

4. The method of claim 1, wherein decompressing the outdated compressed data comprises decompressing the outdated compressed data in an order the outdated compressed data was received for storage.

5. The method of claim 4, wherein deleting the outdated data comprises deleting the outdated data in the order the outdated data was decompressed.

6. The method of claim 1, wherein decompressing the outdated compressed data comprises decompressing the outdated compressed data in an order the outdated compressed data was compressed and stored in the compressed logical data object.

7. The method of claim 6, wherein deleting the outdated data comprises deleting the outdated data in the order the outdated data was compressed and stored in the compressed logical data object.

8. A system for optimization of a compressed logical data object including a plurality of storage sections storing compressed data, said method comprising:

a processor; and

memory coupled to said processor, wherein the processor is configured to store computer code that, when executed by the processor, causes the processor to;

receive a request to store data in the compressed logical data object,

search the plurality of storage sections to identify a storage section comprising a mixture of outdated compressed data and live compressed data, and including greater than a predetermined percentage of outdated compressed data, wherein the outdated compressed data includes a first timestamp prior to a predefined time and the live compressed data includes a second timestamp subsequent to the predefined time,

decompress all of the outdated compressed data and all of the live compressed data in the storage section to generate a first portion of outdated data and a second portion of live data,

delete the first portion of outdated data,

recompress the second portion of live data,

store the recompressed second portion of live data in a different storage section, wherein the recompressed second portion of live data keeps the second timestamp subsequent to recompression and storage of the second portion of live data, and

release the storage section, wherein physical space corresponding to the storage section is available for re-allocation to a new storage section subsequent to releasing the storage section.

9. The system of claim 8, wherein the processor is further configured to:

receive the data for storage in the compressed logical data object;

compress the data to generate compressed data, and

store the compressed data in a space in an identified storage section vacated by said releasing.

10. The system of claim 9, wherein the processor is further configured to sequentially store the compressed data in the space in an order the data was received.

11. The system of claim 8, wherein, when decompressing the outdated compressed data, the processor is configured to decompress the outdated compressed data in an order the outdated compressed data was received for storage.

12. The system of claim 11, wherein, when deleting the outdated data, the processor is configured to delete the outdated data in the order the outdated data was decompressed.

13. The system of claim 8, wherein, when decompressing the outdated compressed data, the processor is configured to decompress the outdated compressed data in an order the outdated compressed data was compressed and stored in the compressed logical data object.

14. The system of claim 13, wherein, when deleting the outdated data, the processor is configured to delete the outdated data in the order the outdated data was compressed and stored in the compressed logical data object.

15. A computer program product comprising a hardware processor configured for:

receiving a request to store data in the compressed logical data object;

searching the plurality of storage sections to identify a storage section comprising a mixture of outdated compressed data and live compressed data, and including greater than a predetermined percentage of outdated compressed data, wherein the outdated compressed data includes a first timestamp prior to a predefined time and the live compressed data includes a second timestamp subsequent to the predefined time;

decompressing all of the outdated compressed data and all of the live compressed data in the storage section to generate a first portion of outdated data and a second portion of live data;

deleting the outdated data;

recompressing the second portion of live data;

storing the recompressed second portion of live data in a different storage section, wherein the recompressed second portion of live data keeps the second timestamp subsequent to recompression and storage of the second portion of live data; and

releasing the storage section, wherein physical space corresponding to the storage section is available for re-allocation to a new storage section subsequent to releasing the storage section.

16. The computer program product of claim 15, wherein the hardware processor is further configured for:

receiving the data for storage in the compressed logical data object;

compressing the data to generate compressed data; and

storing the compressed data in a space in an identified storage section vacated by said releasing.

17. The computer program product of claim 16, wherein the hardware processor is further configured for sequentially storing the compressed data in the space in an order the data was received.

18. The computer program product of claim 15, wherein, when decompressing the outdated compressed data, the hardware processor is configured for decompressing the outdated compressed data in an order the outdated compressed data was received for storage.

19. The computer program product of claim 18, wherein, when deleting the outdated data, the hardware processor is configured for deleting the outdated data in the order the outdated data was decompressed.

20. The computer program product of claim 15, wherein, when decompressing the outdated compressed data, the hardware processor is configured for decompressing the outdated compressed data in an order the outdated compressed data was compressed and stored in the compressed logical data object.

21. The computer program product of claim 20, wherein, when deleting the outdated data, the hardware processor is configured for deleting the outdated data in the order the outdated data was compressed and stored in the compressed logical data object.