Systems and methods for transformation of logical data objects for storage

• US 8,930,329 B2
• Filed: 08/21/2013
• Issued: 01/06/2015
• Est. Priority Date: 05/31/2006
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Koifman, Chaim, Kedem, Nadav, Zohar, Avi
• Primary Examiner(s): Saeed, Usmaan
• Assistant Examiner(s): Kim, Paul
• Application Number:
  US13/972,463
• Publication Number:
  US 20130339324A1
• Time in Patent Office: 503 Days
• Field of Search: 707/693, 707/687, 707/697
• US Class Current: 707/693
• CPC Class Codes: G06F 12/023 Free address space management
• CPC Class Codes: G06F 12/1408 by using cryptography for digital transmission H04L9/00
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/1727 Details of free space management performed by the file system saving storage space on storage systems G06F3/0608; management of blocks in storage devices G06F3/064
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 21/62 Protecting access to data via a platform, e.g. using keys or access control rules
• CPC Class Codes: G06F 21/6209 to a single file or object, e.g. in a secure envelope, encrypted and accessed using a key, or with access control rules appended to the object itself
• CPC Class Codes: G06F 21/78 to assure secure storage of data address-based protection against unauthorised use of memory G06F12/14; record carriers for use with machines and with at least a part designed to carry digital markings G06K19/00
• CPC Class Codes: G06F 2212/1044 Space efficiency improvement
• CPC Class Codes: G06F 2212/152 Virtualized environment, e.g. logically partitioned system
• CPC Class Codes: G06F 2212/401 Compressed data
• CPC Class Codes: G06F 2212/402 Encrypted data
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0623 in relation to content
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0638 Organizing or formatting or addressing of data
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0661 : Format or protocol conversion arrangements
• CPC Class Codes: G06F 3/0665 : at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 : Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0673 : Single storage device
• CPC Class Codes: H03M 7/30 : Compression speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N; Expansion; Suppression of unnecessary data, e.g. redundancy reduction

Abstract

Methods and systems for transforming a logical data object for storage in a storage device configured to operate with at least one storage protocol. One method comprises creating in the storage device a transformed logical data object comprising a one or more allocated storage sections with a predefined size and receiving one or more data chunks corresponding to the transformed logical data object. The method further comprises determining if each received data chunk comprises a predefined criterion, transforming each data chunk that comprises the predefined criterion, maintaining each data chuck in raw form that does not comprise the predefined criterion, and sequentially storing each transformed data chuck and data chunk in raw form into said one or more allocated storage sections in accordance with an order said transformed data chunks and data chunks in raw form are received. One system comprises a processor configured to perform the above method.

Claims

1. A method for compressing a logical data object for storage in a storage device configured to operate with at least one storage protocol, said method comprising:

a) creating, in the storage device, a compressed logical data object comprising one or more allocated compressed sections with a predefined size;

b) receiving variable size chunks of raw data corresponding to the compressed raw logical data object, wherein the received chunks of raw data are in non-compressed form;

c) sequentially storing said received chunks of raw data in non-compressed form in a compressed section in accordance with an order said chunks of raw data in non-compressed form are received;

d) providing an indication that the compressed section holds data in a non-transformed form;

e) mapping the data in the logical data object and the data stored in the allocated compressed sections;

f) scanning the compressed logical data object to identify one or more compressed sections holding data in non-compressed form;

g) compressing each data chunk stored in each identified compression section, wherein the compression is provided via an adaptive dictionary, and at least two data chunks are stored in different compressed sections via different dictionaries; and

h) sequentially storing the data chunks including compressed data in one or more newly allocated compressed sections.

2. The method of claim 1, wherein operations g) and h) are provided in accordance with a predefined time schedule.

3. The method of claim 1, wherein the size of the compressed section is selected from a list of predefined sizes in accordance with a predetermined criterion.

4. The method of claim 1, wherein the predefined size is equal for all compressed sections.

5. The method of claim 1, wherein each compressed section is associated with a flag indicating a use/re-use condition of a respective physical location of the compressed section, and each data chunk is associated with a same flag as the compressed section storing each data chunk.

6. A method for compressing a logical data object for storage in a storage device configured to operate with at least one storage protocol, said method comprising:

a) creating, in the storage device, a compressed logical data object comprising one or more allocated compressed sections with a predefined size;

b) receiving variable size chunks of raw data corresponding to the compressed raw logical data object, wherein the received chunks of raw data are in non-compressed form;

c) sequentially storing said received chunks of raw data in non-compressed form in a compressed section in accordance with an order said chunks of raw data in non-compressed form are received;

d) providing an indication that the compressed section holds data in a non-transformed form;

e) mapping the data in the logical data object and the data stored in the allocated compressed sections;

f) scanning the compressed logical data object to identify one or more compressed sections holding data in non-compressed form;

g) compressing each data chunk stored in each identified compressed section; and

h) sequentially storing the data chunks including compressed data in one or more newly allocated compressed sections,

wherein the compression is provided via an adaptive dictionary and the data chunks are stored in the newly allocated compressed sections via the adaptive dictionary.

7. The method of claim 6, wherein operations g) and h) are provided in accordance with a predefined time schedule.

8. The method of claim 6, wherein the size of the compressed section is selected from a list of predefined sizes in accordance with a predetermined criterion.

9. The method of claim 6, wherein the predefined size is equal for all compressed sections.

10. The method of claim 6, wherein each compressed section is associated with a flag indicating a use/re-use condition of a respective physical location of the compressed section, and each data chunk is associated with a same flag as the compressed section storing each data chunk.

11. A system for compressing a logical data object for storage in a storage device configured to operate with at least one storage protocol, said system comprising:

a processor; and

memory coupled to the processor, wherein the memory is configured to store computer code that, when executed by the processor, causes the processor to;

a) create, in the storage device, a compressed logical data object comprising one or more allocated compressed sections with a predefined size,

b) receive variable size chunks of raw data corresponding to the compressed raw logical data object, wherein the received chunks of raw data are in non-compressed form,

c) sequentially store said received chunks of raw data in non-compressed form in a compressed section in accordance with an order said chunks of raw data in non-compressed form are received,

d) provide an indication that the compressed section holds data in a non-transformed form,

e) map the data in the logical data object and the data stored in the allocated compressed sections,

f) scan the compressed logical data object to identify one or more compressed sections holding data in non-compressed form,

g) compress each data chunk stored in each identified compressed section, and

h) sequentially store the data chunks including compressed data in one or more newly allocated compressed sections,

wherein the compression is provided via an adaptive dictionary and the data chunks are stored in the newly allocated compressed sections via the adaptive dictionary.

12. The system of claim 11, wherein operations g) and h) are provided in accordance with a predefined time schedule.

13. The system of claim 11, wherein the size of the compressed section is selected from a list of predefined sizes in accordance with a predetermined criterion.

14. The system of claim 11, wherein the predefined size is equal for all compressed sections.

15. The system of claim 11, wherein each compressed section is associated with a flag indicating a use/re-use condition of a respective physical location of the compressed section, and each data chunk is associated with a same flag as the compressed section storing each data chunk.

16. A computer program product comprising a non-transitory computer readable storage medium including a computer readable program code embodied therein for compressing a logical data object for storage in a storage device configured to operate with at least one storage protocol, said computer program product comprising:

a) computer code for creating, in the storage device, a compressed logical data object comprising one or more allocated compressed sections with a predefined size;

b) computer code for receiving variable size chunks of raw data corresponding to the compressed raw logical data object, wherein the received chunks of raw data are in non-compressed form;

c) computer code for sequentially storing said received chunks of raw data in non-compressed form in a compressed section in accordance with an order said chunks of raw data in non-compressed form are received;

d) computer code for providing an indication that the compressed section holds data in a non-transformed form;

e) computer code for mapping the data in the logical data object and the data stored in the allocated compressed sections;

f) computer code for scanning the compressed logical data object to identify one or more compressed sections holding data in non-compressed form;

g) computer code for compressing each data chunk stored in each identified compressed section; and

h) computer code for sequentially storing data chunks including compressed data in one or more newly allocated compressed sections,

wherein the compression is provided via an adaptive dictionary and the data chunks are stored in the newly allocated compressed sections via the adaptive dictionary.

17. The computer program product of claim 16, wherein the computer code for performing operations g) and h) are provided in accordance with a predefined time schedule.

18. The computer program product of claim 16, wherein the size of the compressed section is selected from a list of predefined sizes in accordance with a predetermined criterion.

19. The computer program product of claim 16, wherein the predefined size is equal for all compressed sections.

20. The computer program product of claim 16, wherein each compressed section is associated with a flag indicating a use/re-use condition of a respective physical location of the compressed section, and each data chunk is associated with a same flag as the compressed section storing each data chunk.