Systems and methods for transformation of logical data objects for storage

• US 9,323,773 B2
• Filed: 03/28/2012
• Issued: 04/26/2016
• Est. Priority Date: 05/31/2006
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Koifman, Chaim, Kedem, Nadav, Zohar, Avi
• Primary Examiner(s): Perveen, Rehana
• Assistant Examiner(s): Wong, Huen
• Application Number:
  US13/432,136
• Publication Number:
  US 20120191738A1
• Time in Patent Office: 1,490 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 12/023 Free address space management
• CPC Class Codes: G06F 12/1408 by using cryptography for digital transmission H04L9/00
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/1727 Details of free space management performed by the file system saving storage space on storage systems G06F3/0608; management of blocks in storage devices G06F3/064
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 21/62 Protecting access to data via a platform, e.g. using keys or access control rules
• CPC Class Codes: G06F 21/6209 to a single file or object, e.g. in a secure envelope, encrypted and accessed using a key, or with access control rules appended to the object itself
• CPC Class Codes: G06F 21/78 to assure secure storage of data address-based protection against unauthorised use of memory G06F12/14; record carriers for use with machines and with at least a part designed to carry digital markings G06K19/00
• CPC Class Codes: G06F 2212/1044 Space efficiency improvement
• CPC Class Codes: G06F 2212/152 Virtualized environment, e.g. logically partitioned system
• CPC Class Codes: G06F 2212/401 Compressed data
• CPC Class Codes: G06F 2212/402 Encrypted data
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0623 in relation to content
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0638 Organizing or formatting or addressing of data
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0661 : Format or protocol conversion arrangements
• CPC Class Codes: G06F 3/0665 : at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 : Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0673 : Single storage device
• CPC Class Codes: H03M 7/30 : Compression speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N; Expansion; Suppression of unnecessary data, e.g. redundancy reduction

Abstract

Systems and methods for transforming a logical data object for storage in a storage device operable with at least one storage protocol, creating, reading, writing, optimization and restoring thereof. Transforming the logical data object comprises creating in the storage device a transformed logical data object comprising one or more allocated storage sections with a predefined size; transforming one or more sequentially obtained chunks of obtained data corresponding to the transforming logical data object; and sequentially storing the processed data chunks into said storage sections in accordance with a receive order of said chunks, wherein said storage sections serve as atomic elements of transformation/de-transformation operations during input/output transactions on the logical data object. The processing may comprise two or more data transformation techniques coordinated in time, concurrently executing autonomous sets of instructions, and provided in a manner preserving the sequence of processing and storing the processed data chunks.

Claims

1. A method for reading data from a transformed logical object comprising a plurality of transformed storage sections, said method comprising:

receiving, by a processor, a plurality of data chunks for storage in the transformed logical object;

comparing a characteristic of each data chunk to a predefined criterion comprising estimated or actual time of transformation of each data chunk;

transforming each data chunk and accumulating each transformed data chunk in a first storage section if the predefined characteristic matches the characteristic of each particular data chunk;

identifying each data chunk as being non-transformed or partly transformed and accumulating each non-transformed data chunk and each partly transformed data chunk in a second storage section if the predefined characteristic does not match the characteristic of each particular data chunk;

receiving, by the processor, a read request to read a range of data stored in the transformed logical object, wherein the read request includes a starting point in time at which the data was stored;

identifying the range of data utilizing the starting point in time included in the read request;

selecting a last-updated data related to the range of data, wherein the last-updated data is included in an index entry that facilitates a one-to-one relationship between the range of data and live data to be extracted from the transformed logical object;

de-transforming the first storage section including a first portion of the live data to generate first data;

determining that the second storage section includes a second portion of the live data;

de-transforming the partly transformed data chunks in the second storage section to generate second data comprising a combination of the de-transformed partly transformed data chunks and the non-transformed data chunks;

identifying each portion of the requested range of data included in the first data and the second data;

storing each portion of the requested range of data identified in the first data and the second data in cache; and

transmitting each portion of the requested range of data identified in the first data and the second data from the cache to a requestor, wherein;

the first portion of the live data is less than the first storage section in its entirety, and

the second portion of the live data is less than the second storage section in its entirety.

2. The method of claim 1, further comprising arranging each portion of the requested range of data identified in the first data and the second data in an order in which the data was transformed prior to transmission to the requestor.

3. The method of claim 2, wherein transmitting each portion of the requested range of data identified in the first data and the second data comprises transmitting the each portion of the requested range of data identified in the first de-transformed data and the second de-transformed data in an order in which the data was stored.

4. The method of claim 1, further comprising:

determining that one or more additional storage sections includes one or more additional portions of the live data;

de-transforming each of the one or more additional storage sections to generate third data; and

identifying each portion of the requested range of data within the third data.

5. The method of claim 4, further comprising:

storing, in the cache, each portion of the requested range of data identified in the third data; and

transmitting each portion of the requested range of data identified in the third data together with each portion of the requested range of data identified in the first data and in the second data to the requestor.

6. The method of claim 5, further comprising arranging each portion of the requested range of data identified in the first data, the second data, and the third data in an order in which the data was stored in each storage section prior to transmission to the requestor.

7. The method of claim 6, wherein transmitting each portion of the requested range of data identified in the first data, the second data, and the third data comprises transmitting each portion of the requested range of data identified in the first data, the second data, and the third data in the order in which the data was stored.

8. A system for reading data from a transformed logical object, said system comprising:

a processor; and

a storage device coupled to said processor, wherein the storage device is configured to store computer code that, when executed by the processor, causes the processor to;

receive a plurality of data chunks for storage in the transformed logical object;

comparing a characteristic of each data chunk to a predefined criterion comprising estimated or actual time of transformation of each data chunk,

transform each data chunk and accumulating each transformed data chunk in a first storage section if the predefined characteristic matches the characteristic of each particular data chunk,

identify each data chunk as being non-transformed or partly transformed and accumulating each non-transformed data chunk and each partly transformed data chunk in a second storage section if the predefined characteristic does not match the characteristic of each particular data chunk,

receive a read request to read a range of data stored in the transformed logical object, wherein the read request includes a starting point in time at which the data was stored,

identifying the range of data utilizing the starting point in time included in the read request,

select a last-updated data related to the range of data, wherein the last-updated data is included in an index entry that facilitates a one-to-one relationship between the range of data and live data to be extracted from the transformed logical object,

de-transform the first storage section including a first portion of the live data to generate first data,

determine that the second storage section includes a second portion of the live data,

de-transform the partly transformed data chunks in the second storage section to generate second data comprising a combination of the de-transformed partly transformed data chunks and the non-transformed data chunks,

identify each portion of the requested range of data included in the first data and the second data,

store each portion of the requested range of data identified in the first data and the second data in cache, and

transmit each portion of the requested range of data identified in the first data and the second data from the cache to a requestor, wherein;

the first portion of the live data is less than the first storage section in its entirety, and

the second portion of the live data is less than the second storage section in its entirety.

9. The system of claim 8, wherein the processor is further configured to arrange each portion of the requested range of data identified in the first data and the second data in an order in which the data was transformed prior to transmission to the requestor.

10. The system of claim 9, wherein, when transmitting each portion of the requested range of data identified in the first data and the second data, the processor is configured to transmit each portion of the requested range of data identified in the first data and the second data in the order in which the data was stored.

11. The system of claim 8, wherein the processor is further configured to:

determine that one or more additional storage sections includes one or more additional portions of the live data;

de-transform each of the one or more additional storage sections to generate third data; and

identify each portion of the data within the third data.

12. The system of claim 11, wherein the processor is further configured to:

store, in the cache, each portion of the requested range of data identified in the third data; and

transmit each portion of the requested range of data identified in the third data together with each portion of the requested range of data identified in the first data and in the second data to the requestor.

13. The system of claim 12, wherein the processor is further configured to arrange each portion of the requested range of data identified in the first data, the second data, and the third data in an order in which the data was stored in each storage section prior to transmission to the requestor.

14. The system of claim 13, wherein, when transmitting each portion of the requested range of data identified in the first data, the second data, and the third data, the processor is configured to transmit each portion of the requested range of data identified in the first data, the second data, and the third data in the order in which the data was stored.

15. A computer program product comprising a computer readable storage medium including a computer readable program code embodied therein for reading data from a transformed logical object comprising a plurality of transformed storage sections, said computer program product comprising:

computer code for receiving, by a processor, a plurality of data chunks for storage in the transformed logical object;

computer code for comparing a characteristic of each data chunk to a predefined criterion comprising an estimated or actual time of transformation of each data chunk;

computer code for transforming each data chunk and accumulating each transformed data chunk in a first storage section if the predefined characteristic matches the characteristic of each particular data chunk;

computer code for identifying each data chunk as being non-transformed or partly transformed and accumulating each non-transformed data chunk and each partly transformed data chunk in a second storage section if the predefined characteristic does not match the characteristic of each particular data chunk;

computer code for receiving, by the processor, a read request to read a range of data stored in the transformed logical object, wherein the read request includes a starting point in time at which the data was stored,

computer code for identifying the range of data utilizing the starting point in time included in the read request;

computer code for selecting a last-updated data related to the range, wherein the last-updated data is included in an index entry that facilitates a one-to-one relationship between the range of data and live data to be extracted from the transformed logical object;

computer code for de-transforming the first storage section including a first portion of the live data to generate first data;

computer code for determining that the second storage section includes a second portion of the live data;

computer code for de-transforming the partly transformed data chunks in the second storage section to generate second data comprising a combination of the de-transformed partly transformed data chunks and the non-transformed data chunks;

computer code for identifying each portion of the requested range of data included in the first data and the second data;

computer code for storing each portion of the requested range of data identified in the first data and the second data in cache; and

computer code for each portion of the requested range of data identified in the first data and the second data from the cache to a requestor, wherein;

the first portion of the live data is less than the first storage section in its entirety, and

the second portion of the live data is less than the second storage section in its entirety.

16. The computer program product of claim 15, further comprising computer code for arranging each portion of the requested range of data identified in the first data and the second data in an order in which the data was transformed prior to transmission to the requestor.

17. The computer program product of claim 16, further comprising computer code for transmitting each portion of the requested range of data identified in the first data and the second data in an order in which the data was stored.

18. The computer program product of claim 15, further comprising:

computer code for determining that one or more additional storage sections includes one or more additional portions of the live data;

computer code for de-transforming each of the one or more additional storage sections to generate third data; and

computer code for identifying each portion of the requested range of data within the third data.

19. The computer program product of claim 18, further comprising:

computer code for storing, in the cache, each portion of the requested range of data identified in the third data; and

computer code for transmitting each portion of the requested range of data identified in the third data together with each portion of the requested range of data identified in the first data and in the second data to the requestor.

20. The computer program product of claim 19, further comprising computer code for arranging each portion of the requested range of data identified in the first data, the second data, and the third data in an order in which the data was stored in each storage section prior to transmission to the requestor.

21. The computer program product of claim 20, wherein the computer code for transmitting each portion of the requested range of data identified in the first data, the second data, and the third data comprises computer code for transmitting each portion of the requested range of data identified in the first data, the second data, and the third data in the order in which the data was stored.