METHOD AND SYSTEM FOR TRANSFORMATION OF LOGICAL DATA OBJECTS FOR STORAGE

• US 20090307249A1
• Filed: 05/31/2007
• Published: 12/10/2009
• Est. Priority Date: 05/31/2006
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: Storwize Limited
• Inventors: Kedem, Nadav, Zohar, Avi, Koifman, Haim
• Granted Patent: US 9,176,975 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 12/023 Free address space management
• CPC Class Codes: G06F 12/1408 by using cryptography for digital transmission H04L9/00
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/1727 Details of free space management performed by the file system saving storage space on storage systems G06F3/0608; management of blocks in storage devices G06F3/064
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 21/62 Protecting access to data via a platform, e.g. using keys or access control rules
• CPC Class Codes: G06F 21/6209 to a single file or object, e.g. in a secure envelope, encrypted and accessed using a key, or with access control rules appended to the object itself
• CPC Class Codes: G06F 21/78 to assure secure storage of data address-based protection against unauthorised use of memory G06F12/14; record carriers for use with machines and with at least a part designed to carry digital markings G06K19/00
• CPC Class Codes: G06F 2212/1044 Space efficiency improvement
• CPC Class Codes: G06F 2212/152 Virtualized environment, e.g. logically partitioned system
• CPC Class Codes: G06F 2212/401 Compressed data
• CPC Class Codes: G06F 2212/402 Encrypted data
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0623 in relation to content
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0638 Organizing or formatting or addressing of data
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0661 : Format or protocol conversion arrangements
• CPC Class Codes: G06F 3/0665 : at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 : Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0673 : Single storage device
• CPC Class Codes: H03M 7/30 : Compression speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N; Expansion; Suppression of unnecessary data, e.g. redundancy reduction

Abstract

A method and system for transforming a logical data object for storage in a storage device operable with at least one storage protocol, creating, reading, writing, optimization and restoring thereof. Transforming the logical data object comprises creating in the storage device a transformed logical data object comprising a header and one or more allocated accommodation sections with predefined size; processing one or more sequentially obtained chunks corresponding to the transforming logical data object thus giving rise to the processed data chunks, wherein at least one of said processed data chunks comprises transformed data resulting from said processing; sequentially accommodating the processed data chunks into said accommodation sections in accordance with an order the chunks received, and generating at least one index section constituting a part of the transformed logical data object, said index section comprising one or more entries containing indicators to physical storage location of respective accommodation sections and, with respect to each processed data chunk, one or more records related to said chunk, wherein said records indicate offset and length of data in the logical data object to be read from the respective processed data chunk, and are associated with a time stamp indicating time of creating said record. The method further facilitates reading a data range from the transformed logical object in response to a read request specifying desired point in time to be read.

Claims

1. A method of transforming a logical data object for storage in a storage device operable with at least one storage protocol, said method comprising:

a) in response to a respective request, creating in the storage device a transformed logical data object comprising a header and one or more allocated accommodation sections with predefined size;

b) processing one or more obtained variable size chunks corresponding to the transforming logical data object thus giving rise to the processed data chunks, wherein at least one of said processed data chunks comprises transformed data resulting from said processing;

c) sequentially accommodating the processed data chunks into said accommodation sections in accordance with an order the chunks received for accommodation, and

d) generating at least one index section constituting a part of the transformed logical data object, said index section comprising one or more entries containing indicators to physical storage location of respective accommodation sections and, with respect to each processed data chunk, one or more records related to said chunk, wherein said records indicate offset and length of data in the logical data object to be read from the respective processed data chunk, and are associated with a time stamp indicating time of creating said record.

2. A method of writing a data range to the transformed logical data object created in accordance with claim 1, said method comprising:

a) in response to respective request, processing one or more obtained data chunks corresponding to said data range, wherein at least one of the processed data chunks comprises transformed data resulting from said processing;

b) sequentially accommodating the processed data chunks in accordance with an order these and previous chunks received for accommodation; and

c) updating one or more index sections, said updating comprising;

i) adding records with respect to new accommodated data chunks, said records comprising information related to the offset and length of data in the logical data object to be read from the respective processed data chunks, and associating these records with corresponding time stamp;

ii) generating updated records with respect to previously accommodated chunks if offset and length of data in the logical data object to be read from respective processed data chunks have been changed; and

associating the records with corresponding time stamp, thus giving rise to updated records;

iii) keeping the records bearing prior time stamps alongside the updated records.

3. A method of reading a data range from a transformed logical object written in accordance with claim 2, said method comprising:

a) obtaining a read request specifying desired point in time to be read;

b) in response to the request, discovering all relevant records in one or more index sections, said records related to the data within the range and bearing time stamps prior or equal to time corresponding to the desired point in time;

c) selecting among the discovered relevant records the most updated records;

d) de-transforming one of the accommodation sections corresponding to said selected records, and extracting the data to be read in accordance with the mapping provided by the records;

e) repeating operation d) to one or more other accommodation sections corresponding to the selected records until extracting all data from the range; and

f) arranging the extracted data in accordance with their order in the range.

4. The method of claim 3 wherein the read operation is be provided for several desired points in time simultaneously.

5. The method of claim 1 further comprising optimization of the transformed logical data object, said optimization including:

a) identifying one or more accommodation sections comprising more than a certain percent of processed data chunk with respective records bearing time stamps meeting certain criterion, thus giving rise to outdated accommodation sections;

b) de-transforming the identified outdated sections and extracting live data;

c) transforming the extracted live data and sequentially accommodating as one or more new processed chunks; and

d) releasing the outdated accommodation sections from the transformed logical data object.

6. The method of claim 1 wherein a transforming technique is selected from a group comprising compression, encryption, encoding, conversion, a combination thereof.

7. The method of claim 1 wherein the predefined size is equal for all accommodation sections.

8. The method of claim 1 wherein the size of the accommodation section is selected from a list of predefined sizes in accordance with certain criterion.

9. The method of claim 1 wherein the header of transformed logical data object comprises a unique descriptor of the transformed logical data object and information related to a size of the transforming logical data object.

10. The method of claim 1 wherein the processed data chunks are accommodated in a log form.

11. A communication device operable in a storage network being configured to perform the method stages of claim 1.

12. A storage device operable in a storage network being configured to perform the method stages of claim 1.

13. A system capable of transforming a logical data object for storage in a storage device operatively coupled to the system in a serial manner, said system acting as a transparent bridge with respect to the storing data and being configured to perform the method stages of claim 1.

14. The method of claim 1 operable with at least file access storage protocol.

15. The method of claim 1 operable with at least block mode access storage protocol.

16. The method of claim 1 applicable to continuous data protection.

17. A method of compressing a raw logical data object for storage in a storage device operable with at least one storage protocol, said method comprising:

a) in response to a respective request, creating in the storage device a compressed logical data object comprising a header and one or more allocated compressed sections with predefined size;

b) processing one or more obtained variable size chunks corresponding to the raw logical data object thus giving rise to the compressed data chunks;

c) sequentially accommodating the compressed data chunks into said compressed sections in accordance with an order the chunks received for accommodation, and

d) generating at least one index section constituting a part of the compressed logical data object, said index section comprising one or more entries containing indicators to physical storage location of respective compressed sections and, with respect to each compressed data chunk, one or more records related to said chunk, wherein said records indicate offset and length of data in the logical data object to be read from the respective compressed data chunk, and are associated with a time stamp indicating time of creating said record.

18. A method of writing a data range to the compressed logical data object created in accordance with claim 17, said method comprising:

a) in response to respective request, processing one or more obtained data chunks corresponding to said data range, wherein at least one of the compressed data chunks comprises compressed data resulting from said processing;

b) sequentially accommodating the compressed data chunks in accordance with an order these and previous chunks received for accommodation; and

c) updating one or more index sections, said updating comprising;

i) adding records with respect to new accommodated data chunks, said records comprising information related to the offset and length of data in the logical data object to be read from the respective compressed data chunks, and associating these records with corresponding time stamp;

ii) generating updated records with respect to previously accommodated chunks if offset and length of data in the logical data object to be read from respective compressed data chunks have been changed; and

associating the records with corresponding time stamp, thus giving rise to updated records;

iii) keeping the records bearing prior time stamps alongside the updated records.

19. A method of reading a data range from a compressed logical object written in accordance with claim 18, said method comprising:

a) obtaining a read request specifying desired point in time to be read;

b) in response to the request, discovering all relevant records in one or more index sections, said records related to the data within the range and bearing time stamps prior or equal to time corresponding to the desired point in time;

c) selecting among the discovered relevant records the most updated records;

d) decompressing one of the compressed sections corresponding to said selected records, and extracting the data to be read in accordance with the mapping provided by the records;

e) repeating operation d) to one or more other compressed sections corresponding to the selected records until extracting all data from the range; and

f) arranging the extracted data in accordance with their order in the range.

20. A system capable of transforming a logical data object for storage in a storage device operable with at least one storage protocol, said system comprising:

a) means for creating in the storage device a transformed logical data object comprising a header and one or more allocated accommodation sections with predefined size;

b) means for processing one or more obtained variable size chunks corresponding to the transforming logical data object thus giving rise to the processed data chunks, wherein at least one of said processed data chunks comprises transformed data resulting from said processing;

c) means for sequentially accommodating the processed data chunks into said accommodation sections in accordance with an order the chunks received for accommodation,

d) means for generating at least one index section constituting a part of the transformed logical data object, said index section comprising one or more entries containing indicators to physical storage location of respective accommodation sections and, with respect to each processed data chunk, one or more records related to said chunk, wherein said records indicate offset and length of data in the logical data object to be read from the respective processed data chunk, and are associated with a time stamp indicating time of creating said record; and

e) means for reading a data range from the transformed logical object in response to a read request specifying desired point in time to be read.

21. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps of transforming a logical data object for storage in a storage device operable with at least one storage protocol, said method comprising:

a) in response to a respective request, creating in the storage device a transformed logical data object comprising a header and one or more allocated accommodation sections with predefined size;

b) processing one or more obtained variable size chunks corresponding to the transforming logical data object thus giving rise to the processed data chunks, wherein at least one of said processed data chunks comprises transformed data resulting from said processing;

c) sequentially accommodating the processed data chunks into said accommodation sections in accordance with an order the chunks received for accommodation, and

d) generating at least one index section constituting a part of the transformed logical data object, said index section comprising one or more entries containing indicators to physical storage location of respective accommodation sections and, with respect to each processed data chunk, one or more records related to said chunk, wherein said records indicate offset and length of data in the logical data object to be read from the respective processed data chunk, and are associated with a time stamp indicating time of creating said record.

22. A computer program product comprising a computer usable medium having computer readable program code embodied therein of transforming a logical data object for storage in a storage device operable with at least one storage protocol, said computer program product comprising:

a) computer readable program code for causing the computer to in response to a respective request, creating in the storage device a transformed logical data object comprising a header and one or more allocated accommodation sections with predefined size;

b) computer readable program code for causing the computer to process one or more obtained variable size chunks corresponding to the transforming logical data object thus giving rise to the processed data chunks, wherein at least one of said processed data chunks comprises transformed data resulting from said processing;

c) computer readable program code for causing the computer to sequentially accommodating the processed data chunks into said accommodation sections in accordance with an order the chunks received for accommodation, and

d) computer readable program code for causing the computer to generate at least one index section constituting a part of the transformed logical data object, said index section comprising one or more entries containing indicators to physical storage location of respective accommodation sections and, with respect to each processed data chunk, one or more records related to said chunk, wherein said records indicate offset and length of data in the logical data object to be read from the respective processed data chunk, and are associated with a time stamp indicating time of creating said record.