System and method of storage optimization

• US 8,458,145 B2
• Filed: 01/20/2011
• Issued: 06/04/2013
• Est. Priority Date: 01/20/2011
• Status: Active Grant

• Current Assignee: Infinidat Ltd.
• Original Assignee: Infinidat Ltd.
• Inventors: Kopylovitz, Haim, Satran, Julian, Yochai, Yechiel
• Primary Examiner(s): CHANNAVAJJALA, SRIRAMA T
• Application Number:
  US13/010,337
• Publication Number:
  US 20120191667A1
• Time in Patent Office: 866 Days
• Field of Search: 707/609, 707/694, 707697-699, 707/713, 707/715, 707821-831, 707/637, 707/648, 707/664, 707/692, 707/747, 707/812, 709/203, 709/206, 709218-223, 711/154, 711/209
• US Class Current: 707/692
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD

Abstract

A method and system are disclosed for storage optimization. Data parts and metadata within a source data unit are identified and the data parts are compared with data which is already stored in the physical storage space. In case identical data parts are found within the physical storage, the data parts from the source data unit are linked to the identified data, while the data parts can be discarded, thereby reducing the required storage capacity. The metadata parts can be separately stored in a designated storage area.

Claims

1. A storage system, the system comprising:

a storage optimization (SO) module operatively coupled to a physical storage space storing a plurality of data units, each data unit of said plurality of data units comprising a data art and a metadata part, wherein said metadata part is added to the data unit by an application that is external to the SO module and wherein the SO module is configured to deduplicate data units that differ only in metadata parts comprised therein, by;

differentiate a first data part from a first metadata part, wherein said first data part and said first metadata part are sequentially comprised within a first data unit; and

identify a second data unit among said plurality of data units that is partly identical to said first data unit, said second data unit comprises a second data part, which is identical to said first data part, and a second metadata part, which is different from said first metadata part;

associate a logical address corresponding to said first data part with a physical address corresponding to said second data part;

store said first metadata part in a designated area in said physical storage space and associate a logical address corresponding to said first metadata part with a physical address corresponding to said designated area; and

discard said first data part of said first data unit, thereby reducing volume of stored data in said physical storage space.

2. The system of claim 1 wherein said designated area is a certain data unit from among said plurality of data units, designated for storing metadata parts of multiple partly identical data units.

3. The system of claim 1 wherein said SO module is further operable to utilize information in respect of a format of said first data unit including at least a size of said first metadata part and its location in respect of said first data unit and is operable to identify said first data part and said first metadata part within said data unit and to differentiate said first data part from said first metadata part, based on said information.

4. The system of claim 1 wherein said SO module is operable to identify a third data among said plurality of data units, having a third data part which is identical to said first data part, and having a third metadata part, which is identical to said first metadata part, by comparing said first data unit with data units from said plurality of data units;

associate a logical address corresponding to said first data unit with a physical address corresponding to said third data unit and discard said first data unit.

5. The system of claim 1 wherein said SO module is operable to calculate a hash value for said first data part;

wherein said identifying is performed by comparing said hash value with other hash values calculated for data parts of at least part of said plurality of data units.

6. The system of claim 1 wherein the SO module is further operable to divide a data unit into a plurality of subunits, and wherein said first data unit is one of subunits generated by such division.

7. The system of claim 6 wherein said SO module is further operable to concatenate data parts corresponding to said plurality of subunits within said data unit and compare said concatenated data parts with concatenated data parts corresponding to other data units stored within said physical storage space.

8. The system of claim 1 wherein said SO module is operatively connectable to one or more hosts, external to the storage system, and is operable to receive said first data unit from said one or more hosts.

9. The system of claim 1 wherein said SO module is operatively connectable to one or more hosts and, in response to a read request issued by a host requesting to read said first data unit operable to perform at least the following:

identify that said first metadata part is characterized by a logical address associated with said designated area;

retrieve said first metadata part from said designated area based on its physical address;

retrieve said first data part from the physical address corresponding to said second data unit;

combine said first data part and said first metadata part and reconstruct said first data unit.

10. The system of claim 1 wherein said SO module is further operable to determine whether to perform the deduplication based on current storage space availability within said physical storage space and/or current load on processing resources of said storage system.

11. An SO (storage optimization) module operatively connectable to a storage control layer in a storage system and coupled to a physical storage space storing a plurality of data units, each data unit of said plurality of data units comprising a data part and a metadata part, wherein said metadata part is added to the data unit by an application that is external to the SO module and wherein the SO module is configured to deduplicate data units that differ only in metadata parts comprised therein, by:

differentiate a first data part from a first metadata part, wherein said first data part and said first metadata part are sequentially comprised within a first data unit; and

identify a second data unit among said plurality of data units that is partly identical to said first data unit, said second data unit comprises a second data part, which is identical to said first data part, and a second metadata part, which is different from said first metadata part;

associate a logical address corresponding to said first data part with a physical address corresponding to said second data part;

store said first metadata part in a designated area in said physical storage space and associate a logical address corresponding to said first metadata part with a physical address corresponding to said designated area; and

discard said first data part of said first data unit, thereby reducing volume of stored data in said physical storage space.

12. A method of allocating data to a physical data storage in a storage system comprising a storage optimization (SO) module operatively coupled to a physical storage space storing a plurality of data units, each data unit of said plurality of data units comprising a data part and a metadata part, the method comprising:

differentiating, by the SO module, a first data part from a first metadata part, wherein said first data part and said first metadata part are sequentially comprised within a first data unit, and wherein said first metadata part is added to said first data unit by an application that is external to the SO module prior to the step of differentiating;

identifying, by the SO module, a second data unit among said plurality of data units that is partly identical to said first data unit, said second data unit comprises a second data part, which is identical to said first data part, and a second metadata part, which is different from said first metadata part;

associating a logical address corresponding to said first data part with a physical address corresponding to said second data part;

storing said first metadata part in a designated area in said physical storage space and associate a logical address corresponding to said first metadata part with a physical address corresponding to said designated area; and

discarding said first data part of said first data unit, thereby reducing volume of stored data in said physical storage space.

13. The method of claim 12 wherein said designated area is a certain data unit from among said plurality of data units, designated for storing metadata parts of multiple partly identical data units.

14. The method of claim 12 further comprising:

utilizing information in respect of a format of said first data unit including at least a size of said first metadata part and its location in respect of said first data unit and wherein the differentiating of said first data part from said first metadata part is based on said information.

15. The method of claim 12 further comprising:

identifying a third data unit among said plurality of data units, having a third data part which is identical to said first data part, and having a third metadata part, which is identical to said first metadata part, by comparing said first data unit with data units from said plurality of data units, thereby giving rise to identical data units;

associating a logical address corresponding to one of said identical data units with a physical address of another identical data unit of said identical data units and discard said one of said identical data units.

16. The method of claim 12 further comprising:

calculating a hash value for at least said first data part;

wherein said identifying is performed by comparing said hash value with other hash values calculated for data parts of at least part of said plurality of data units.

17. The method of claim 16 further comprising dividing a data unit into a plurality of subunits, and wherein said first data unit is one of the subunits generated by such division.

18. The method of claim 17 further comprising concatenating data parts corresponding to said plurality of subunits within said data unit and comparing said concatenated data parts with concatenated data parts corresponding to other data units stored within said physical storage space.

19. The method of claim 12 wherein said storage system is operatively connectable to one or more hosts, external to the storage system, and wherein the method comprising receiving said first data unit from said one or more hosts.

20. The method of claim 12 further comprising:

receiving a read request issued by a host requesting for a data unit comprising said first data part and said first metadata part and in response;

identifying that said first metadata part is characterized by a logical address associated with said designated area;

retrieving said first metadata part from said designated area based on its physical address;

retrieving said first data part from the physical address corresponding to said second data unit;

combining said first data part and said first metadata part and reconstructing said first data unit.

21. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps of allocating data to a physical data storage associated with a storage system comprising a storage optimization (SO) module operatively coupled to a plurality of physical storage devices storing a plurality of data units, each data unit of said plurality of data units comprising a data part and a metadata part, the method comprising:

differentiating, by the SO module, a first data part from a first metadata part, wherein said first data part and said first metadata part are sequentially comprised within a first data unit, and wherein said first metadata part is added to said first data unit by an application that is external to the SO module prior to the step of differentiating;

identifying, by the SO module, a second data unit among said plurality of data units that is partly identical to said first data unit, said second data unit comprises a second data part, which is identical to said first data part, and a second metadata part, which is different from said first metadata part;

associating a logical address corresponding to said first data part with a physical address corresponding to said second data part;

storing said first metadata part in a designated area in said physical storage space and associate a logical address corresponding to said first metadata part with a physical address corresponding to said designated area; and

discarding said first data part of said first data unit, thereby reducing volume of stored data in said physical storage space.