Method and apparatus for reducing the amount of data in a storage system

• US 20070226444A1
• Filed: 03/22/2006
• Published: 09/27/2007
• Est. Priority Date: 03/22/2006
• Status: Active Grant

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Yagawa, Yuichi
• Granted Patent: US 7,457,934 B2
• Time in Patent Office: Days
• US Class Current: 711/170
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0638 Organizing or formatting or addressing of data
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Groups of volumes having a certain amount of commonality of data are identified within a storage system. For each group, the process defines a base volume, such as the volume containing the most commonality with the other volumes. For each volume in the group, the process extracts differential data between the base volume and each other volume, saves the differential data in a pool volume, and updates a mapping table. Commonality between volumes in the group is managed in the base volume, the pool volume, and the mapping table. Then the original volumes except the base volume and the pool volume may be deleted, and the original volumes are replaced with virtual volumes accessible through the mapping table. As a result, the amount of data in a storage system can be reduced while maintaining data integrity and accessibility.

Claims

1. A method for storing data in a storage system comprising:

determining a group of volumes having a commonality of data;

defining a base volume for the group;

extracting differential data between the base volume and each other volume in said group;

saving said differential data in a pool volume;

updating a mapping table regarding a location of the differential data in said pool volume.

2. The method of claim 1, further including the step of:

defining as the base volume a volume in said group having a greatest degree of commonality of data in relation to the other volumes in said group.

3. The method of claim 2, further including the steps of:

calculating distances in commonality for each volume with respect to the other volumes in said group and said base volume;

summing said distances;

sorting the volumes in order of the summed distances; and

setting the volume with a smallest summed distance as the base volume.

4. The method of claim 1, further including the steps of:

defining the base volume by creating a new volume as the base volume, and

storing data in the new volume based upon a comparison of chunks of data in the volumes of the group, whereby data corresponding to the chunks having the greatest commonality among corresponding chunks of the volumes in the group is stored in the base volume.

5. The method of claim 1, further including the step of:

deleting the other volumes other than the pool volume and the base volume so as to reduce the amount of data stored in the storage system.

6. The method of claim 1, further including the steps of:

determining the group of volumes by extracting a temporary group of volumes having a same size;

for each volume in the temporary group;

dividing the volume into chunks, and for each chunk setting a code for the chunk corresponding to approximate data content;

calculating distances of commonality of the volume from the other volumes in the temporary group by a comparison of encoded chunks;

setting the volume having a smallest distance from the other volumes as a temporary base volume;

calculating the distances of the other volumes in the temporary group from the temporary base volume, and comparing the calculated distances with a predetermined distance threshold to determine whether each volume should remain part of the group of volumes.

7. The method of claim 1, further including the step of extracting differential data by:

dividing each volume into chunks and setting a code for each chunk representing approximate data content;

comparing the code for each chunk of each other volume with the code for the corresponding chunk of the base volume, and, if the codes are the same,

comparing the data corresponding to the chunk of the other volume with the data of the base volume corresponding to the chunk of the base volume, and, if the compared data is the same,

storing a pointer to the chunk of the base volume in the mapping table as the location of the data of the other volume corresponding to the chunk of the other volume.

8. The method of claim 7, wherein if the codes are different or the compared data is different, further including the steps of:

comparing the code for each chunk of each other volume with the codes of any corresponding chunks in the pool volume, and, if the codes are the same,

comparing the data corresponding to the chunk of the other volume with the data of the pool volume corresponding to the chunk of the pool volume, and, if the compared data is the same,

storing a pointer to the chunk of the pool volume in the mapping table as the location of the data of the other volume corresponding to the chunk of the other volume.

9. The method of claim 8, wherein if the codes of the other volume and the pool volume are different or the compared data of the other volume and pool volume is different, further including the steps of:

saving the data corresponding to the chunk of the other volume as differential data in the pool volume, and

storing a pointer to the chunk in the mapping table as the location of the data of the other volume corresponding to the chunk of the other volume.

10. The method of claim 6, wherein the temporary group of volumes is determined using configuration information of data replication.

11. The method of claim 6, wherein the temporary group of volumes is determined by discovering copy configuration files and determining therefrom volumes having a particular copy relation.

12. The method of claim 2, wherein said base volume is not an original parent volume of the volumes in the group.

13. The method of claim 1, wherein said volumes are offline and not storing any new write requests, whereby any write requests are stored in a buffer.

14. The method of claim 3, further including the step of calculating the degree to which the stored data can be reduced based upon the calculated distances in commonality for each volume with respect to the other volumes in said group and said base volume.

15. A method of reducing an amount of data stored in a storage system comprising the steps of:

selecting a first set volumes in which an amount of data is to be reduced;

defining a group of volumes in the first set of volumes, said group of volumes having a determined degree of redundant data;

within said group, defining a base volume and storing differential data in a differential volume for each other volume in said group relative to said base volume, the differential data representing a difference between the base volume and each other volume;

storing a mapping relationship of each other volume, wherein the mapping relationship points to data of each other volume as being stored in either the base volume or the differential volume; and

deleting the other volumes while retaining the base volume and the differential volume.

16. The method according to claim 15, further including the step of:

defining said base volume from an existing volume having a calculated greatest degree of commonality with the other volumes in said group.

17. The method according to claim 15, further including the step of:

defining base volume from a newly created volume said newly created volume, and

storing data in the new volume based upon a comparison of chunks of data in the volumes of the group, whereby data corresponding to the chunks having the greatest commonality among corresponding chunks of the volumes in the group is stored in the base volume.

18. The method of claim 15, further including the step of extracting differential data by:

dividing each volume into chunks and setting a code for each chunk representing approximate data content;

comparing the code for each chunk of each other volume with the code for the corresponding chunk of the base volume, and, if the codes are the same,

comparing the data corresponding to the chunk of the other volume with the data of the base volume corresponding to the chunk of the base volume, and, if the compared data is the same,

storing a pointer to the chunk of the base volume in the mapping table as the location of the data of the other volume corresponding to the chunk of the other volume.

19. The method of claim 18, wherein if the codes are different or the compared data is different, further including the steps of:

comparing the code for each chunk of each other volume with the codes of any corresponding chunks in the differential volume, and, if the codes are the same,

comparing the data corresponding to the chunk of the other volume with the data of the differential volume corresponding to the chunk of the differential volume, and, if the compared data is the same,

storing a pointer to the chunk of the differential volume in the mapping table as the location of the data of the other volume corresponding to the chunk of the other volume.

20. The method of claim 19, wherein if the codes of the other volume and the differential volume are different or the compared data of the other volume and differential volume is different, further including the steps of:

saving the data corresponding to the chunk of the other volume as differential data in the differential volume, and

storing a pointer to the chunk in the mapping table as the location of the data of the other volume corresponding to the chunk of the other volume.

21. A method for reducing the amount of stored data in a storage system comprising:

determining a group of volumes having a commonality of data;

extracting differential data between a pool volume and each volume in said group;

saving said differential data in the pool volume;

updating a mapping table regarding the location of the differential data in said pool volume; and

deleting the volumes of the group while maintaining the pool volume and the mapping table.

22. The method of claim 21, further including the step of extracting differential data by:

dividing each group volume into chunks and setting a code for each chunk representing approximate data content;

comparing the code for each chunk of each group volume with the codes of any chunks in the pool volume, and, if the codes are the same,

comparing the data corresponding to the chunk of the group volume with the data corresponding to the chunk of the pool volume, and, if the compared data is the same,

storing a pointer to the chunk of the pool volume in the mapping table as the location of the data of the group volume corresponding to the chunk of the group volume.

23. The method of claim 22, wherein if the code of the group volume and the code of the pool volume are different or the compared data of the group volume and the pool volume are different, further including the steps of:

saving the data corresponding to the chunk of the group volume as differential data in the pool volume, and

storing a pointer to the chunk in the mapping table as the location of the data of the group volume corresponding to the chunk of the group volume.

24. A system comprising:

at least one storage system having a plurality of volumes containing data;

a virtual volume providing module;

a mapping table; and

a data reduction module,

wherein for a group of said volumes having a commonality of data, the data reduction module;

defines a base volume for the group,

divides the base volume and other volumes in said group into chunks of a predetermined size,

extracts differential data between the base volume and each other volume in said group for each said chunk,

saves each chunk having said differential data in a pool volume, and

updates the mapping table regarding a location of chunks containing the differential data in said pool volume or a location of chunks not containing differential data in said base volume.

25. The system of claim 24, further comprising:

a virtualization system coupled to said at least one storage system,

wherein said data reduction module, said virtual volume providing module, and said mapping table are stored on said virtualization system.