Method and apparatus for de-duplication after mirror operation

• US 7,873,809 B2
• Filed: 03/29/2007
• Issued: 01/18/2011
• Est. Priority Date: 03/29/2007
• Status: Expired due to Fees

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Kano, Yoshiki
• Primary Examiner(s): Gu; Shawn X
• Application Number:
  US11/727,946
• Publication Number:
  US 20080244172A1
• Time in Patent Office: 1,391 Days
• Field of Search: None
• US Class Current: 711/216
• CPC Class Codes: G06F 11/2069 Management of state, configuration or failover
• CPC Class Codes: G06F 11/2071 using a plurality of controllers
• CPC Class Codes: G06F 11/2082 Data synchronisation
• CPC Class Codes: G06F 16/221 Column-oriented storage; Management thereof

Abstract

An amount of storage capacity used during mirroring operations is reduced by applying de-duplication operations to the mirror volumes. Data stored to a first volume is mirrored to a second volume. The second volume is a virtual volume having a plurality of logical addresses, such that segments of physical storage capacity are allocated for a specified logical address as needed when data is stored to the specified logical address. A de-duplication operation is carried out on the second volume following a split from the first volume. A particular segment of the second volume is identified as having data that is the same as another segment in the second volume or in the same consistency group. A link is created from the particular segment to the other segment and the particular segment is released from the second volume so that physical storage capacity required for the second volume is reduced.

Claims

1. A method of operating an information system, comprising:

storing data to a first volume and mirroring the data to a second volume, said second volume being a virtual volume having a plurality of logical storage addresses, wherein a segment of physical storage capacity is allocated for a specified logical address as required when the data is to be stored to said specified logical address;

selecting segments of the second volume during a de-duplication operation on the second volume;

calculating a hash value for a particular segment representative of data contained in the particular segment;

comparing the calculated hash value with previously-stored hash values for other segments in the second volume;

linking the particular segment to another segment having a previously-stored hash value that matches the calculated hash value of the particular segment;

releasing the particular segment from the second volume when another segment has a previously-stored hash value that matches the calculated hash value of the particular segment, whereby physical storage capacity required for the second volume is reduced;

resynchronizing the data stored on said second volume with the data stored on said first volume;

splitting said second volume from mirroring said first volume prior to said step of selecting segments of the second volume during the de-duplication operation on the second volume; and

reverse synchronizing the data stored on said first volume with data stored on said second volume, wherein before reverse synchronizing whether the second volume has been de-duplicated or not is checked and if the second volume has been de-duplicated, said reverse synchronizing is conducted on to the first volume from the de-duplicated volume.

2. A method according to claim 1, further including steps of

providing a first storage system in communication with a host computer, said first storage system including a first controller and multiple first disk devices, said first volume representing physical storage capacity on said first disk devices;

and

providing said second volume on said first storage system as a local mirror of said first volume.

3. A method according to claim 1, further including steps of

providing a first storage system in communication with a host computer, said first storage system including a first controller and multiple first disk devices, said first volume representing physical storage capacity on said first disk devices;

providing a second storage system in communication with said first storage system and in a location separate from said first storage system, said second storage system including a second controller and multiple second disk devices;

providing said second volume on said second storage system as a remote mirror of said first volume; and

checking whether the second volume has been designated for de-duplication operations before said selecting segments of the second volume,

wherein said de-duplication operation on the second volume is performed after pair configuration of the second volume to the first volume is suspended and determined that the second volume has been designated for de-duplication operations based on an information maintained in a memory of said information system.

4. A method according to claim 3, further including steps of

providing a plurality of said first volumes and a plurality of said second volumes for forming a consistency group in which each first volume is mirrored to one of said second volumes;

wherein said step of comparing the calculated hash value with previously-stored hash values for other segments in the virtual volume includes comparing the calculated hash value with previously-stored hash values for segments of all said second volumes in the consistency group, and

wherein second volumes in the same consistency group have the same status on whether the second volume has been designated for de-duplication operations, and said status indicated by a flag in said memory.

5. A method according to claim 1, further including steps of

providing said first volume and said second volume on a network attached storage system;

and

creating a first file system on said first volume and a second file system on said second volume, said second file system being a copy of said first file system, such that said second file system mirrors said first file system prior to said de-duplication operation.

6. A method according to claim 1, further including a step of

providing a first virtualization apparatus for presenting the first volume and the second volume to a host computer, said first virtualization apparatus being in communication with a first storage system, said first storage system including a first controller for communicating with said first virtualization apparatus and a plurality of disk devices for providing physical storage space for said first volume and said second volume.

7. A method according to claim 1, further including a step of

providing a first virtualization apparatus for presenting the first volume to a host computer, said first virtualization apparatus being in communication with a first storage system, said first storage system including a first controller for communicating with said first virtualization apparatus and a plurality of first disk devices for providing physical storage space for said first volume;

and

providing a second virtualization apparatus for presenting the second volume to a host computer, said second virtualization apparatus being in communication with a second storage system, said second storage system including a second controller for communicating with said second virtualization apparatus and a plurality of second disk devices for providing physical storage space for said second volume.

8. A method according to claim 1, further including a step of

reading requested data from said second volume following the de-duplication operation by:

identifying a requested logical block address;

determining a segment of said second volume corresponding to the identified logical block address; and

when the segment corresponding to the logical block address is linked to another segment, reading the requested data from the linked other segment.

9. A method according to claim 1, wherein a portion of the mirror data stored in the second volume is not selected for the de-duplication operation.

10. An information system, comprising:

a first storage system including a first controller and a plurality of storage devices, said first controller configured to present physical storage space on said storage devices as volumes for storage of data;

a first host computer able to communicate with said first storage system via a network; and

a second storage system in communication with said first storage system, said second storage system including a second controller and a plurality of second disk devices, said second storage system presenting said second volume as said virtual volume and being configured to carry out said de-duplication operation;

wherein said first storage system is configured to store write data received from said computer to a first volume on said first storage system and mirror the write data to a second volume, said second volume being a virtual volume having a plurality of logical addresses, wherein segments of physical storage capacity are allocated for a logical address as required when the write data is stored to said logical address;

wherein said second volume is de-duplicated by selecting segments of the second volume during a de-duplication operation and a hash value is calculated for a particular segment representative of data contained in the particular segment;

wherein said calculated hash value is compared with previously-stored hash values for other segments in the second volume, and the particular segment is linked to another segment having a previously-stored hash value that matches the calculated hash value of the particular segment,

wherein the particular segment is released from the second volume when another segment has a previously-stored hash value that matches the calculated hash value of the particular segment, whereby physical storage capacity required for the second volume is reduced;

wherein before said selecting segments of the second volume whether the second volume has been designated for de-duplication operations is checked, and

wherein said de-duplication operation on the second volume is performed after pair configuration of the second volume to the first volume is suspended and determined that the second volume has been designated for de-duplication operations based on an information maintained in a memory of said information system.

11. The system according to claim 10, further comprising:

a first virtualization apparatus for presenting the first volume and the second volume to the first host computer, said virtualization apparatus being in communication with the first storage system and the first host computer, wherein said first volume and said second volume are virtual volumes that map to corresponding volumes on said first storage system.

12. The system according to claim 10, further comprising:

a first virtualization apparatus for presenting the first volume to the first host computer, said first virtualization apparatus being in communication with the first storage system and the first host computer, wherein said first volume is a first virtual volume that maps to a corresponding logical volume on said first storage system;

a second virtualization apparatus for presenting the second volume to a second host computer, said second virtualization apparatus being in communication with the second storage system and the second host computer.

13. The system according to claim 10, further comprising:

an intelligent switch for presenting the first volume and the second volume to a host computer, said intelligent switch being in communication with the first storage system and the host computer, wherein said first volume and said second volume are virtual volumes that map to corresponding volumes on said first storage system.

14. The system according to claim 10, further comprising:

said first storage system being a network attached storage system, wherein a first file system is created on said first volume and a second file system is created on said second volume, said second file system being a copy of said first file system, such that said second file system mirrors said first file system prior to said de-duplication operation on the second volume.

15. The system according to claim 10, further comprising:

said first storage system being a first network attached storage system, wherein a first file system is created on said first volume; and

said second storage system being a second network attached storage system, wherein a second file system is created on said second volume, said second file system being a copy of said first file system, such that said second file system mirrors said first file system prior to said de-duplication operation on the second volume.

16. A method of reducing an amount of storage capacity used during mirroring operations, comprising:

storing data to a first volume and mirroring the data to a second volume, said second volume being a virtual volume having a plurality of logical addresses, wherein segments of physical storage capacity are allocated for a specified logical address as required when the data is to be stored to said specified logical address; and

carrying out a de-duplication operation on the second volume following a split from the first volume by identifying a particular segment of said second volume having data that is the same as another segment;

creating a link from the particular segment to the other segment; and

releasing the particular segment from the second volume, whereby physical storage capacity required for the second volume is reduced;

resynchronizing the data stored on said second volume with the data stored on said first volume prior to splitting said second volume from mirroring said first volume; and

reverse resynchronizing the data stored on said first volume with data stored on said second volume, wherein before reverse resynchronizing whether the second volume has been de-duplicated or not is checked and if the second volume has been de-duplicated, said reverse resynchronizing is conducted on to the first volume from the de-duplicated volume.

17. A method according to claim 16, further including steps of selecting segments of the second volume during the de-duplication operation on the second volume;

calculating a hash value for the particular segment representative of the data contained in the particular segment;

comparing the calculated hash value with previously-stored hash values for other segments in the second volume;

linking the particular segment to the another segment having a previously-stored hash value that matches the calculated hash value of the particular segment; and

releasing the particular segment from the second volume when the other segment has a previously-stored hash value that matches the calculated hash value of the particular segment.

18. A method according to claim 16, further including steps of

providing a first storage system in communication with a host computer, said first storage system including a first controller and multiple first disk devices, said first volume representing physical storage area on said disk devices;

and

providing a second storage system in communication with said first storage system and in a location separate from said first storage system, said second storage system including a second controller and multiple second disk devices; and

providing said second volume on said second storage system as a remote mirror of said first volume.

19. A method of operating an information system, comprising:

storing data to a first volume and mirroring the data to a second volume, said second volume being a virtual volume having a plurality of logical storage addresses, wherein a segment of physical storage capacity is allocated for a specified logical address as required when the data is to be stored to said specified logical address;

selecting segments of the second volume during a de-duplication operation on the second volume;

calculating a hash value for a particular segment representative of data contained in the particular segment;

comparing the calculated hash value with previously-stored hash values for other segments in the second volume;

linking the particular segment to another segment having a previously-stored hash value that matches the calculated hash value of the particular segment;

releasing the particular segment from the second volume when another segment has a previously-stored hash value that matches the calculated hash value of the particular segment, whereby physical storage capacity required for the second volume is reduced;

providing a first storage system in communication with a host computer, said first storage system including a first controller and multiple first disk devices, said first volume representing physical storage capacity on said first disk devices;

providing a second storage system in communication with said first storage system and in a location separate from said first storage system, said second storage system including a second controller and multiple second disk devices;

providing said second volume on said second storage system as a remote mirror of said first volume; and

checking whether the second volume has been designated for de-duplication operations before said selecting segments of the second volume,

wherein said de-duplication operation on the second volume is performed after pair configuration of the second volume to the first volume is suspended and determined that the second volume has been designated for de-duplication operations based on an information maintained in a memory of said information system.

20. A method according to claim 19, further comprising:

providing a plurality of said first volumes and a plurality of said second volumes for forming a consistency group in which each first volume is mirrored to one of said second volumes;

wherein said step of comparing the calculated hash value with previously-stored hash values for other segments in the virtual volume includes comparing the calculated hash value with previously-stored hash values for segments of all said second volumes in the consistency group; and

wherein second volumes in the same consistency group have the same status on whether the second volume has been designated for de-duplication operations, and said status indicated by a flag in said memory.