Power efficient storage with data de-duplication

• US 20090019246A1
• Filed: 07/10/2007
• Published: 01/15/2009
• Est. Priority Date: 07/10/2007
• Status: Active Grant

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Murase, Atsushi
• Granted Patent: US 7,669,023 B2
• Time in Patent Office: Days
• US Class Current: 711/162
• CPC Class Codes: G06F 1/3203 Power management, i.e. event-based initiation of a power-saving mode
• CPC Class Codes: G06F 1/3268 Power saving in hard disk drive
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0625 Power saving in storage systems
• CPC Class Codes: G06F 3/0634 by changing the state or mode of one or more devices
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

Power consumption in a storage system is reduced by selectively controlling power supplied to the storage devices, while also incorporating a de-duplication function to reduce the amount of required storage capacity. First storage devices are initially in a powered on condition and second storage devices are in a powered off condition. Write data received by the controller is initially stored to a first volume allocated from the first storage devices. While the second storage devices are powered on, content of the write data stored in the first volume is compared with content of any existing data stored in a second volume allocated on the second storage devices. When results of the comparison show that the content of the write data does not match the content of the existing data, the write data is stored to the second volume.

Claims

1. A storage system comprising:

a controller in communication with one or more first storage mediums and one or more second storage mediums;

a first volume having storage capacity allocated from said one or more first storage mediums; and

a second volume having storage capacity allocated from said one or more second storage mediums,

wherein initially said one or more first storage mediums are configured in a powered on condition and said one or more second storage mediums are configured in a powered off condition,

wherein write data received by said controller is stored to said first volume,

wherein, while said one or more second storage mediums are configured in a powered on condition, a flush process is carried out whereby content of said write data stored in said first volume is compared with content of any existing data stored in said second volume, and

wherein when results of the comparison show that the content of said write data does not match the content of said existing data, said write data is stored to said second volume.

2. A storage system according to claim 1,

wherein when the results of the comparison show that the content of the write data does match the content of said existing data, a record is stored linking a logical block address of the write data with the existing data, and the write data is discarded.

3. A storage system according to claim 2,

wherein, following storing of said write data to said second volume when the content does not match or storing of the record when the content does match, said one or more second storage mediums are powered off and the write data deleted from said first volume.

4. A storage system according to claim 1, further comprising:

a virtual volume presented by said controller for receiving said write data as a target from a host computer, said write data targeting said virtual volume being initially stored to said first volume.

5. A storage system according to claim 1,

wherein a first hash value is calculated for the content of said write data, and during said comparison, said first hash value is compared with second hash values calculated for the content of any existing data stored on said second volume to determine whether a direct comparison of the content of the write data with the existing data should be conducted.

6. A storage system according to claim 1,

wherein data stored to said first volume is stored as chunks of a predetermined size,

wherein data stored to said second volume is stored as chunks of said predetermined size,

wherein said write data received by said controller is divided into divided portions, and content of each divided portion is compared with any existing chunks already stored on said first volume,

wherein, for each divided portion, when results of said comparison of said divided portion show that the content of said divided portion does not match the content of said existing chunks on said first volume, said divided portion is stored to said first volume as a new chunk, and

wherein when the results of the comparison of said divided portion show that the content of said divided portion does match the content of one of said existing chunks on said first volume, a record is stored linking a logical block address of the divided portion with the existing chunk on said first volume, and the divided portion is discarded.

7. A storage system according to claim 6,

wherein a first hash value is calculated for the content of each said divided portion, and during said comparison of said divided portion, said first hash value is compared with second hash values calculated for the content of any existing data stored on said first volume to determine whether a direct comparison of the content of the write data with the existing data should be conducted.

8. A storage system according to claim 1,

wherein said one or more second storage mediums are powered on in response to a read request received by said storage controller, and said flush process is initiated and said comparison takes place following completion of the read request.

9. A storage system according to claim 1,

wherein said flush process is initiated by a triggering event including one of an instruction from an administrator, the first volume reaching a predetermined capacity, or passage of a predetermined period of time, and

wherein said one or more second storage mediums are powered on in response to initiation of said flush process so that said comparison can be carried out.

10. A storage system according to claim 1,

wherein there are a plurality of said first storage mediums, said first storage mediums being configured in a first array group, said first volume having storage capacity allocated from said first array group,

wherein there are a plurality of said second storage mediums, said second storage mediums being configured in a second array group, said second volume having storage capacity allocated from said second array group, and

wherein said second array group is configured to be powered on and off independently of said first array group by said controller.

11. A storage system according to claim 1, further comprising:

one or more third storage mediums in communication with said controller; and

a third volume having storage capacity allocated from said third array group,

wherein when said controller determines that a first hash value calculated for the write data is the same as a second hash value calculated for existing data stored on one of said second volume or said third volume, the controller powers on the storage mediums for the one of said second or third volume in which the existing data having the matching hash value is stored to compare the content of the existing data with the content of the write data, while leaving the storage mediums for the other of the second or third volume powered off.

12. A storage system comprising:

a controller for controlling input/output (I/O) operations to a plurality of first storage mediums arranged in a first array group and to a plurality of second storage mediums arranged in a second array group;

a buffer volume having storage capacity allocated from said first array group, wherein data stored to said buffer volume is stored as chunks of a predetermined size;

a base volume having storage capacity allocated from said second array group, wherein data stored to said base volume is stored as chunks of said predetermined size; and

a virtual volume presented by said controller as a target of the I/O operations,

wherein said first storage mediums are initially in a powered on condition and said second storage mediums are in a powered off condition,

wherein write data targeting said virtual volume received by said controller is initially stored to the buffer volume by dividing said write data into divided portions of said predetermined size for comparing content of each divided portion with content of any existing chunks already stored on said buffer volume, and storing the divided portion as a new chunk when the content of the divided portion does not match the content of any existing chunks,

wherein, while said one or more second storage mediums are powered on, content of the chunks stored in said buffer volume is compared with content of any chunks stored in said base volume, and

wherein when results of said comparison for each chunk of said buffer volume show that the content of the chunk of said buffer volume does not match the content of any of said chunks on said base volume, the chunk from the buffer volume is stored to said base volume.

13. A storage system according to claim 12,

wherein when the results of the comparison of the chunks show that the content of the chunk of the buffer volume data does match the content of any chunks on the base volume, a record is stored linking a logical block address of the chunk on the buffer volume with a matching existing chunk on the base volume, and the chunk on the buffer volume is discarded.

14. A storage system according to claim 13,

wherein following the comparison of the chunks in said buffer volume with the content of any existing chunks stored in said base volume, and storing of said chunks to said base volume in the case that the content does not match or storing of the record in the case that the content does match, said one or more second storage mediums are powered off and the chunks deleted from said buffer volume.

15. A storage system according to claim 12,

wherein first hash values are calculated for the content of said chunks on said buffer volume, and during said comparison of chunks, said first hash values are compared with second hash values previously calculated for the content of any existing chunks stored on said base volume to determine whether a direct comparison of the content of the chunks on the buffer volume with the chunks on the base volume should be conducted.

16. A storage system according to claim 12, wherein said base volume is a first base volume of a plurality of base volumes, said storage system further comprising:

a plurality of third storage mediums arranged in a third array group, said controller controlling I/O operations to said third storage mediums; and

a second base volume of said plurality of base volumes having storage capacity allocated from said third array group, wherein data stored to said second base volume is stored as chunks of said predetermined size,

wherein when said controller determines that a first hash value calculated for one of the chunks on said buffer volume is the same as a second hash value previously calculated for an existing chunk stored on one of said first base volume or said second base volume, the controller powers on the storage mediums for the one of said first base volume or said second base volume in which the existing chunk having the matching hash value is stored to compare the content of the existing chunk with the content of the chunk on the base volume, while leaving the storage mediums for the other of the first base volume or the second base volume powered off.

17. A storage system comprising:

a controller in communication with one or more first storage mediums, one or more second storage mediums, and one or more third storage mediums;

a first volume having storage capacity allocated from said one or more first storage mediums;

a second volume having storage capacity allocated from said one or more second storage mediums;

a third volume having storage capacity allocated from said one or more third storage mediums; and

wherein said one or more first storage mediums are initially in a powered on condition and said one or more second storage mediums and said one or more third storage mediums are in a powered off condition,

wherein write data received by said controller is initially stored to said first volume,

wherein when said controller determines that a first hash value calculated for the write data is the same as a second hash value calculated for existing data stored on one of said second volume or said third volume, the controller is configured to power on the storage mediums for the one of said second or third volume in which the existing data having the matching hash value is stored to compare the content of the existing data with the content of the write data, while leaving the storage mediums for the other of the second or third volume powered off.

18. A storage system according to claim 17,

wherein when results of the comparison show that the content of said write data does not match the content of said existing data, said write data is stored to the one of said second volume or third volume having a largest remaining available capacity.

19. A storage system according to claim 17,

wherein when the results of the comparison show that the content of the write data does match the content of said existing data, a record is stored linking a logical block address of the write data with the existing data, and the write data is discarded.

20. A storage system according to claim 17,

wherein data stored to said first volume is stored as chunks of a predetermined size,

wherein data stored to said second volume and said third volume is stored as chunks of said predetermined size,

wherein said write data received by said controller is divided into divided portions, and content of each divided portion is compared with any existing chunks already stored on said first volume,

wherein, for each divided portion, when results of said comparison of said divided portion show that the content of said divided portion does not match the content of said existing chunks stored on said first volume, said divided portion is stored to said first volume as a new chunk, and

wherein when the results of the comparison of said divided portion show that the content of said divided portion does match the content of one of said existing chunks on said first volume, a record is stored linking a logical block address of the divided portion with the existing chunk on said first volume, and the divided portion is discarded.