COMPUTER PROGRAM, APPARATUS, AND METHOD FOR MANAGING DATA

• US 20110138144A1
• Filed: 12/03/2010
• Published: 06/09/2011
• Est. Priority Date: 12/04/2009
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: TAMURA, Masahisa, Oe, Kazuichi, Watanabe, Takashi, Ogihara, Kazutaka, Noguchi, Yasuo, Tsuchiya, Yoshihiro, Maruyama, Tetsutaro, Kumano, Tatsuo
• Granted Patent: US 8,458,425 B2
• Time in Patent Office: Days
• US Class Current: 711/166
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A computer in a disk node executes a data management program. A deduplication-eligible data unit detection module detects a data unit whose deduplication grace period after last write time has expired. A deduplication address fetch module interacts with an index server to obtain a deduplication address associated with a unique value of data stored in a constituent storage space allocated to the data unit that is found to be deduplication-eligible. A constituent storage space deallocation module stores the obtained deduplication address in a data unit record memory, together with information indicating the detected data unit. Simultaneously a constituent storage space deallocation module releases the allocated constituent storage space from the detected data unit.

Claims

1. A computer-readable, non-transitory medium storing a data management program for use in a multi-node storage system formed from a plurality of disk nodes each managing a storage device to store data in a distributed manner, the data management program, when executed by a computer serving as one of the disk nodes, causing the computer to perform a procedure comprising:

allocating one of constituent storage spaces in a storage device coupled to the computer, to one of data units constituting a logical volume that provides a virtual storage space, in response to a write request specifying the one of data units as a destination of write data, and writing the write data to the allocated constituent storage space;

recording, upon the writing of the write data, a current time in a data unit record memory as a record of last write time of the data unit to which the write data has been written;

detecting, by consulting the data unit information memory, a data unit whose deduplication grace period after the last write time has expired;

obtaining, from an index server, one of deduplication addresses that is associated with a first unique value obtained by applying a predetermined computation to data stored in the constituent storage space allocated to the detected data unit, wherein the index server manages the deduplication addresses each including an identifier of a disk node managing a deduplicate unit and a second unique value obtained by applying the predetermined computation to deduplication target data stored in the deduplicate unit, and wherein the deduplicate unit is provided in a plurality to constitute a deduplicate volume that provides another virtual storage space; and

storing the obtained deduplication address in the data unit record memory, together with the detected data unit, while canceling the allocation of the constituent storage spaces to the detected data unit.

2. The computer-readable, non-transitory medium according to claim 1, wherein the procedure further comprises:

allocating, in response to a deduplicate unit allocation request received from the index server which specifies an identifier of one of the deduplicate units, one of the constituent storage spaces to the specified deduplicate unit, obtaining the deduplication target data in the specified deduplicate unit by making access to the disk node having a link to the specified deduplicate unit, and storing the obtained deduplication target data in the allocated constituent storage space; and

sending, in response to a data read request specifying an identifier of the deduplicate unit, the deduplication target data read out of the constituent storage space allocated to the specified deduplicate unit.

3. The computer-readable, non-transitory medium according to claim 2, wherein the procedure further comprises:

deallocating the constituent storage space from the deduplicate unit when there is no disk node that has a link to the deduplicate unit to read the deduplication target data therefrom.

4. The computer-readable, non-transitory medium according to claim 3, wherein the procedure further comprises:

storing a retention time limit of a deduplicate unit in a deduplicate unit record memory, the retention time limit being calculated by adding a predetermined retention period to a last read time at which a latest read operation has been performed on the constituent storage space allocated to the deduplicate unit; and

searching the deduplicate unit record memory to detect a deduplicate unit whose retention time limit has been reached, and deallocating the constituent storage space from the detected deduplicate unit.

5. The computer-readable, non-transitory medium according to claim 2, wherein the procedure further comprises:

deallocating the constituent storage space allocated to the data unit after transferring data from the constituent storage space to the disk node specified by the deduplication address, when the deduplication address obtained from the index server is of a newly allocated deduplicate unit.

6. The computer-readable, non-transitory medium according to claim 1, wherein the procedure further comprises:

patrolling the data unit record memory at regular intervals to find a data unit associated with a deduplication address, and obtaining data from the disk node specified by the deduplication address by specifying an identifier of the deduplicate unit which is indicated in the deduplication address.

7. The computer-readable, non-transitory medium according to claim 1, wherein the second unique value is a hash value calculated by applying a predetermined hash function to the deduplication target data.

8. The computer-readable, non-transitory medium according to claim 1, wherein the identifier of the deduplicate unit is an offset indicating a location of the deduplicate unit in the deduplicate volume.

9. The computer-readable, non-transitory medium according to claim 1, wherein the procedure further comprises:

searching, in response to a read request specifying a data unit that has not been allocated a constituent storage space, the data unit record memory to find a deduplication address of the specified data unit, and obtaining deduplication target data from the disk node indicated in the found deduplication address by specifying an identifier of the deduplicate unit indicated in the found deduplication address.

10. The computer-readable, non-transitory medium according to claim 1, wherein the procedure further comprises:

reading, in response to a read request specifying a data unit, data out of the constituent storage space allocated to the specified data unit.

11. A computer-readable, non-transitory medium storing a data management program for managing storage spaces in a multi-node storage system formed from a plurality of disk nodes each managing a storage device to store data in a distributed manner, the data management program, when executed by a computer, causing the computer to perform a procedure comprising:

storing, in a deduplicate unit record memory, deduplicate unit records each including information indicating use of a deduplicate unit in a disk node, a first unique value obtained by applying a predetermined computation to deduplication target data stored in a constituent storage space allocated to the deduplicate unit being used, and an identifier of the deduplicate unit, wherein the deduplicate unit records are stored together with an identifier of a disk node that manages the deduplicate units, and wherein the deduplicate units constitute a deduplicate volume that provides another virtual storage space;

receiving from one of the disk nodes a deduplication address request specifying a second unique value obtained by applying the predetermined computation to data in a constituent storage space allocated to a deduplication-eligible data unit, and searching the deduplicate unit record memory to find a deduplicate unit record that contains the second unique value specified in the deduplication address request;

returning a first deduplication address to the disk node that has issued the deduplication address request when a relevant deduplicate unit record is found as a result of the searching, wherein the first deduplication address includes an identifier of a deduplicate unit which is contained in the found deduplicate unit record, and an identifier of a disk node that manages the deduplicate unit corresponding to the found deduplicate unit record; and

consulting the deduplicate unit record memory to select one of the deduplicate units that is not used when no relevant record is found as a result of the searching, sending an allocation request to a disk node that manages the selected deduplicate unit for allocation of a constituent storage space to the selected deduplicate unit, storing an updated deduplicate unit record of the selected deduplicate unit in the deduplicate unit record memory to record the constituent storage space allocated to the selected deduplicate unit, and returning a second deduplication address to the disk node that has issued the deduplication address request, wherein the second deduplication address includes an identifier of the selected deduplicate unit and an identifier of the disk node managing the selected deduplicate unit.

12. The computer-readable, non-transitory medium according to claim 11, wherein the procedure further comprises:

collecting the deduplicate unit records from each of the plurality of disk nodes, and storing the collected deduplicate unit records in the deduplicate unit record memory, together with identifiers of the disk nodes that have provided the deduplicate unit records.

13. The computer-readable, non-transitory medium according to claim 11, wherein:

the deduplicate unit record of a deduplicate unit includes a retention time limit calculated by adding a predetermined retention period to a last read time at which a latest read operation has been performed on the constituent storage space allocated to the deduplicate unit; and

the procedure further comprises;

searching the deduplicate unit record memory to detect a deduplicate unit record whose retention time limit has been reached, and

obtaining a latest version of the detected deduplicate unit record from the disk node that has originally provided the detected deduplicate unit record.

14. An apparatus for managing data in a multi-node storage system formed from a plurality of disk nodes each managing a storage device to store data in a distributed manner, the apparatus comprising:

write access means that allocates one of constituent storage spaces in a storage device coupled to the computer, to one of data units constituting a logical volume that provides a virtual storage space, in response to a write request specifying the one of data units as a destination of write data, and writes the write data to the allocated constituent storage space;

last write time update means that records a current time in the memory as a record of last write time of the data unit to which the write data has been written;

deduplication-pending data unit detection means that detects, by consulting the memory, a data unit whose deduplication grace period after the last write time has expired;

deduplication address fetch means that obtains, from an index server, one of deduplication addresses that is associated with a first unique value obtained by applying a predetermined computation to data stored in the constituent storage space allocated to the detected data unit, wherein the index server manages the deduplication addresses each including an identifier of a disk node managing a deduplicate unit and a second unique value obtained by applying the predetermined computation to deduplication target data stored in the deduplicate unit, and wherein the deduplicate unit is provided in a plurality to constitute a deduplicate volume that provides another virtual storage space;

constituent storage space deallocation means that stores the obtained deduplication address in the data unit record memory, together with the detected data unit, while canceling the allocation of the constituent storage spaces to the detected data unit.

15. An apparatus for managing data in a multi-node storage system formed from a plurality of disk nodes each managing a storage device to store data in a distributed manner, the apparatus comprising:

a processor configured to execute a procedure, the procedure comprising;

allocating one of constituent storage spaces in a storage device coupled to the computer, to one of data units constituting a logical volume that provides a virtual storage space, in response to a write request specifying the one of data units as a destination of write data, and writing the write data to the allocated constituent storage space;

recording, upon the writing of the write data, a current time in a data unit record memory as a record of last write time of the data unit to which the write data has been written;

detecting, by consulting the data unit information memory, a data unit whose deduplication grace period after the last write time has expired;

obtaining, from an index server, one of deduplication addresses that is associated with a first unique value obtained by applying a predetermined computation to data stored in the constituent storage space allocated to the detected data unit, wherein the index server manages the deduplication addresses each including an identifier of a disk node managing a deduplicate unit and a second unique value obtained by applying the predetermined computation to deduplication target data stored in the deduplicate unit, and wherein the deduplicate unit is provided in a plurality to constitute a deduplicate volume that provides another virtual storage space.

16. A method executed by a computer in one of a plurality of disk nodes constituting a multi-node storage system, each disk node managing a storage device to store data in a distributed manner, the method comprising:

allocating one of constituent storage spaces in a storage device coupled to the computer, to one of data units constituting a logical volume that provides a virtual storage space, in response to a write request specifying the one of data units as a destination of write data, and writing the write data to the allocated constituent storage space;

recording, upon the writing of the write data, a current time in a data unit record memory as a record of last write time of the data unit to which the write data has been written;

detecting, by consulting the data unit information memory, a data unit whose deduplication grace period after the last write time has expired;

obtaining, from an index server, one of deduplication addresses that is associated with a first unique value obtained by applying a predetermined computation to data stored in the constituent storage space allocated to the detected data unit, wherein the index server manages the deduplication addresses each including an identifier of a disk node managing a deduplicate unit and a second unique value obtained by applying the predetermined computation to deduplication target data stored in the deduplicate unit, and wherein the deduplicate unit is provided in a plurality to constitute a deduplicate volume that provides another virtual storage space;

storing the obtained deduplication address in the data unit record memory, together with the detected data unit, while canceling the allocation of the constituent storage spaces to the detected data unit.