Data de-duplication using thin provisioning

• US 7,822,939 B1
• Filed: 09/25/2007
• Issued: 10/26/2010
• Est. Priority Date: 09/25/2007
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Veprinsky, Alexandr, Michael, Ofer E., Scharland, Michael J.
• Primary Examiner(s): Gu; Shawn X
• Application Number:
  US11/903,869
• Time in Patent Office: 1,127 Days
• Field of Search: None
• US Class Current: 711/170
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD

Abstract

A system for de-duplicating data includes providing a first volume including at least one pointer to a second volume that corresponds to physical storage space, wherein the first volume is a logical volume. A first set of data is detected as a duplicate of a second set of data stored on the second volume at a first data chunk. A pointer of the first volume associated with the first set of data is modified to point to the first data chunk. After modifying the pointer, no additional physical storage space is allocated for the first set of data.

Claims

1. A method of de-duplicating data, comprising:

providing a first volume including at least one pointer to a second volume that corresponds to physical storage space, wherein the first volume is a logical volume;

detecting a first set of data as a duplicate of a second set of data stored on the second volume at a first data chunk; and

modifying a pointer of the first volume associated with the first set of data to point to the first data chunk, wherein, after modifying the pointer, no additional physical storage space is allocated for the first set of data, wherein the first volume includes a first chunk and a second chunk, and wherein the first chunk of the first volume includes the pointer to the first data chunk of the second volume, and wherein the second chunk of the first volume includes a pointer that points to the first chunk of the first volume.

2. The method according to claim 1, further comprising:

updating a counter associated with the first data chunk that identifies a number of pointers pointing to the first data chunk.

3. The method according to claim 1, wherein the second volume is at least one of:

a physical storage area that contains the physical storage space and a logical volume that maps to allocations of the physical storage space.

4. The method according to claim 1, wherein the first set of data previously exists on the second volume, and wherein the method further includes:

de-allocating physical storage space associated with the first set of data that previously existed on the second volume.

5. The method according to claim 1, wherein the first set of data is associated with a new write request, and wherein the method further includes:

intercepting the new write request after detecting that the first set of data of the new write request is a duplicate of the second set of data stored on the second volume.

6. The method according to claim 1, wherein the first volume includes a thin device.

7. The method according to claim 1, wherein the first chunk of the first volume includes a first thin chunk and the second chunk of the first volume includes a second thin chunk.

8. The method according to claim 1, further comprising:

intercepting a write request to change data associated with a particular one of the first set of data and the second set of data;

copying the particular one of the first set of data and the second set of data to a different data chunk;

modifying a pointer of a volume associated with the particular one of the first set of data and the second set of data to point to the different data chunk;

decrementing a counter associated with a first data chunk of the particular one of the first set of data and the second set of data; and

performing the write request to change the particular one of the first set of data and the second set of data at the different data chunk.

9. A computer program product, stored on a computer-readable medium, for de-duplicating data, comprising:

executable code that provides a first volume including at least one pointer to a second volume that corresponds to physical storage space, wherein the first volume is a logical volume;

executable code that detects a first set of data as a duplicate of a second set of data stored on the second volume at a first data chunk; and

executable code that modifies a pointer of the first volume associated with the first set of data to point to the first data chunk, wherein, after modifying the pointer, no additional physical storage space is allocated for the first set of data, wherein the first volume includes a first chunk and a second chunk, and wherein the first chunk of the first volume includes the pointer to the first data chunk of the second volume, and wherein the second chunk of the first volume includes a pointer that points to the first chunk of the first volume.

10. The computer program product according to claim 9, further comprising:

executable code that updates a counter associated with the first data chunk that identifies a number of pointers pointing to the first data chunk.

11. The computer program product according to claim 9, wherein the second volume is at least one of:

a physical storage area that contains the physical storage space and a logical volume that maps to allocations of the physical storage space.

12. The computer program product according to claim 9, wherein the first set of data previously exists on the second volume, and wherein the computer program product further includes:

executable code that de-allocates physical storage space associated with the first set of data that previously existed on the second volume.

13. The computer program product according to claim 9, wherein the first set of data is associated with a new write request, and wherein the computer program product further includes:

executable code that intercepts the new write request after detecting that the first set of data of the new write request is a duplicate of the second set of data stored on the second volume.

14. A computer storage device, comprising:

a plurality of interconnected directors, wherein at least some of the directors handle reading and writing data for the computer storage device;

a plurality of disk drives, coupled to at least some of the directors, that store data for the computer storage device; and

computer software, provided on a computer readable medium of at least one of the directors, having executable code that provides a first volume including at least one pointer to a second logical volume that maps allocations of physical storage space, wherein the first volume is a logical volume, executable code that detects a first set of data as a duplicate of a second set of data stored on the second volume at a first data chunk, and executable code that modifies a pointer of the first volume associated with the first set of data to point to the first data chunk, wherein, after modifying the pointer, no additional physical storage space is allocated for the first set of data, wherein the first volume includes a first chunk and a second chunk, and wherein the first chunk of the first volume includes the pointer to the first data chunk of the second logical volume, and wherein the second chunk of the first volume includes a pointer that points to the first chunk of the first volume.

15. The computer storage device according to claim 14, wherein the computer software further includes:

executable code that updates a counter associated with the first data chunk that identifies a number of pointers pointing to the first data chunk.

16. The computer storage device according to claim 14, wherein the second volume is at least one of:

a physical storage area that contains the physical storage space and a logical volume that maps to allocations of the physical storage space.

17. The computer storage device according to claim 14, wherein the first set of data previously exists on the second volume, and wherein the computer software further includes:

executable code that de-allocates physical storage space associated with the first set of data that previously existed on the second volume.

18. The computer storage device according to claim 14, wherein the first set of data is associated with a new write request, and wherein the computer software further includes:

executable code that intercepts the new write request after detecting that the first set of data of the new write request is a duplicate of the second set of data stored on the second volume.

19. A computer storage system, comprising:

a remote storage device; and

a local storage device coupled to the remote storage device via a data link, the local storage device having at least one processor that controls a first logical volume of the local storage device, the first logical volume including at least one pointer to a second volume of the local storage device that corresponds to physical storage space, wherein in response to the local storage device detecting duplicate data prior to transmission of data to the remote storage device, the local storage device transmits information indicative of the duplicate data instead of the data, wherein the first logical volume includes a first chunk and a second chunk, and wherein the first chunk of the first logical volume includes the at least one pointer to the second volume, and wherein the second chunk of the first logical volume includes a pointer that points to the first chunk of the first logical volume.

20. The computer storage system according to claim 19, wherein the second volume is at least one of:

a physical storage area that contains the physical storage space and a logical volume that maps to allocations of the physical storage space.

21. The computer storage system according to claim 19, wherein a first set of data is associated with a new write request, and wherein the at least one processor intercepts the new write request after detecting that the first set of data of the new write request is a duplicate of a second set of data stored on the second volume.

22. The computer storage system according to claim 19, wherein the remote storage device updates a counter associated with data that identifies a number of pointers pointing to a data chunk of the second volume.

23. The computer storage system according to claim 19, wherein, following transmitting the information indicative of duplicate data, the local storage device provides an acknowledgement.

24. A method of cloning data, comprising:

providing a first volume including at least one pointer to a second volume that corresponds to physical storage space, wherein the at least one pointer of the first volume points to a data chunk stored on the second volume, and wherein the first volume is a logical volume, wherein the first volume includes a first chunk and a second chunk, and wherein the first chunk of the first volume includes the at least one pointer to the data chunk stored on the second volume, and wherein the second chunk of the first volume includes a pointer that points to the first chunk of the first volume;

providing a third volume including at least one pointer, wherein the third volume is another logical volume; and

modifying the at least one pointer of the third volume to point to the data chunk without allocating any additional physical storage space.

25. The method according to claim 24, further comprising:

updating a counter associated with the data chunk to indicate that more than one logical volume points to the data chunk.