Streamlining data deduplication

• US 9,934,236 B2
• Filed: 02/23/2015
• Issued: 04/03/2018
• Est. Priority Date: 02/23/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Barajas Gonzalez, Emmanuel, Harrington, Shaun E., McGregor, Harry, Moore, Christopher B.
• Primary Examiner(s): Burke, Jeff A
• Application Number:
  US14/629,264
• Publication Number:
  US 20160246812A1
• Time in Patent Office: 1,135 Days
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 3/0641 De-duplication techniques

Abstract

Various embodiments for streamlining data deduplication by a processor. In file storage utilizing re-writable storage media with data deduplication, a host layer file system is informed of available file space based on at least one of an available plurality of tracking operations performed by the processor, thereby providing an adjustable virtual free capacity file of deduplication pointers to facilitate deduplication functionality across a plurality of storage devices. As one of the available plurality of tracking devices, an actual block utilization tracking operation is utilized for gathering periodic statistics from deduplicating sources to enable analysis of trends in storage capacity utilization.

Claims

1. A method for streamlining data deduplication by a processor, comprising:

in file storage utilizing re-writable storage media with data deduplication, informing a host layer file system of available file space based on at least one of an available plurality of tracking operations performed by the processor, thereby providing an adjustable virtual free capacity file of deduplication pointers to facilitate deduplication functionality across a plurality of storage devices;

wherein as one of the available plurality of tracking devices, an actual block utilization tracking operation is utilized for gathering periodic statistics from deduplicating sources to enable analysis of trends in storage capacity utilization.

2. The method of claim 1, wherein the adjustable virtual free capacity file of deduplication pointers facilitates deduplication functionality across the plurality of storage devices notwithstanding whether the plurality of storage devices are configured to perform deduplication operations.

3. The method of claim 1, further including, as one of the available plurality of tracking operations, utilizing an un-deduplicated utilization tracking operation for collecting and analyzing historical trends of file system level utilization to assist in predicting future storage use.

4. The method of claim 1, further including, as one of the available plurality of tracking operations, utilizing a virtual capacity tracking and modification operation for increasing a virtual capacity reported by a storage device as determined by a utilization of a virtual and physical capacity of the storage device.

5. The method of claim 1, further including, as one of the available plurality of tracking operations, utilizing a file system level modification and reporting operation having a block of fully deduplicable data that can be logically increased or decreased to provide free space elasticity.

6. The method of claim 1, further including, as one of the available plurality of tracking operations, utilizing a deduplication rate-over-time operation for retaining calculated ongoing deduplication rates to be used to facilitate a manipulation of available capacity at a file system layer based on at least one of actual and predicted future utilization.

7. A system for streamlining data deduplication, comprising:

a processor that, in file storage, utilizes re-writable storage media with data deduplication, informing a host layer file system of available file space based on at least one of an available plurality of tracking operations performed by the processor, thereby providing an adjustable virtual free capacity file of deduplication pointers to facilitate deduplication functionality across a plurality of storage devices;

wherein as one of the available plurality of tracking devices, an actual block utilization tracking operation is utilized for gathering periodic statistics from deduplicating sources to enable analysis of trends in storage capacity utilization.

8. The system of claim 7, wherein the adjustable virtual free capacity file of deduplication pointers facilitates deduplication functionality across the plurality of storage devices notwithstanding whether the plurality of storage devices are configured to perform deduplication operations.

9. The system of claim 8, wherein the processor, pursuant to one of the available plurality of tracking operations, utilizes an un-deduplicated utilization tracking operation for collecting and analyzing historical trends of file system level utilization to assist in predicting future storage use.

10. The system of claim 8, wherein the processor, pursuant to one of the available plurality of tracking operations, utilizes a virtual capacity tracking and modification operation for increasing a virtual capacity reported by a storage device as determined by a utilization of a virtual and physical capacity of the storage device.

11. The system of claim 8, wherein the processor, pursuant to one of the available plurality of tracking operations, utilizes a file system level modification and reporting operation having a block of fully deduplicable data that can be logically increased or decreased to provide free space elasticity.

12. The system of claim 8, wherein the processor, pursuant to one of the available plurality of tracking operations, utilizes a deduplication rate-over-time operation for retaining calculated ongoing deduplication rates to be used to facilitate a manipulation of available capacity at a file system layer based on at least one of actual and predicted future utilization.

13. A computer program product for streamlining data deduplication by a processor, the computer program product comprising a non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:

a first executable portion that, in file storage, utilizes re-writable storage media with data deduplication, informing a host layer file system of available file space based on at least one of an available plurality of tracking operations performed by the processor, thereby providing an adjustable virtual free capacity file of deduplication pointers to facilitate deduplication functionality across a plurality of storage devices;

wherein as one of the available plurality of tracking devices, an actual block utilization tracking operation is utilized for gathering periodic statistics from deduplicating sources to enable analysis of trends in storage capacity utilization.

14. The computer program product of claim 13, wherein the adjustable virtual free capacity file of deduplication pointers facilitates deduplication functionality across the plurality of storage devices notwithstanding whether the plurality of storage devices are configured to perform deduplication operations.

15. The computer program product of claim 13, further including a second executable portion that, pursuant to one of the available plurality of tracking operations, utilizes an actual block utilization tracking operation for gathering periodic statistics from deduplicating sources to enable analysis of trends in storage capacity utilization.

16. The computer program product of claim 13, further including a second executable portion that, pursuant to one of the available plurality of tracking operations, utilizes an un-deduplicated utilization tracking operation for collecting and analyzing historical trends of file system level utilization to assist in predicting future storage use.

17. The computer program product of claim 13, further including a second executable portion that, pursuant to one of the available plurality of tracking operations, utilizes a virtual capacity tracking and modification operation for increasing a virtual capacity reported by a storage device as determined by a utilization of a virtual and physical capacity of the storage device.

18. The computer program product of claim 13, further including a second executable portion that, pursuant to one of the available plurality of tracking operations, utilizes a file system level modification and reporting operation having a block of fully deduplicable data that can be logically increased or decreased to provide free space elasticity.

19. The computer program product of claim 13, further including a second executable portion that, pursuant to one of the available plurality of tracking operations, utilizes a deduplication rate-over-time operation for retaining calculated ongoing deduplication rates to be used to facilitate a manipulation of available capacity at a file system layer based on at least one of actual and predicted future utilization.