Method and system for data compression and compression estimation in a virtual tape library environment

• US 7,437,492 B2
• Filed: 05/14/2003
• Issued: 10/14/2008
• Est. Priority Date: 05/14/2003
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Johnston, Craig Anthony, Trimmer, Don Alvin, Blaser, Rico, Stager, Roger
• Primary Examiner(s): Chen; Alan
• Application Number:
  US10/436,612
• Publication Number:
  US 20040230724A1
• Time in Patent Office: 1,980 Days
• Field of Search: 711111-113, 710/5, 710/6, 710 15- 17, 710/19, 710/56, 710/57, 710/68, 710/74
• US Class Current: 710/68
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0638 Organizing or formatting or addressing of data
• CPC Class Codes: G06F 3/0664 at device level, e.g. emulation of a storage device or system
• CPC Class Codes: G06F 3/0686 Libraries, e.g. tape libraries, jukebox
• CPC Class Codes: G11B 20/00007 Time or data compression or expansion audio compression based on psychoacoustics G10L19/00; data processing for reproducing audio data at different playback speeds G10L21/04; video compression H04N19/00; data compression per se H03M7/30
• CPC Class Codes: H03M 7/30 Compression speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N; Expansion; Suppression of unnecessary data, e.g. redundancy reduction

Abstract

A method and system for efficiently storing and transferring data in a virtual tape library environment is disclosed. Data is written to a virtual tape library that emulates a physical tape library. Data stored in the virtual tape library may be compressed and an estimated compression ratio may be dynamically computed. While data is written to the virtual tape library, an end-of-tape signal is provided based on the estimated compression ratio.

Claims

1. A method for dynamically computing an estimated compression ratio comprising:

compressing a predetermined number of random samples of data written to a virtual tape library;

measuring a compression ratio achieved when the samples are compressed;

measuring a deviation of compression ratios between two statistical measures;

computing an average estimated compression ratio of the data written to the virtual tape library; and

providing an end-of-tape signal based on the average estimated compression ratio indicating when to stop writing data to a virtual tape of the virtual tape library.

2. The method of claim 1 further including the step of compressing data written to the virtual tape library.

3. The method of claim 2 wherein the compression of data and computation of the estimated compression ratio are performed independent of each other.

4. The method of claim 2 wherein data is compressed asynchronously.

5. The method of claim 1 wherein the estimated compression ratio is computed taking into account results of data compression performed on data written to the virtual tape library.

6. The method of claim 1 further including the step of exporting data stored on the virtual tape to a physical tape based on the end-of-tape signal.

7. The method of claim 6 wherein the data is exported in compressed format.

8. The method of claim 6 wherein the data is decompressed prior to being exported to the physical tape.

9. The method of claim 1 wherein the frequency of random samples is adjusted manually.

10. The method of claim 1 wherein the frequency of random samples is adjusted dynamically.

11. The method of claim 1 wherein the frequency of the random samples is reduced when the deviation is below a first predetermined value.

12. The method of claim 1 wherein the frequency of the random samples is increased when the deviation is not below a first predetermined value and is above a second predetermined value.

13. The method of claim 1 wherein the frequency of the random samples is maintained at its current frequency when the deviation is not below a first predetermined value and is not above a second predetermined value.

14. A system for efficiently storing and transferring data in a virtual tape library, the system comprising:

a compressor compressing data written to the virtual tape library; and

a compression estimator for computing an estimated compression ratio, the compression estimator;

configured to compress a predetermined number of random samples of data written to the virtual tape library, measure a compression ratio achieved when the samples are compressed, measure a deviation of compression ratios between two statistical measures, compute an average estimated compression ratio of the data written to the virtual tape library and provide an end-of-tape signal based on the average estimated compression ratio, the end-of-tape signal indicating when to stop writing data to a virtual tape of the virtual tape library.

15. The system of claim 14 wherein the compression estimator computes the estimated compression ratio taking into account results of data compression performed on data written to the virtual tape library.

16. The system of claim 14 wherein the compressor is a back-end asynchronous compression agent for compressing data asynchronously.