Method and system for data compression and compression estimation in a virtual tape library environment

• US 20040230724A1
• Filed: 05/14/2003
• Published: 11/18/2004
• Est. Priority Date: 05/14/2003
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Johnston, Craig Anthony, Trimmer, Don Alvin, Blaser, Rico, Stager, Roger
• Granted Patent: US 7,437,492 B2
• Time in Patent Office: Days
• US Class Current: 710/68
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0638 Organizing or formatting or addressing of data
• CPC Class Codes: G06F 3/0664 at device level, e.g. emulation of a storage device or system
• CPC Class Codes: G06F 3/0686 Libraries, e.g. tape libraries, jukebox
• CPC Class Codes: G11B 20/00007 Time or data compression or expansion audio compression based on psychoacoustics G10L19/00; data processing for reproducing audio data at different playback speeds G10L21/04; video compression H04N19/00; data compression per se H03M7/30
• CPC Class Codes: H03M 7/30 Compression speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N; Expansion; Suppression of unnecessary data, e.g. redundancy reduction

Abstract

A method and system for efficiently storing and transferring data in a virtual tape library environment is disclosed. Data is written to a virtual tape library that emulates a physical tape library. Data stored in the virtual tape library may be compressed and an estimated compression ratio may be dynamically computed. While data is written to the virtual tape library, an end-of-tape signal is provided based on the estimated compression ratio.

Claims

1. A method for dynamically computing an estimated compression ratio comprising:

computing an estimated compression ratio for data written to a virtual tape library; and

providing an end-of-tape signal based on the estimated compression ratio indicating when to stop writing data to a virtual tape of the virtual tape library.

2. The method of claim 1 further including the step of compressing data written to the virtual tape library.

3. The method of claim 2 wherein the compression of data and computation of the estimated compression ratio are performed independent of each other.

4. The method of claim 2 wherein data is compressed in real-time.

5. The method of claim 2 wherein data is compressed asynchronously.

6. The method of claim 4 wherein the estimated compression ratio is computed taking into account results of data compression performed on data written to the virtual tape library.

7. The method of claim 5 wherein the estimated compression ratio is computed by:

compressing a predetermined number of random samples of data written to the virtual tape library;

measuring a compression ratio achieved when the samples are compressed;

measuring a deviation of compression ratios between two statistical measures; and

computing an average estimated compression ratio of the data written to the virtual tape library.

8. The method of claim 1 wherein the estimated compression ratio is computed by:

compressing a predetermined number of random samples of data written to the virtual tape library;

measuring a compression ratio achieved when the samples are compressed;

measuring a deviation of compression ratios between two statistical measures; and

computing an average estimated compression ratio of the data written to the virtual tape library.

9. The method of claim 1 further including the step of exporting data stored on a virtual tape to a physical tape based on the end of tape signal.

10. The method of claim 9 wherein the data is exported in compressed format.

11. The method of claim 9 wherein the data is decompressed prior to being exported to the physical tape.

12. The method of claim 8 wherein the frequency of random samples is adjusted manually.

13. The method of claim 8 wherein the frequency of random samples is adjusted dynamically.

14. The method of claim 8 wherein the frequency of the random samples is reduced wherein the deviation is below a first predetermined value.

15. The method of claim 8 wherein the frequency of the random samples is increased wherein the deviation is not below a first predetermined value and is above a second predetermined value.

16. The method of claim 8 wherein the frequency of the random samples is maintained at its current frequency when the deviation is not below a first predetermined value and is not above a second predetermined value.

17. A method for compressing data in a virtual tape library and computing an estimated compression ratio comprising:

compressing data written to a virtual tape library;

computing an estimated compression ratio for data written to the virtual tape library; and

providing an end-of-tape signal based on the estimated compression ratio indicating when to stop writing data to a virtual tape of the virtual tape library.

18. The method of claim 17 wherein the compression of data and computation of the estimated compression ratio are performed independent of each other.

19. A system for efficiently storing and transferring data in a virtual tape library, the system comprising:

means for compressing data written to a virtual tape library; and

a compression estimator for computing an estimated compression ratio and providing an end-of-tape signal based on the compression ratio, the end-of-tape signal indicating when to stop writing data to a virtual tape of the virtual tape library.

20. The system of claim 19 wherein the means for compressing data written to the virtual tape library is a real-time compressor for compressing data in real time.

21. The system of claim 20 wherein the compression estimator computes the estimated compression ratio taking into account results of data compression performed on data written to the virtual tape library.

22. The system of claim 19 wherein the means for compressing data written to the virtual tape library is a back-end asynchronous compression agent for compressing data asynchronously.

23. The system of claim 22 wherein the compression estimator computes the compression ratio by:

compressing a predetermined number of random samples of data written to the virtual tape library;

measuring a compression ratio achieved when the samples are compressed;

measuring a deviation of compression ratios between two statistical measures; and

computing an average estimated compression ratio of the data written to the virtual tape library.