Dynamic reconfiguration of data storage devices to balance recycle throughput

• US 5,875,481 A
• Filed: 01/30/1997
• Issued: 02/23/1999
• Est. Priority Date: 01/30/1997
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Ashton, Lyn Lequam, Pearson, Anthony Steve, Pence, Jerry Wayne, West, Stephen Charles
• Primary Examiner(s): Chan, Eddie
• Assistant Examiner(s): Nguyen, Hiep T.
• Application Number:
  US08/792,826
• Time in Patent Office: 754 Days
• Field of Search: 711/165, 711/162, 711/112, 711/173, 711/159, 395/182.04, 395/872, 707/204, 707/206
• US Class Current: 711/165
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: Y10S 707/99953 Recoverability
• CPC Class Codes: Y10S 707/99957 Garbage collection

Abstract

A data processing system includes a data storage subsystem containing a plurality of data storage devices for reading and writing data on a plurality of data storage volumes. The data processing system is controlled by software to perform a method for recycling the data storage volumes containing a percentage of still valid data and a percentage of invalid data by transferring the still valid data on a plurality of input data storage volumes to one or more output data storage volumes. In accordance with the recycling method, the data processing system assigns a selected number of the data storage devices as input devices and a selected number of the data storage devices as output devices in a predetermined input/output ratio of the form m:k, where m is the number of input devices and k is the number of output devices. The system then commences transferring still valid data from input volumes mounted on the input devices to output volumes mounted on the output devices. As each volume is transferred, the system monitors the data transfer efficiency from the input devices to the output devices and adjusts the input/output ratio based on the measured data transfer efficiency.

Claims

1. In a data processing system having a data storage subsystem containing a plurality (n) of data storage devices for reading and writing data on a plurality of data storage volumes, a method for recycling data storage volumes of said plurality of data storage volumes which contain a percentage of still valid data and a percentage of invalid data by transferring the still valid data on a plurality of input data storage volumes to one or more output data storage volumes, said method comprising the steps of:

assigning a selected number of said data storage devices as input devices and a selected number of said data storage devices as output devices in a predetermined input/output ratio of the form m;

k, where m is the number of input devices, k is the number of output devices and m+k=n;

transferring still valid data from input data storage volumes mounted on said input devices to output data storage volumes mounted on said output devices;

measuring data transfer efficiency from said input devices to said output devices, wherein the data transfer efficiency is a function of an average processing time and an average wait time over a selected number of data storage volumes previously processed; and

adjusting said input/output ratio based on the measured data transfer efficiency.

2. The method of claim 1 wherein said data storage volumes are processed in order from data storage volumes containing the lowest percentage of still valid data to data storage volumes containing the highest percentage of still valid data, and wherein said adjusting step includes incrementally increasing the number of output devices and incrementally decreasing the number of input devices.

3. The method of claim 2 wherein said input/output ratio m:

k is initially set to m=(n-1) and k=1, such that there is initially only a single output device.

4. The method of claim 3 wherein said monitoring step includes monitoring data storage volume throughput and said adjusting step includes increasing the number of output devices by one device and decreasing the number of input devices by one device when said throughput would increase as a result of said adjustment.

5. The method of claim 4 wherein said monitoring step includes measuring the processing time per data storage volume (ViPt) and the wait time per data storage volume (ViWt) and calculating an average processing time (AVPT) and an average wait time (AVWT) over a selected number of data storage volumes previously processed (NINV).

6. The method of claim 5 wherein said adjusting step is performed to increase the number of output devices by one device and decrease the number of input devices by one device if AVWT>

AVPT/NINV and the adjustment would not result in more output devices than input devices.

7. The method of claim 5 wherein said adjusting step further includes incrementally decreasing the number of output devices and incrementally increasing the number of input devices when the average wait time (AVWT) is less than a predetermined percentage of the average data storage volume processing time (AVPT).

8. The method of claim 7 wherein said adjusting step is performed to decrease the number of output devices by one device and increase the number of input devices by one device when said average wait time (AVWT) is equal to or less than about 2%-5% of the average data storage volume processing time AVPT.

9. The method of claim 1 wherein said data transferring step includes selecting an output device to write still valid data to by selecting a first available device from a variable device set ordering.

10. The method of claim 5 wherein said average data storage volume processing time (AVPT) and said average data storage volume wait time (AVWT) are calculated using a fading memory filter.

11. The method of claim 1 wherein said data transferring step includes buffering said still valid data as necessary based on output device availability.

12. A data processing system including a data storage subsystem containing a plurality (n) of n data storage devices for reading and writing data on a plurality of data storage volumes and a recycle control system for recycling data storage volumes of said plurality of data storage volumes which contain a percentage of still valid data and a percentage of invalid data by transferring the still valid data on a plurality of input data storage volumes to one or more output data storage volumes, said recycle control system comprising:

means for assigning a selected number of said data storage devices as input devices and a selected number of said data storage devices as output devices in a predetermined input/output ratio of the form m;

k, where m is the number of input devices, k is the number of output devices and m+k=n;

means for transferring still valid data from input data storage volumes mounted on said input devices to output data storage volumes mounted on said output devices;

means for measuring data transfer efficiency from said input devices to said output devices, wherein the data transfer efficiency is a function of an average processing time and an average wait time over a selected number of data storage volumes previously processed; and

means for adjusting said input/output ratio based on the measured data transfer efficiency.

13. The data processing system of claim 12 wherein said data storage volumes are processed in order from data storage volumes containing the lowest percentage of still valid data to data storage volumes containing the highest percentage of still valid data, and wherein said adjusting means includes means for incrementally increasing the number of output devices and incrementally decreasing the number of input devices.

14. The data processing system of claim 13 wherein said input/output ratio m:

k is initially set to m=(n-1) and k=1, such that there is initially only a single output device.

15. The data processing system of claim 14 wherein said monitoring means includes means for monitoring data storage volume throughput and said adjusting means includes means for increasing the number of output devices by one device and decreasing the number of input devices by one device when said throughput would increase as a result of said adjustment.

16. The data processing system of claim 15 wherein said monitoring means includes means for measuring the processing time per data storage volume (ViPt) and the wait time per data storage volume (ViWt) and means for calculating an average processing time (AVPT) and an average wait time (AVWT) over a selected number of data storage volumes previously processed (NINV).

17. The data processing system claim 16 wherein said adjusting means increases the number of output drives by one device and decreases the number of input drives by one device if AVWT>

AVPT/NINV and the adjustment would not result in more output devices than input devices.

18. The data processing system of claim 16 wherein said adjusting means incrementally decreases the number of output devices and incrementally increases the number of input devices when the average wait time (AVWT) is less than a predetermined percentage of the average data storage volume processing time (AVPT).

19. The data processing system of claim 17 wherein said adjusting means decreases the number of output devices by one device and increases the number of input devices by one device when said average wait time (AVWT) is equal to or less than about 2%-5% of the average data storage volume processing time AVPT.

20. The data processing system of claim 12 wherein said data transferring means includes means for selecting an output device to write still valid data to by selecting a first available device from a variable device set ordering.

21. The data processing system of claim 16 wherein said average data storage volume processing time (AVPT) and said average data storage volume wait time (AVWT) are calculated using a fading memory filter.

22. The data processing system of claim 12 wherein said data transferring means includes means for buffering said still valid data as necessary based on output device availability.

23. In a data processing system having a data storage subsystem containing a plurality of n data storage devices for reading and writing data on a plurality of data storage volumes, a method for recycling said data storage volumes containing a percentage of still valid data and a percentage of invalid data by transferring the still valid data on a plurality of input data storage volumes to one or more output data storage volumes, said method comprising the steps of:

assigning a selected number of said data storage devices as input devices and a selected number of said data storage devices as output devices in a predetermined input/output ratio of the form m;

k, where m is the number of input devices, k is the number of output devices and m+k=n, said input/output ratio m;

k being initially set to m=(n-1) and k=1, such that there is initially only a single output device;

arranging said data storage volumes for processing in order from data storage volumes containing the lowest percentage of still valid data to data storage volumes containing the highest percentage of still valid data;

transferring still valid data from input data storage volumes mounted on said input devices to output data storage volumes mounted on said output devices;

monitoring the data transfer efficiency from said input devices to said output devices, including measuring the processing time per data storage volume (ViPt). and the wait time per data storage volume (ViWt) and calculating an average processing time (AVPT) and an average wait time (AVWT) over a selected number of data storage volumes previously processed (NINV); and

adjusting said input/output ratio by incrementally increasing the number of output devices and incrementally decreasing the number of input devices when said throughput would increase as a result of said adjustment if AVWT>

AVPT/NINV and the adjustment would not result in more output devices than input devices, and refraining from adjusting said input/output ratio if AVWT<

=AVPT/NINV or if the adjustment would result in more output devices than input devices.