Deduplicated data processing congestion control

• US 8,340,039 B2
• Filed: 05/04/2010
• Issued: 12/25/2012
• Est. Priority Date: 05/04/2010
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Akirav, Shay H., Bachar, Yariv, Levy, Asaf, Meiri, Ehud
• Primary Examiner(s): Chou, Albert T
• Application Number:
  US12/773,136
• Publication Number:
  US 20110273982A1
• Time in Patent Office: 966 Days
• Field of Search: 370/229, 370/230, 714/748, 714/781
• US Class Current: 370/329
• CPC Class Codes: G06F 16/90 Details of database functions independent of the retrieved data types
• CPC Class Codes: H04L 47/122 by diverting traffic away from congested entities
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Various embodiments for deduplicated data processing congestion control in a computing environment are provided. In one such embodiment, a single congestion metric is determined from a sampling of a plurality of combined deduplicated data processing congestion statistics in a number of active deduplicated data processes. The congestion limit is calculated from a comparison of the single congestion metric to a congestion target setpoint, the congestion target setpoint being a virtual dimension setpoint and the congestion limit being a manipulated variable. The number of active deduplicated data processes is compared to the congestion limit. If the number of active deduplicated data processes is less than the congestion limit, a new deduplicated data process is spawned.

Claims

1. A method for controlling deduplicated data processing congestion using at least one processor device in a computing environment, the method comprising:

obtaining a sampling of a plurality of combined deduplicated data processing congestion statistics of a number of active deduplicated data processes being performed by one or more deduplication systems;

determining a single congestion metric from the sampling;

calculating a congestion limit from a comparison of the single congestion metric to a congestion target setpoint, the congestion target setpoint being a virtual dimension setpoint and the congestion limit being a manipulated variable;

comparing the congestion limit to the number of active deduplicated data processes being performed by the one or more deduplication systems; and

instructing the one or more deduplication systems to initiate and perform a new deduplicated data process of the number of active deduplication data processes, if the number of active deduplicated data processes are less than the congestion limit.

2. The method of claim 1, further including, if the number of active deduplicated data processes is greater than the congestion limit, instructing the one or more deduplication systems to abort at least one of the number of active deduplicated data processes.

3. The method of claim 2, further including, if the number of active deduplicated data processes is equal to the congestion limit, instructing the one or more deduplication systems to wait a predetermined time.

4. The method of claim 1, further including collecting the sampling of the plurality of combined deduplicated data processing congestion statistics from at least one of read commands to a deduplicated storage, write commands to a deduplicated storage, and an average queue size of each the number of active deduplicated data processes, the average queue size being a ratio of a total time spent compared to a total time window duration.

5. The method of claim 1, further including assigning to each of the sampling of the plurality of combined deduplicated data processing congestion statistics at least one of a scalable metric value representative of a congestion range.

6. The method of claim 1, further including measuring the average response time of each of the number of active deduplicated data processes over a window of time, the window of time being a measurement of at least one of a normalized latency time period and a combination of a sum, a average, and a maximum time period.

7. The method of claim 1, further including, determining the manipulated variable in the congestion limit by a control loop feedback mechanism.

8. The method of claim 1, further including, categorizing each of the number of active deduplicated data processes being performed by one or more deduplication systems by a plurality of deduplicated data process types, and comparing the congestion limit to the number of active deduplicated data processes being performed by one or more deduplication systems for each of the plurality of deduplicated data process types.

9. A system for deduplicated data processing congestion control in a computing environment, comprising:

one or more deduplication systems configured to process deduplicated data; and

at least one processor device operable in the computing environment, wherein the at least one processor device is adapted for;

obtaining a sampling of a plurality of combined deduplicated data processing congestion statistics of a number of active deduplicated data processes being performed by the one or more deduplication systems;

determining a single congestion metric from the sampling;

calculating a congestion limit from a comparison of the single congestion metric to a congestion target setpoint, the congestion target setpoint being a virtual dimension setpoint and the congestion limit being a manipulated variable;

comparing the congestion limit to the number of active deduplicated data processes being performed by the one or more deduplication systems; and

instructing the one or more deduplication systems to initiate and perform a new deduplicated data process of the number of active deduplication data processes,

if the number of active deduplicated data processes are less than the congestion limit.

10. The system of claim 9, wherein the at least one processor device is further adapted for instructing the one or more deduplication systems to abort one of the number of active deduplicated data processes if the number of active deduplicated data processes is greater than the congestion limit.

11. The system of claim 9, wherein the at least one processor device is adapted for collecting the sampling of the plurality of combined deduplicated data processing congestion statistics from at least one of read commands to a deduplicated storage, write commands to a deduplicated storage, and an average queue size of each the number of active deduplicated data processes, the average queue size being a ratio of a total time spent compared to a total time window duration.

12. The system of claim 9, wherein the at least one processor device is adapted for assigning to each of the sampling of the plurality of combined deduplicated data processing congestion statistics at least one of a scalable metric value representative of a congestion range.

13. The system of claim 9, wherein the at least one processor device is adapted for measuring an average response time of each of the number of active deduplicated data processes over a window of time, a window of time being a measurement of at least one of a normalized latency time period and a combination of a sum, an average, and a maximum time period.

14. The system of claim 9, wherein the at least one processor device is adapted for determining the manipulated variable in the congestion limit by a control loop feedback mechanism.

15. The system of claim 9, wherein the at least one processor device is adapted for categorizing each of the number of active deduplicated data processes being performed by the one or more deduplication systems by a plurality of deduplicated data process types, and comparing the congestion limit to the number of active deduplicated data processes being performed by the one or more deduplication systems for each of the plurality of deduplicated data process types.

16. A computer program product for deduplicated data processing congestion control using at least one processor device in a computing environment, the computer program product comprising a non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:

a first executable portion for obtaining a sampling of a plurality of combined deduplicated data processing congestion statistics of a number of active deduplicated data processes being performed by one or more deduplication systems;

a second executable portion for determining a single congestion metric from the sampling;

a third executable portion for calculating a congestion limit from a comparison of the single congestion metric to a congestion target setpoint, the congestion target setpoint being a virtual dimension setpoint and the congestion limit being a manipulated variable;

a fourth executable portion for comparing the congestion limit to the number of active deduplicated data processes being performed by the one or more deduplication systems; and

a fifth executable portion for instructing the one or more deduplication systems to initiate and perform a new deduplicated data process of the number of active deduplication data processes, if the number of active deduplicated data processes are less than the congestion limit.

17. The program product of claim 16, further including a sixth executable portion for providing instructions to the one or more deduplication systems to abort one of the number of active deduplicated data processes if the number of active deduplicated data processes is greater than the congestion limit.

18. The program product of claim 16, further including a sixth executable portion for collecting the sampling of the plurality of combined deduplicated data processing congestion statistics from at least one of read commands to a deduplicated storage, write commands to a deduplicated storage, and an average queue size of each the number of active deduplicated data processes, the average queue size being a ratio of a total time spent compared to a total time window duration.

19. The program product of claim 16, further including a sixth executable portion for assigning to each of the sampling of the plurality of combined deduplicated data processing congestion statistics at least one of a scalable metric value representative of a congestion range.

20. The program product of claim 16, further including a sixth executable portion for measuring an average response time of each of the number of active deduplicated data processes over a window of time, a window of time being a measurement of at least one of a normalized latency time period and a combination of a sum, an average, and a maximum time period.

21. The program product of claim 16, further including a sixth executable portion for determining the manipulated variable in the congestion limit by a control loop feedback mechanism.

22. The program product of claim 16, further including a sixth executable portion for categorizing each of the number of active deduplicated data processes being performed by the one or more deduplication systems by a plurality of deduplicated data process types, and comparing the congestion limit to the number of active deduplicated data processes being performed by the one or more deduplication systems for each of the plurality of deduplicated data process types.