Systems and methods for managing resource utilization in information management environments

• US 20030046396A1
• Filed: 04/05/2002
• Published: 03/06/2003
• Est. Priority Date: 03/03/2000
• Status: Abandoned Application

• Current Assignee: Surgient Networks, Inc. (Quest Software, Inc.)
• Original Assignee: Surgient Networks, Inc. (Quest Software, Inc.)
• Inventors: Qiu, Chaoxin C., Richter, Roger K., Johnson, Scott C.
• Application Number:
  US10/117,028
• Publication Number:
  US 20030046396A1
• Time in Patent Office: Days
• US Class Current: 709/226
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3433 for load management allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 2209/508 Monitor
• CPC Class Codes: G06F 9/505 considering the load

Abstract

Resource usage accounting may be implemented in information management environments using resource utilization values. Resource usage accounting may be employed, for example, to make possible run-time enforcement of system operations on one or more subsystems or processing engines of an information management system, such as a content delivery system, for example, to advantageously provide intelligent admission control in a distributed environment. In one embodiment, resource usage accounting may be implemented to make possible the management of system resources on a per subsystem or processing engine basis, for example, based on at least two types of resource utilization indicative information: 1) resource usage that has been tracked internally throughout the life span of the overload and policy finite state machine module; and 2) resource status messages received directly or indirectly from one or more subsystems or processing engines.

Claims

1. A method of performing run-time enforcement of system operations in an information management environment in which multiple information management tasks are performed, comprising:

monitoring resource consumption for each of said multiple information manipulation tasks performed in said information management environment based on an individual resource utilization value that is reflective of the resource consumption required to perform each of said multiple information manipulation tasks;

tracking total resource consumption to perform said multiple information manipulation tasks in said information management environment based on said individual resource utilization values; and

controlling said total resource consumption to avoid over utilization of one or more resources within said information management environment.

2. The method of claim 1, wherein said controlling comprises at least one of differentiated service enforcement, overload protection, resource utilization threshold enforcement, or a combination thereof.

3. The method of claim 1, wherein said multiple information manipulation tasks comprise current information manipulation tasks performed in said information management environment;

wherein said tracking comprises tracking current total resource consumption to perform said current multiple information manipulation tasks; and

wherein said method further comprises;

responding to requests to perform new requested information manipulation tasks in said information management environment by predicting a new resource consumption associated with the performance of said new information manipulation tasks;

calculating an incremental total resource consumption based on a combination of said new resource consumption and said current total resource consumption; and

controlling said current total resource consumption by performing admission control on said requests for new information manipulation tasks to admit or reject each of said requests for performance in said information management environment.

4. The method of claim 3, wherein said method comprises using estimation-based resource usage accounting to track said current total resource consumption.

5. The method of claim 4, wherein said method further comprises measuring a value of current total resource consumption;

comparing said measured current total resource consumption with said estimated total resource consumption; and

transitioning between status-driven usage accounting and said estimation-based resource usage accounting based on the magnitude of the difference between said measured value of current total resource consumption and said estimated value of total resource consumption.

6. The method of claim 5, wherein said method further comprises transitioning to a transient state in an attempt to confirm said measured value of current total resource consumption;

said transitioning to said transient state being based on a magnitude of the difference between said measured value of current total resource consumption and said estimated value of total resource consumption.

7. The method of claim 1, further comprising generating at least a portion of said individual resource utilization values in real time based on performance testing.

8. The method of claim 3, further comprising performing arrival shaping on said requests for new information manipulation tasks prior to performing said responding, calculating and controlling.

9. The method of claim 8, further comprising differentially performing at least one admitted request for new information manipulation task/s relative to at least one other admitted request for new information manipulation task/s to achieve differentiated service within said information management environment.

10. The method of claim 1, wherein said information management environment comprises a heterogeneous information management system environment.

11. The method of claim 10, wherein said method comprises performing said run-time enforcement of system operations in a network connectable information management system comprising multiple processing engines assigned separate information manipulation tasks in an asymmetrical multi-processor configuration, said plurality of processing engines being coupled together with a distributed interconnect.

12. The method of claim 11, wherein said distributed interconnect comprises a virtual distributed interconnect.

13. The method of claim 11, wherein said distributed interconnect comprises a switch fabric.

14. The method of claim 13, further comprising characterizing a relative state of resource utilization within one or more of said multiple processing engines based on a value of a total resource utilization value within said one or more of said multiple processing engines relative to one or more state thresholds.

15. The method of claim 14, further comprising load-balancing among two or more of said multiple processing engines based on said characterizing.

16. The method of claim 13, wherein said controlling comprises using at least one of said individual multiple processing engines to independently perform admission control on at least one request for a new information manipulation task based on availability of resources of said individual processing engine that are available to perform said requested information manipulation task.

17. The method of claim 13, wherein said information management system comprises a content delivery system.

18. The method of claim 17, wherein said method comprises performing said run-time enforcement of system operations for each processing engine of said multiple processing engines.

19. The method of claim 17, wherein said method comprises performing said run-time enforcement of system operations for each processing engine in the data processing path implemented by a given request to perform new requested information manipulation tasks in said information management environment.

20. The method of claim 18, wherein each of said multiple processing engines comprises one or more resource principals;

and wherein said method further comprises quantifying one or more of said resource principals for each of said multiple processing engines to determine said resource utilization values.

21. The method of claim 20, wherein said resource utilization values are expressed in terms of a number of resource capacity utilization units to support delivery of said content at a designated rate;

and wherein said controlling comprises implementing overload protection using said resource utilization values to ensure said delivery of said content at said designated rate.

22. The method of claim 20, wherein said multiple resource principals comprise at least one of memory resources, compute resources, I/O resources, number of buffers, number of current connections, number of new connections number of dropped-out connections, number of interfaces, transaction latency, number or outstanding I/O requests, disk drive utilization, loading of applications, or a combination thereof.

23. The method of claim 20, wherein said multiple resource principals comprise at least one of memory resources, compute resources, I/O resources, or a combination thereof.

24. The method of claim 20, wherein said method comprises determining at least one of said resource utilization values based on a combination of two or more of said resource principals.

25. The method of claim 22, wherein said tracking comprises performing resource usage accounting for each of said multiple processing engines in said information management system.

26. The method of claim 1, further comprising using said run-time enforcement of system operations to achieve differentiated service within said information management environment.

27. The method of claim 26, wherein said information management system comprises a multi-tenant environment, a multi-class of service environment, or a combination thereof;

and wherein said method further comprises enforcing policy-based access and delivery of said resources in said multi-tenant environment, said multi-class of service environment, or a combination thereof.

28. A method of enforcing differentiated service in an information management environment in which multiple information management tasks are performed, comprising:

performing resource usage accounting in said information management environment; and

enforcing said differentiated service with respect to the performance of at least one of said information management tasks based at least in part on said resource usage accounting.

29. The method of claim 28, wherein sad differentiated service enforcement comprises enforcing policy-based access and delivery of system or subsystem resources in a multi-tenant environment, a multi-service environment, or a combination thereof.

30. The method of claim 28, wherein said differentiated service enforcement comprises using said resource usage accounting to track current resource utilization relative to maximum resource utilization thresholds;

and allocating available system or subsystem resources to multiple tenants based upon said tracked current resource utilization according to one or more differentiated service policies in a manner that guarantees sufficient system or subsystem resource availability to satisfy said one or more differentiated service policies without degradation of service quality.

31. The method of claim 28, wherein said differentiated service enforcement comprise using said resource usage accounting to ensure said information management task may be performed in a manner that satisfies a guaranteed service level for performing said information manipulation task.

32. The method of claim 31, wherein said differentiated service enforcement comprises enforcing a given SLA contract to ensure that the guaranteed throughput of the contract will be realized and that best effort services will be supported with potential discriminations.

33. The method of claim 28, wherein said differentiated service enforcement comprises dispatching for processing at least one admitted request for a new information manipulation task in a manner that is differentiated relative to dispatching for processing at least one other admitted request for a new information manipulation task.

34. The method of claim 33, performing said dispatching by using a task scheduling policy by which admitted requests for new information manipulation tasks are placed in multiple dispatch queues and dispatched for processing based on a respective CoS priority associated with each of said requests.

35. The method of claim 28, wherein said resource usage accounting is performed by a method comprising:

characterizing resource consumption for each of said multiple information manipulation tasks performed in said information management environment based on an individual resource utilization value that is reflective of the resource consumption required to perform each of said multiple information manipulation tasks; and

tracking total resource consumption to perform said multiple information manipulation tasks in said information management environment based on said individual resource utilization values.

36. The method of claim 35, further comprising generating at least a portion of said individual resource utilization values in real time based on performance testing.

37. The method of claim 28, wherein said method further comprises:

monitoring resource consumption for each of said multiple information manipulation tasks performed in said information management environment based on an individual resource utilization value that is reflective of the resource consumption required to perform each of said multiple information manipulation tasks;

tracking total resource consumption to perform said multiple information manipulation tasks in said information management environment based on said individual resource utilization values; and

controlling said total resource consumption to differentiate between the performance of said at least two of said information management tasks.

38. The method of claim 37, wherein said multiple information manipulation tasks comprise current information manipulation tasks performed in said information management environment;

wherein said tracking comprises tracking current total resource consumption to perform said current multiple information manipulation tasks; and

wherein said method further comprises;

responding to requests to perform new requested information manipulation tasks in said information management environment by predicting a new resource consumption associated with the performance of said new information manipulation tasks;

calculating an incremental total resource consumption based on a combination of said new resource consumption and said current total resource consumption; and

controlling said current total resource consumption by performing admission control on said requests for new information manipulation tasks to admit or reject each of said requests for performance in said information management environment.

39. The method of claim 37, wherein said method comprises using estimation-based resource usage accounting to track said current total resource consumption.

40. The method of claim 39, wherein said method further comprises measuring a value of current total resource consumption;

comparing said measured current total resource consumption with said estimated total resource consumption; and

transitioning between status-driven usage accounting and said estimation-based resource usage accounting based on the magnitude of the difference between said measured value of current total resource consumption and said estimated value of total resource consumption.

41. The method of claim 40, wherein said method further comprises transitioning to a transient state in an attempt to confirm said measured value of current total resource consumption;

said transitioning to said transient state being based on a magnitude of the difference between said measured value of current total resource consumption and said estimated value of total resource consumption.

42. The method of claim 28, further comprising performing arrival shaping on requests for new information manipulation tasks prior to performing said resource usage accounting and enforcing said differentiated service.

43. The method of claim 28, further comprising differentially performing at least one admitted request for new information manipulation task/s relative to at least one other admitted request for new information manipulation task/s to achieve differentiated service within said information management environment.

44. The method of claim 28, wherein said information management environment comprises a heterogeneous information management system environment.

45. The method of claim 44, wherein said method comprises enforcing said differentiated service in an information management system comprising multiple processing engines assigned separate information manipulation tasks in an asymmetrical multi-processor configuration, said plurality of processing engines being coupled together with a distributed interconnect.

46. The method of claim 45, wherein said distributed interconnect comprises a virtual distributed interconnect.

47. The method of claim 45, wherein said distributed interconnect comprises a switch fabric.

48. The method of claim 47, further comprising characterizing a relative state of resource utilization within one or more of said multiple processing engines based on a value of a total resource utilization value within said one or more of said multiple processing engines relative to one or more state thresholds.

49. The method of claim 48, further comprising load-balancing among two or more of said multiple processing engines based on said characterizing.

50. The method of claim 47, wherein said enforcing comprises using at least one of said individual multiple processing engines to independently perform admission control on at least one request for a new information manipulation task based on availability of resources of said individual processing engine that are available to perform said requested information manipulation task.

51. The method of claim 47, wherein said information management system comprises a content delivery system.

52. The method of claim 51, wherein said method comprises enforcing said differentiated service for each processing engine of said multiple processing engines.

53. The method of claim 51, wherein said method comprises enforcing said differentiated service for each processing engine in the data processing path implemented by a given request to perform new requested information manipulation tasks in said information management environment.

54. The method of claim 52, wherein said resource usage accounting is performed by a method comprising characterizing resource consumption for each of said multiple information manipulation tasks performed in said information management environment based on an individual resource utilization value that is reflective of the resource consumption required to perform each of said multiple information manipulation tasks, and tracking total resource consumption to perform said multiple information manipulation tasks in said information management environment based on said individual resource utilization values;

and

wherein each of said multiple processing engines comprises one or more resource principals; and

wherein said method further comprises quantifying one or more of said resource principals for each of said multiple processing engines to determine said resource utilization values.

55. The method of claim 54, wherein said resource utilization values are expressed in terms of a number of resource capacity utilization units to support delivery of said content at a designated rate;

and wherein said controlling comprises implementing overload protection using said resource utilization values to ensure said delivery of said content at said designated rate.

56. The method of claim 54, wherein said multiple resource principals comprise at least one of memory resources, compute resources, I/O resources, number of buffers, number of current connections, number of new connections number of dropped-out connections, number of interfaces, transaction latency, number or outstanding I/O requests, disk drive utilization, loading of applications, or a combination thereof.

57. The method of claim 54, wherein said multiple resource principals comprise at least one of memory resources, compute resources, I/O resources, or a combination thereof.

58. The method of claim 54, wherein said method comprises determining at least one of said resource utilization values based on a combination of two or more of said resource principals.

59. The method of claim 54, wherein said tracking comprises performing resource usage accounting for each of said multiple processing engines in said information management system.

60. The method of claim 28, further comprising using said run-time enforcement of system operations to achieve differentiated service within said information management environment.

61. The method of claim 60, wherein said information management system comprises a multi-tenant environment, a multi-class of service environment, or a combination thereof;

and wherein said method further comprises enforcing policy-based access and delivery of said resources in said multi-tenant environment, said multi-class of service environment, or a combination thereof.

62. A determinism module for use in an information management environment, comprising an overload and policy finite state machine module and a resource usage accounting module.

63. The determinism module of claim 62, further comprising a resource utilization table module, a subsystem status monitor, a load information distribution module, and a self calibration module.

64. The determinism module of claim 63, wherein said subsystem status monitor is configured to receive resource status messages from at least one of a wellness/availability module in communication with said determinism module, one or more individual subsystems of an information management system that are in communication with said determinism module, or a combination thereof.

65. The determinism module of claim 63, wherein said resource utilization table module is configured to make available resource utilization values to said resource usage accounting module.

66. The determinism module of claim 62, wherein said determinism module comprises a resource usage accounting module that is configured to track workload within said information management environment.

67. The determinism module of claim 66, wherein said resource usage accounting module is configured to track current workload within said information management environment;

and to track incremental workload estimated to be required to fulfill both current workload and requested workload within said information management environment.

68. The determinism module of claim 67, wherein said overload and policy finite state machine module is configured to obtain said current workload and said incremental workload from said resource usage accounting module;

and wherein said overload and policy finite state machine module is further configured to compare said current workload with said incremental workload, and to decide whether or not the requested workload is to be performed within said information management environment based on said comparison.

69. The determinism module of claim 68, further comprising a subsystem status monitor configured to receive information on said current workload from at least one of a wellness/availability module in communication with said determinism module, from one or more individual subsystems of an information management system that are in communication with said determinism module, or a combination thereof.

70. A network connectable information management system, comprising:

a plurality of multiple processing engines coupled together by a distributed interconnect; and

a determinism module coupled to said multiple processing engines via said distributed interconnect.

71. The system of claim 70, wherein said distributed interconnect comprises a virtual distributed interconnect.

72. The system of claim 70, wherein said distributed interconnect comprises a switch fabric.

73. The system of claim 72, wherein at least one of said individual multiple processing engines is configured to independently perform admission control for requests for new information manipulation tasks.

74. The system of claim 72, wherein said multiple processing engines comprise a system management processing engine;

and wherein said determinism module is implemented on said system management processing engine.

75. The system of claim 74, wherein said information management system comprises a content delivery system;

and wherein said multiple processing engines further comprise at least one application processing engine, at least one storage processing engine, at least one transport processing engine, and at least one network interface processing engine.

76. The system of claim 75, wherein said determinism module comprises an overload and policy finite state machine module, a resource usage accounting module, a resource utilization table module, a subsystem status monitor, a load information distribution module, and a self calibration module.

77. The system of claim 76, wherein said subsystem status monitor is configured to receive resource status messages from at least one of a wellness/availability module in communication with said determinism module, one or more individual subsystems of an information management system that are in communication with said determinism module, or a combination thereof.

78. The system of claim 76, wherein said resource utilization table module is configured to make available resource utilization values to said resource usage accounting module.

79. The system of claim 76, wherein said resource utilization table module is further configured to make available resource utilization values to said resource usage accounting module, at least a portion of said resource utilization values being generated in real time by said self-calibration module based on one or more performance measurements.

80. The system of claim 75, wherein said determinism module comprises a resource usage accounting module that is configured to track system and/or subsystem workloads.

81. The system of claim 80, wherein said resource usage accounting module is configured to track current system/subsystem workloads to fulfill current admitted requests;

and to track incremental system/subsystem workloads estimated to be required to fulfill both existing requests and new request/s that are not yet admitted.

82. The system of claim 81, wherein said determinism module further comprises an overload and policy finite state machine module that is configured to obtain said current system/subsystem workload and said incremental system/subsystem workload from said resource usage accounting module;

and wherein said overload and policy finite state machine module is further configured to compare said current system/subsystem workload with said incremental system/subsystem workload, and to decide whether or not the new request is to be granted based upon said comparison.

83. The system of claim 81, wherein said determinism module further comprises a subsystem status monitor configured to receive information on said current system/subsystem workload from at least one of a wellness/availability module in communication with said determinism module, from one or more individual subsystems of an information management system that are in communication with said determinism module, or a combination thereof.