Predictive resource management

• US 9,965,498 B2
• Filed: 12/29/2010
• Issued: 05/08/2018
• Est. Priority Date: 12/29/2010
• Status: Active Grant

• Current Assignee: Teradata US, Inc. (Teradata Corporation)
• Original Assignee: Teradata US, Inc. (Teradata Corporation)
• Inventors: Coutts, Michael G., Scheie, Eric
• Primary Examiner(s): Cheema, Azam
• Application Number:
  US12/980,562
• Publication Number:
  US 20120173477A1
• Time in Patent Office: 2,687 Days
• CPC Class Codes: G06F 16/217 Database tuning G06F16/2282 takes precedence; database performance monitoring G06F11/3409

Abstract

Apparatus, systems, and methods may operate to monitor database system resource consumption over various time periods, in conjunction with scheduled data loading, data export, and query operations. Additional activities may include generating a database system resource consumption map based on the monitoring, and adjusting database system workload throttling to accommodate predicted database system resource consumption based on the resource consumption map and current system loading, prior to the current database resource consumption reaching a predefined critical consumption level. The current system loading may be induced by data loading, data export, or query activity. Other apparatus, systems, and methods are disclosed.

Claims

1. An apparatus, comprising:

a monitoring module to monitor database system resource consumption over time, in conjunction with scheduled data loading, data export, and query operations, said monitoring comprising;

receiving event messages associated with an occurrence of data loading, data export, or query jobs;

receiving automatic registration information associated with the data loading, data export, or query jobs; and

categorizing the data loading, data export, or query jobs as the scheduled data loading, data export, and query operations based on a number of repeated occurrences associated with the data loading, data export, or query jobs; and

a processor to;

generate a database system resource consumption map based on the database system resource consumption over time and in consideration of maximum resource consumption of the database system resource consumption taken over the time with respect to data loading, data export, and query operations of the database system;

determine, based on the database system resource consumption map and current database system loading associated with current database resource consumption, a predicted database system resource consumption as a projection of the system resource map;

adjust database system workload throttling to accommodate the predicted database system resource consumption based on the database system resource consumption map and current system loading induced by at least one of data loading, data export, or query activity, prior to the current database resource consumption reaching a predefined critical consumption level; and

wherein the adjusting comprises;

at least one of shifting data loads over time, restricting processor usage, restricting a number of concurrent user sessions, or restricting a number of reports generated for a selected time period.

2. The apparatus of claim 1, wherein the monitoring module comprises:

a processing node housing a systems manager to receive messages associated with the database system resource consumption.

3. The apparatus of claim 1, further comprising:

multiple database processors, including the processor, coupled to corresponding storage nodes that form a portion of the database system.

4. The apparatus of claim 1, further comprising:

a first node to house a storage medium used to store data supplied by the data loading; and

a second node to house the processor comprising at least one data access module processor.

5. The apparatus of claim 1, further comprising:

a display to display the current database resource consumption and the predicted database system resource consumption.

6. A processor-implemented method to execute on one or more processors that perform the method, comprising:

monitoring database system resource consumption over time, in conjunction with scheduled data loading, data export, and query operations, said monitoring comprising;

receiving event messages associated with an occurrence of data loading, data export, or query jobs;

receiving automatic registration information associated with the data loading, data export, or query jobs; and

categorizing the data loading, data export, or query jobs as the scheduled data loading, data export, and query operations based on a number of repeated occurrences associated with the data loading, data export, or query jobs;

generating a database system resource consumption map based on the monitoring; and

adjusting database system workload throttling to accommodate predicted database system resource consumption based on the database system resource consumption map and current system loading induced by at least one of data loading, data export, or query activity, prior to current database system resource consumption reaching a predefined critical consumption level; and

wherein the adjusting comprises;

at least one of shifting data loads over time, restricting processor usage, restricting a number of concurrent user sessions, or restricting a number of reports generated for a selected time period.

7. The method of claim 6, wherein the monitoring comprises:

monitoring at least one of processor usage, input/output operation quantity, or memory usage as the scheduled data loading, data export, and query operations occur over time.

8. The method of claim 6, wherein the monitoring comprises:

receiving messages generated by a systems manager, the messages communicating at least one of timing, periodicity, or resource consumption associated with external loads.

9. The method of claim 6, wherein the generating comprises:

generating a map of maximum resource consumption amounts per unit time associated with the current database system resource consumption.

10. The method of claim 6, wherein the adjusting comprises:

changing workload definitions to provide reduced access to data warehouse resources associated with the database system.

11. The method of claim 6, wherein the adjusting comprises:

at least one of filtering or rejecting requests initiated by a selected user for a selected time period.

12. The method of claim 6, wherein the adjusting comprises:

removing workload access rights associated with at least one of the data loading, data export, or query activity.

13. The method of claim 6, further comprising:

activating an alarm to indicate that the current database system resource consumption is expected to reach the predefined critical consumption level within a selected alarm time period.

14. The method of claim 6, wherein the monitoring comprises:

accessing data provided by a time-based data collection service to determine at least one of timing, periodicity, or resource consumption associated with external loads.

15. The method of claim 6, further comprising:

displaying the predicted database system resource consumption in conjunction with a specific future time along a scaled timeline that includes a current time and the specific future time.

16. A processor-implemented method to execute on one or more processors that perform the method, comprising:

monitoring current database system resource consumption over time across multiple database systems, in conjunction with scheduled data loading and query operations, said monitoring comprising;

receiving event messages associated with an occurrence of data loading or query jobs;

receiving automatic registration information associated with the data loading or query jobs; and

categorizing the data loading or query jobs as the scheduled data loading and query operations based on a number of repeated occurrences associated with the data loading or query jobs;

generating a database system resource consumption map based on the monitoring; and

adjusting database system workload throttling to accommodate predicted database resource consumption based on the database system resource consumption map and current system loading induced by at least one of data loading or query activity, prior to the current database system resource consumption for a first one of the multiple database systems reaching a predefined critical consumption level; and

wherein the adjusting comprises;

redirecting a loading process or queries from a first one of the multiple database systems to a second one of the multiple database systems, wherein predicted relative database system resource consumption for the first one is greater than predicted relative database system resource consumption for the second one.

17. The method of claim 16, wherein the adjusting comprises:

moving data from the first one of the multiple database systems to a second one of the multiple database systems, wherein the predicted relative database system resource consumption for the first one is greater than the predicted relative database system resource consumption for the second one.

18. A non-transitory computer readable storage medium storing at least executable computer code, wherein the executable computer code includes:

executable computer code for monitoring database system resource consumption of one or more database systems over time, in conjunction with scheduled data loading, data export, and query operations of the one or more database systems, said monitoring comprising;

receiving event messages associated with an occurrence of data loading, data export, and query jobs;

receiving automatic registration information associated with the data loading, data export, and query jobs; and

categorizing the data loading, data export, and query jobs as the scheduled data loading, data export, and query operations based on a number of repeated occurrences associated with the data loading, data export, and query jobs; and

executable computer code for generating a database system resource consumption map based on the database system resource consumption over time, and to adjust database system workload throttling to accommodate predicted database system resource consumption of the one or more database systems based on the database system resource consumption map and current system loading induced by at least one of data loading, data export, or query activity of the one or more database systems, prior to current database resource consumption reaching a predefined critical consumption level; and

wherein the adjusting comprises;

at least one of shifting data loads over time, restricting processor usage, restricting a number of concurrent user sessions, or restricting a number of reports generated for a selected time period.