System for managing and controlling storage access requirements

• US 20040205206A1
• Filed: 02/19/2003
• Published: 10/14/2004
• Est. Priority Date: 02/19/2003
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Narang, Inderpal Singh, Pandey, Prashant, Naik, Vijay K., Varma, Pradeep, Sharma, Upendra, Alur, Nagraj, Gogate, Vitthal Mahadeo, Jadav, Divyesh
• Granted Patent: US 7,827,283 B2
• Time in Patent Office: Days
• US Class Current: 709/230
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/61 taking into account QoS or priority requirements
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04L 9/40 Network security protocols

Abstract

A Resource Management and Reservation System (RMRS) for managing and reserving storage bandwidth, is a platform independent middleware layer that provides an interface to applications, their database management systems, or some other higher level data management systems like ADRS which does data management on behalf of the applications. RMRS is highly relevant in hosted environments where one or more applications may be run on behalf of multiple customers each with a unique service level agreement with the Service Provider. Through its interface to the aforementioned applications, RMRS allows each application or an application side management system to communicate expected future storage access requirements (e.g., periodic access for backups). The interface also allows applications to request urgent storage access (e.g., recovery actions may be requested without being planned ahead of time).

Claims

1. A Resource Management and Reservation System (RMRS)for managing and reserving storage bandwidth by controlling storage access requests which provides an interface to applications comprising:

a computer network to which it is connected

an ADRS interface element and a SAN interface element;

a scheduler/optimizer handler element and a RMRS repository element, each connected to said ADRS interface and to each other;

a SAN Gateway handler which is connected to said SAN interface and to said scheduler/optimizer handler element and said RMRS repository element; and

a schedule adjuster and a schedule optimizer each connected to said scheduler/optimizer handler element and said RMRS repository element.

2. The Resource Management and Reservation System defined in claim 1 wherein said SAN interface comprises subcomponents comprising a SAN Gateway Monitor, a SAN Gateway Controller and a SAN QoS Analyzer.

3. The Resource Management and Reservation System defined in claim 2 wherein said ADRS is a client of said RMRS, interacts with RMRS over the Internet for scheduling I/O intensive backup and recovery I/O tasks associated with multiple applications hosted by a Service Provider.

4. The Resource Management and Reservation System defined in claim 3 wherein when said backup or recovery task is scheduled, said RMRS interact with said SAN Gateway and monitors in real soft time the effective bandwidth realized by said task.

5. The method using the Resource Management and Reservation System defined in claim 1 comprising:

the ADRS interface uses a computer network to communicate with ADRS and/or hosted applications;

said schedule handler interacts with said RMRS repository, said schedule optimizer and said schedule adjuster;

said schedule optimizer and said schedule adjuster communicate with said RMRS repository and with said schedule handler;

said RMRS repository also interacts with said SAN Gateway Handler component which in turn interacts with said SAN Gateway via said SAN interface.

6. The method using the Resource Management and Reservation System defined in claim 2 comprising;

deploying said RMRS as a web service wherein its services are accessed over a network using web services invocation protocols;

using interfaces provided by RMRS, said ADRS invokes at least one RMRS APIs to convey to RMRS information related to dynamic changes in I/O requirements taking place in response to changes in the state of hosted applications.

7. The method using the Resource Management and Reservation System defined in claim 6 wherein said RMRS interfaces allow ADRS:

(i) lo register itself as a client of RMRS, (ii) to define and register a new task and its attributes, (iii) to update the attributes of an already registered task, (iv) to un-register a previously registered task, (v) to generate multiple instances of defined task, (vi) to request a registered task schedule for a single instance or multiple instances for said task), (vii) to verify the current validity of a schedule.

8. The method using the Resource Management and Reservation System defined in claim 7 wherein using said interfaces, said ADRS defines and registers said tasks, said tasks comprising Sequential Backup, Parallel Backup, Incremental Backup, Sequential Recovery, Parallel Recovery.

9. The method using the Resource Management and Reservation System defined in claim 8 wherein attributes of said task include task priority, task frequency, constraints on when a task may be performed, expected time to completion, maximum time limit for completion, range for the data size being accessed or stored, and a list of resources needed to perform the task.

10. The method using the Resource Management and Reservation System defined in claim 9 wherein defining said task with RMRS makes RMRS register said task information in said RMRS Repository and the object information on which said task is performed.

11. The method using the Resource Management and Reservation System defined in claim 10 wherein said object information is collected from a file set or the database associated with the particular instance of the application on which the task is performed, and this information is stored in said RMRS repository.

12. The method using the Resource Management and Reservation System defined in claim 11 wherein said task defined with RMRS is invoked in a plurality of instances, and each instance of said task inherits the attributes of the originally defined task, with the ability of the defined task to be overridden for a particular invocation of said task.

13. The method using the Resource Management and Reservation System defined in claim 12 wherein upon defining a task with RMRS, a unique ID is generated for that task.

14. The method using the Resource Management and Reservation System defined in claim 13 wherein upon defining a task, RMRS evaluates and returns if additional resource capacity is required to service the new task in the presence of other tasks already defined and scheduled.

15. The method using the Resource Management and Reservation System defined in claim 14 wherein said evaluation is performed irrespective of the priority of said new task.

16. The method using the Resource Management and Reservation System defined in claim 15 wherein the event the task cannot be scheduled because of resource shortage, RMRS registers the task successfully, and indicates to said ADRS that additional resource capacity is required for servicing the new task, without compromising the previously defined tasks, said analysis being provided to assist ADRS and service Provider in capacity planning whenever a new application is launched or when a new customer is signed in.

17. The method using the Resource Management and Reservation System defined in claim 16 wherein in the event said new application is launched or a new customer is signed in, ADRS has one of the three choices:

(i) provisioning of additional resources before scheduling the task, (ii) continuing with available resources, or (iii) sending a request to RMRS to un-register the task definition. Policies may be set that would allow ADRS to make a decision or the Service Provider optionally directly intervene and make a decision.

18. The method using the Resource Management and Reservation System defined in claim 17 wherein said registering the task with RMRS does not imply reservation of any resources since said resources are allocated and reserved for a task only when an instance of the task is scheduled.

19. The method using the Resource Management and Reservation System defined in claim 18 wherein a separate interface is provided to schedule one or more instances of a registered task and said separation of task definition/registration from scheduling of individual instances allows RMRS to incorporate dynamic changes associated with individual invocation of a task while keeping track of long-term requirements and goals.

20. The method using the Resource Management and Reservation System defied in claim 19 wherein said dynamic changes are brought about by unpredictable events, which last over a short term duration, whereas stable trends are set by long term and predictable repetitive events and RMRS provides interfaces to gather information on both short term and long term events and processes them accordingly.

21. The Method using the Resource Management and Reservation System defined in claim 20 wherein in the event a task is being scheduled for execution during a specified time window and with a specific instance priority, depending on task priority, RMRS optionally considers all the constraints specified when the task is defined.

22. The method using the Resource Management and Reservation System defined in claim 21 wherein when said priority is set as “

must complete, ”

RMRS ignores all constraints and schedules the task at a specified time as a highest priority task and if said priority is not set to be “

must complete, ”

then RMRS schedules the task instance by considering the task constraints and the task instance is scheduled optimally, close to a specified start time.

23. The method using the Resource Management arid Reservation System defined in claim 2 wherein RMRS interacts with ADRS-RMRS using an interaction flow comprising the following stops:

ADRS registers itself as a client of RMRS using the registerClient( ) API;

ADRS registers at least one hosted application with itself for backup/recovery wherein each application is associated with an Application Recovery Set (ARSet) comprising a set of application objects comprising Databases, Tablespaces, Files, which application objects need to be backed up and restored whenever required;

During said application registration phase, ADRS collects initial ARSet information comprising Data Size, Data Volatility, which determines optimal backup frequency, Constraint Windows, Technology to use for backup, for each object in the ARSet;

ADRS then invokes setTaskDefinition( ) API to register with RMRS multiple types of tasks, comprising backup and restore to be performed on said ARSet objects such that one task definition is registered per <

Object, TaskType>

tuple;

In the event setTaskDefinition( ) API returns failure, it is an unexpected system error and if said API returns success, but requests more resources, in such event, said ADRS selects from one of the three available choices;

provision additional resources, continue with existing resources, or call unSetTaskDefinition( ) API and not serve the application;

said selection being made based on the current policy applicable to said application;

When a task is successfully registered, RMRS returns a TaskDefintionID which ADRS stores during said lifetime of said task and uses said return to schedule individual instances of said task;

ADRS invokes generateTaskSchedule( ) API for the <

object, backup>

task that is defined using setTaskDefinition( ) API in previous setup, to generate the backup schedule for a meaningful period of time and RMRS returns a plan for the set of task instances detailing when each backup task is to be scheduled;

ADRS uses said plan created by RMRS to invoke the backup tasks at said scheduled time intervals;

just prior to invoking a backup task, ADRS does a final check with RMRS using the is TaskRunnable( ) API to check if the task can be invoked at this time;

if RMRS has rescheduled the task to a future time, then ADRS receives a new invocation time for that task and updates its internal event table accordingly;

when a backup task is in progress, statistics on several performance metrics are monitored and gathered on behalf of ADRS which invokes updateTaskDefinition( ) API at the end of a backup task if any of the task definition parameters are found to have changed;

if the change in task definition affects the current plan for the task schedule, RMRS first invalidates the existing backup plan for the task with the result that ADRS invalidates its internal plan and invokes generateTaskSchedule( ) API to rescheduled the task with updated task definition parameters;

when a backup task completes successfully, ADRS invokes the taskComplete( ) API to indicate to RNRS that task has completed successfully and informs RMRS the actual time taken to complete the backup task. RMRS has an internal estimate for the expected completion time for the task and compares the expected completion against the actual completion time;

in the event a task is completed successfully and ahead of expected completion time, RMRS unblocks resources reserved for said specific task and makes those available for allocation to other tasks wherein the blocking and unblocking within RMRS is a logic concept internal to RMRS and there is no blocking of the actual resource;

if a task ends prematurely for any reason, then ADRS indicates this to RMRS using the taskComplete( ) API and also indicates to RMRS whether the allocated resources should be released or continued to be allocated to the task;

if ADRS intends to rescheduled the task immediately, said resources are not released;

actual task completion time and type of completion (successful/abnormal) indicated through taskComplete( ) API along with expected task completion time helps RMRS learn the trend of task completion Lime over the period and helps it in better planning and reservation of future instances of the task;

when all said tasks in said plan have been scheduled or otherwise taken care of, ADRS invokes generateTaskSchedule( ) API again to generate another plan for next plan period;

for recovery, rescheduling of failed backup, hot backups tasks, ADRS uses the schedule Task( ) API to schedule said tasks;

in the event of a change in the attributes associated with a task definition, ADRS invokes updateTaskDefinition( ) API and said API updates said task definition values stored in the RMRS Repository whereby any change in attribute values can invalidate the existing task schedule and indicate that to ADRS;

a schedule is re-computed if requested by ADRS.

24. The Resource Management and Reservation System defined in claim 2 wherein said RMRS interfaces with said SAN Gateway via said SAN Interface component.

25. The Resource Management and Reservation System defined in claim 24 wherein said SAN Gateway manages and controls I/O streams associated with I/O requests.

26. The Resource Management and Reservation System defined in claim 25 wherein at a point in time when a backup or recovery task is initiated, two logical I/O stream connections are managed by said SAN Gateway.

27. The Resource Management and Reservation System defined in claim 26 wherein said two streams comprise an Application Stream Group.

28. The Resource Management and Reservation System defined in claim 27 wherein said Application Stream Group comprises an I/O stream between the host machine where the application associated with the backup/recovery running and logical volume where an application database object resides;

and between the host machine and logical volume where the backed-up data resides.

29. The Resource Management and Reservation System defined in claim 28 wherein for each backup/recovery task scheduled by RMRS, a Service Level specification is specified for the corresponding Application Stream Group managed by said SAN Gateway.

30. The Resource Management and Reservation System defined in claim 29 wherein said Service Level specification specifies the number of I/O operations per second (bandwidth in terms of Megabytes/sec);

the use of the desired response time for each I/O and the priority for the I/O operations, and SAN Gateway guarantees specified I/O response time as long as I/O operations per second are equal or below the specified value.

31. The Resource Management and Reservation System defined in claim 30 wherein when said system is configured, each ASG is assigned with the default bandwidth required for the regular OLTP workload and is assigned with a default priority based upon a customer/application priority that originates the workload.

32. The Resource Management and Reservation System defined in claim 31 wherein when said priority is raised or lowered dynamically by said RMRS when additional tasks are initiated on the ASG.

33. The Resource Management and Reservation System defined in claim 32 wherein when a backup task is scheduled on an ASG, a default ASG priority is lowered, and the bandwidth allocated to the ASG is raised as required by backup task along with OLTP workload

34. The Resource Management and Reservation System defined in claim 33 wherein when a recovery task is scheduled on an ASC, a default ASG priority is raised and the bandwidth allocated to the ASG is raised as required by said recovery task.

35. The Resource Management and Reservation System defined in claim 2 wherein said SAN Gateway monitor periodically monitor information from an ASG defined at said SAN Gateway.

36. The Resource Management and Reservation System defined in claim 35 wherein said monitored information is stored in a ResourceUsageMonitor table in a repository.

37. The Resource Management and Reservation System defiled in claim 34 wherein said SAN Gateway Controller comprises two subcomponents comprising an Event Generator and an Event Dispatcher.

38. The Resource Management and Reservation System defined in claim 36 wherein said Event generator scans TASKSCHEDULE and APPLICATIONRESOURCEUSAGE tables stored in an RMRS Repository.

39. The Resource Management and Reservation System defined in claim 37 wherein said TASKSCHEDULE table has information on currently planned schedules and said APPLICATIONRESOURCEUSAGE table stores the information about an OLTP workload.

40. The Resource Management and Reservation System defined in claim 38 wherein said SAN Gateway Controller Event Generator generates bandwidth allocation events whenever there is a change in the allocated bandwidth capacity arid/or the priority for any of said ASGs when a new task starts or ends on said ASC.

41. The Resource Management and Reservation System defined in claim 39 wherein said Event Dispatcher component processes events generated by Event Generator by invoking an controlASGResourceAllocation( ) API to apply the new SLS for said ASGs which is accomplished by sending a request to the SAN Gateway with revised bandwidth allocation requirements and priorities for said ASGs.

42. The Resource Management and Reservation System defined in claim 41 wherein said SAN Gateway QoS Analyzer analyzes a RESOURCEUSAGEMONITOR table along with TASKSCHEDULE table and provides hints to a Schedule Adjuster in said system for possible contentions with certain set of application tasks scheduled in the future.

43. The method using the Resource Management and Reservation System defined in claim 42 wherein said SAN Gateway Handler insures that tasks are completed on schedule using an interaction flow comprising the following steps:

ADRS provides an estimate of the average run time of a task based upon a statistical analysis of similar tasks run in the past;

using said estimate, RMRS calculates the normal bandwidth requirement for completing said task within said estimated time interval;

ADRS optionally provides a limit of a maximum expected run time;

such limit being driven by a service level agreements with a customer of said application wherein RMRS monitors the progress of the task to insure that the task completes at the expected time;

said completion being accomplished in two stages comprising;

whenever a task begins to execute, said Event Generator generates an end of the task event as per scheduled end time for the task;

event Dispatcher checks to see if the corresponding task has completed by checking to see if a notification has been received from ADRS (through the taskComplete( ) API);

if said task is not completed, then said event Dispatcher looks in ResourceUsageMonitor table to get an estimate on how far the corresponding ASG is lagging in meeting the specified SLS;

from this result said event Dispatcher estimates how much more time the task may need to complete and then sends a request to RMRS scheduler to immediately extend the current task for that time period or

till the hard limit of task finish time, which ever is earlier;

said extension of task is similar to a request for scheduling an unplanned task;

the priority of said task that is being delayed typically remains unaltered;

however, if a hard limit seems unlikely to be met, then Event Dispatcher optionally requests allocation of additional bandwidth for said task.

44. The Resource Management and Reservation System defined in claim 2 wherein said RMRS-SAN interface has components comprising:

registerWithResourceManager( ) for Registration whereby RMRS registers itself as a client with SAN Gateway;

configureApplicationsStreamsConnections( ) for configuration whereby RMRS configures said I/O streams (ASGs) with SAN Gateway for different applications.

controlASGResourceAllocation( ) for dynamic manipulation of ASGs wherein through said interface, RMRS dynamically changes the resource Allocation and Priorities of a specified ASG;

monitorASG( ) for monitoring an ASG at regular interval to check whether ASG meets the specified requirement.

45. The method using the Resource Management and Reservation System defined in claim 42 wherein when a set of task instances need to be scheduled and bandwidth resource needs to be allocated and reserved, said Schedule Optimizer component and/or said Scheduler Adjuster component are invoked by use of active components in said RMRS Repository in response to invocations from ADRS.

46. The method using the Resource Management and Reservation System defined in claim 45 wherein objectives for said Schedule Optimizer arc set by policies or objectives set by the Service Provider wherein said policy schedules all I/O tasks corresponding to a set of customers within specified time window.

47. The method using the Resource Management and Reservation System defined in claim 46 wherein said scheduler includes in its objective function:

1. Minimizing the average deviation from the periodic expected schedule times for all the task instances belonging to applications for a class of customers, 2. Avoiding stacking up of multiple tasks over each other as far as possible, considering and using the following criteria;

the total available bandwidth, latest time when a task could be scheduled, business policies such as when a type of task can be performed.

48. The method using the Resource Management and Reservation System defined in claim 47 wherein said scheduler formulates the scheduling problem as optimization problem and use quantization based optimization approach in the Schedule Optimizer to linearize the optimization problem.

49. The method using the Resource Management and Reservation System defined in claim 48 where Scheduler Optimizer uses the approach for modeling the objective cost function using multiple band model of the network bandwidth in order to spread out use of network capacity in time and to maximize spare network capacity at any time in order to accommodate unplanned recovery tasks, other contingencies.

50. The method using the Resource Management and Reservation System defined in claim 47 wherein said Scheduler Adjuster is triggered whenever a scheduled task instance causes the bandwidth usage to exceed the available or allocated bandwidth, this fact is memorized and the Adjuster is triggered to perform priority based adjustment in the schedules over or around the period of excess usage.

51. The method using the Resource Management and Reservation System defined in claim 48 wherein said Schedule Adjuster lowers the excess usage as much possible using heuristic rules.

52. The method using the Resource Management and Reservation System defined in claim 48 can be used for reservation, planning and scheduling of not only data protection data movement tasks like backup, recovery, replication but all predictable arid unpredictable I/O intensive tasks initiated by applications or by any data management agents on behalf of the applications.

53. The method using the Resource Management and Reservation System defined in claim 48 which supports more granular differentiation at the level of individual application tasks by considering task priorities along with the application/customer priority.

54. A system for managing and controlling storage access requirements comprising articles of manufacture which comprise computer-usable media having computer-readable program code means embodied therein for enabling said system to manage and control storage access requirements comprising:

said computer readable program code means in a first article of manufacture comprising means to allow an ADRS to register itself as a client of RMRS using the registerClient( ) API;

said computer readable program code means in a second article of manufacture comprising means to allow ADRS to register at least one hosted application with itself for backup/recovery wherein each application is associated with an Application Recovery Set (ARSet) comprising a set of application objects comprising Databases, Tablespaces, Files, which application objects need to be backed up and restored whenever required;

said computer readable program code means in a third article of manufacture comprising means to allow during said application registration phase, said ADRS to collect initial ARSet information comprising Data Size, Data Volatility, which determines optimal backup frequency, Constraint Windows, Technology to use for backup, for each object in the ARSet;

ADRS then invokes setTaskDefinition( ) API to register with RMRS multiple types of tasks, comprising backup and restore to be performed on said ARSet objects such that one task definition is registered per <

Object, TaskType>

tuple,

In the event setTaskDefinition( ) API returns failure, it is an unexpected system error and if said API returns success, but requests more resources, in such event, said ADRS selects from one of the three available choices;

provision additional resources, continue with existing resources, or call unSetTaskDefinition( ) API and not serve the application;

said selection being made based on the current policy applicable to said application;

said computer readable program code means in a fourth article of manufacture comprising means to allow when a task is successfully registered, RMRS to returns a TaskDefinitionID which ADRS stores during said lifetime of said task and uses said return lo schedule individual instances of said task;

and said ADRS invokes generateTaskSchedule( ) API for the <

object, backup>

task that is defined using setTaskDefinition( ) API in previous setup, to generate the backup schedule for a meaningful period of time and RMRS returns a plan for the set of task instances detailing when each backup task is to be scheduled;

said computer readable program code means in a fifth article of manufacture comprising means to allow ADRS to use said plan created by RMRS to invoke the backup tasks at said scheduled time intervals;

and just prior to invoking a backup task, said ADRS does a final check with RMRS using the is TaskRunnable( ) API to check if the task can be invoked at this time;

said computer readable program code means in a sixth article of manufacture comprising means to calculate if RMRS has rescheduled the task to a future time, then to enable ADRS to receive a new invocation time for that task and updates its internal event table accordingly;

and when a backup task is in progress, statistics on several performance metrics are monitored and gathered on behalf of ADRS which invokes updateTaskDefinition( ) API at the end of a backup task if any of the task definition parameters are found to have changed;

said computer readable program code means in a seventh article of manufacture comprising means to determine if said change in task definition affects the current plan for the task schedule, then to enable said RMRS to first invalidate the existing backup plan for the task with the result that ADRS invalidates its internal plan and invokes generateTaskSchedule( ) API to rescheduled the task with updated task definition parameters;

and when a backup task completes successfully, said ADRS invokes the taskComplete( ) API to indicate to said RMRS that task has completed successfully and informs RMRS the actual time taken to complete the backup task. RMRS has an internal estimate for the expected completion time for the task and compares the expected completion against the actual completion time;

said computer readable program code means in an eighth article of manufacture comprising means to determine in the event a task is completed successfully and ahead of expected completion time, to enable RMRS to unblock resources reserved for said specific task and make those resources available for allocation to other tasks wherein the blocking and unblocking within RMRS is a logic concept internal to RMRS and there is no blocking of the actual resource;

and if a task ends prematurely for any reason, then ADRS indicates this to RMRS using the taskComplete( ) API and also indicates to RMRS whether the allocated resources should be released or continued to be allocated to the task;

said computer readable program code means in a ninth article of manufacture comprising means to determine if said ADRS intends to rescheduled said task immediately, and not release said resources; and

the actual task completion time and type completion (successful/abnormal) is indicated through taskComplete( ) API along with expected task completion time to help RMRS learn the trend of task completion time over the period and help in better planning and reservation of future instances of the task;

said computer readable program code means in a tenth article of manufacture comprising means to determine when all said tasks in said plan have been scheduled or otherwise taken care of, then top activate said ADRS to invoke generateTaskSchedule( ) API again to generate another plan for next plan period;

and for recovery, rescheduling of failed backup, hot backups tasks, ADRS uses the scheduleTask( ) API to schedule said tasks;

wherein in the event of a change in, the attributes associated with said task definition, ADRS invokes updateTaskDefinition( ) API and said API updates said task definition values stored in the RMRS Repository whereby any change in attribute values can invalidate the existing task schedule and indicate that to ADRS;

said computer readable program code means in an eleventh article of manufacture comprising means to re-compute a schedule if requested by said ADRS.