METHODS, APPARATUS AND COMPUTER PROGRAMS FOR MANAGING PERSISTENCE

• US 20100017441A1
• Filed: 06/26/2007
• Published: 01/21/2010
• Est. Priority Date: 07/01/2006
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Todd, Stephen J.
• Granted Patent: US 9,495,229 B2
• Time in Patent Office: Days
• US Class Current: N/A
• CPC Class Codes: G06F 11/008 Reliability or availability analysis
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0629 Configuration or reconfiguration of storage systems
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: G06F 9/546 Message passing systems or structures, e.g. queues
• CPC Class Codes: G06Q 10/0635 Risk analysis of enterprise or organisation activities

Abstract

Described are methods, apparatus and computer programs for managing persistence within a data processing system such as a messaging system, database system or file system. The method for managing persistence includes a deferred evaluation (230,330,430) of at least one criterion associated with costs and/or benefits of saving to persistent storage, instead of persistence behaviour being fully predefined. The evaluation may be performed when a disk write is about to be performed (240,340,440), or at various times during processing of a data update and at various points within a data processing network. In a messaging solution, the method for managing persistence includes a dynamic evaluation (230,330,430) of costs and/or benefits of saving to persistent storage, with the evaluation being performed at various points in the messaging network after the message has been created and sent by the originating entity. The method includes determining whether message data and/or log records relating to the message require saving to persistent storage, in accordance with the costs and/or the benefits of saving to persistent storage. The benefits of persisting may be determined with reference to a message value (400).

Claims

1. A data processing apparatus comprising:

a processor;

a volatile data store;

a non-volatile data store; and

a persistence manager for managing saving of data to the non-volatile data store, wherein the persistence manager includes means for evaluating at least one criterion of a set of criteria, the set of criteria representing costs of saving and risks associated with not saving data to the non-volatile data store, to determine whether a data update held in the volatile data store requires saving to the non-volatile data store.

2. A data processing apparatus according to claim 1, wherein the means for evaluating comprises means for evaluating at least a first criterion representing a cost of saving data to the non-volatile data store, and a second criterion representing a risk associated with not saving data to the non-volatile data store;

and means for comparing the cost and risk to make said determination.

3. A data processing apparatus according to claim 1, wherein the persistence manager includes a function for evaluating a risk of loss of data in the absence of a save of the data update to the non-volatile data store.

4. A data processing apparatus according to claim 3, wherein the persistence manager includes a function for computing a potential cost of loss of data based on a value of the data update and said risk of loss of data.

5. A data processing apparatus according to claim 1, wherein the persistence manager includes a function for evaluating a risk of duplicating a data update in the absence of a save of the data update to the non-volatile data store.

6. A data processing apparatus according to claim 1, wherein the persistence manager includes means for evaluating a performance characteristic of the data processing apparatus.

7. A data processing apparatus according to claim 1, wherein the persistence manager includes a function for evaluating the time period remaining before log records will be written to the non-volatile data store.

8. A data processing apparatus according to claim 1, further comprising a messaging manager, wherein the persistence manager includes functions for:

evaluating at least one criterion of a set of criteria, the set of criteria representing costs of saving and risks associated with not saving data relating to a message to the non-volatile data store, during handling of the message by the messaging manager;

determining from said evaluation whether to save the data relating to the message to the non-volatile data store; and

in response to a positive determination, initiating a save of the data relating to the message to the non-volatile data store.

9. A data processing apparatus according to claim 8, wherein the evaluation is performed in response to a message attribute indicating that the message is a candidate for saving to the non-volatile data store.

10. A data processing apparatus according to claim 9, including a messaging API providing message persistence options including at least one message attribute value selected from the group consisting of:

persistent;

non-persistent;

persistence determined by reference to message queue definition; and

persistence determined by reference to message value.

11. A data processing apparatus according to claim 10, wherein the messaging API provides means for specifying a message value and provides at least the message persistence option ‘

persistence determined by reference to message value’

.

12. A data processing apparatus according to claim 8, wherein the persistence manager'"'"'s function for evaluating comprises means for evaluating a risk of loss of data if data relating to a message is not saved to the non-volatile data store, and means for evaluating a cost associated with said loss based on the value of the message.

13. A data processing apparatus according to claim 12 comprising a plurality of data processing systems connected for communication via a network, each data processing system comprising:

a processor, a volatile data store, a non-volatile data store, a messaging manager and an associated persistence manager for determining whether to save message related data to the respective non-volatile data store;

wherein a first messaging manager at a first data processing system comprises means for determining a message value for use by a first persistence manager at the first data processing system, and means for sending the determined message value to a second messaging manager at a second data processing system; and

wherein the second messaging manager comprises means for identifying the message value and providing the identified message value to a second persistence manager at the second data processing system, to enable a determination by the second persistence manager of whether to save message related data to a non-volatile data store at the second data processing system.

14. A data processing apparatus according to claim 8, wherein the persistence manager'"'"'s function for evaluating comprises means for evaluating the benefits of saving data relating to the message to the non-volatile data store based on a calculated risk of losing said data relating to the message.

15. A data processing apparatus according to claim 8, wherein the persistence manager is responsive to a message queue reaching a threshold queue depth, to initiate the functions for evaluating, determining and initiating a save to the non-volatile data store.

16. A data processing apparatus according to claim 8, wherein the persistence manager is responsive to the messaging manager performing an operation in relation to the message, to initiate the functions for evaluating, determining and initiating a save to the non-volatile data store.

17. A data processing apparatus according to claim 16, wherein the operation comprises placing the message on a queue managed by the messaging manager.

18. A data processing apparatus according to claim 1, further comprising a database manager for applying data updates to database tables held in the volatile data store, wherein the persistence manager implements functions for determining whether data updates, additions and deletions require saving to the non-volatile data store based on said evaluating of at least one criterion.

19. A method for managing persistence of data within a data processing apparatus, comprising the steps of:

evaluating at least one criterion of a set of criteria, the set of criteria representing costs of saving and risks associated with not saving data to persistent storage, to determine whether to save a data update to persistent storage; and

in response to a determination that the data update should be saved to persistent storage, initiating a save of the data update to persistent storage.

20. The method of claim 19, for use in a messaging system, wherein the step of evaluating at least one criterion comprises evaluating a risk of loss of data relating to a message and evaluating a potential cost of said loss based on a value of the message.

21. The method of claim 20, wherein the value of the message is determined from a monetary value within the message content.

22. The method of claim 20, wherein the value of the message is inferred from attributes of a message queue to which the message is sent.

23. The method of claim 20, wherein the value of the message is an attribute of the message specified via a messaging API.

24. The method of claim 19, for use in a messaging system, wherein the step of evaluating at least one criterion comprises determining the current size of a queue of persistent messages.

25. The method of claim 19, for use in a messaging system, wherein the step of evaluating is performed in response to performance of an operation in relation to a message, to determine whether data relating to the performed operation requires saving to persistent storage.

26. The method of claim 19, for use in a messaging system, wherein the step of evaluating comprises comparing the time period for which a message has been enqueued at the messaging system with a threshold time period.

27. The method of claim 19, for use in a messaging system, wherein the step of evaluating is performed in response to identification of a message attribute indicating that the message is a candidate for saving to persistent storage.

28. The method of claim 19, wherein the step of evaluating is performed by a persistence manager that comprises:

a data collector component; and

an evaluator component for applying a stored set of rules to data collected by the data collector component, to determine whether to save a data update to persistent storage.

29. A computer program comprising a set of instructions implemented in program code for controlling performance of a data processing apparatus on which the computer program executes, to perform a method comprising the steps of:

evaluating at least one criterion of a set of criteria the set of criteria representing costs of saving and risks associated with not saving data to persistent storage, to determine whether to save a data update to persistent storage; and

in response to a determination that the data update should be saved to persistent storage, initiating a save of the data update to persistent storage.