METHODS, APPARATUS AND COMPUTER PROGRAMS FOR MANAGING PERSISTENCE

• US 20200133750A1
• Filed: 11/20/2019
• Published: 04/30/2020
• Est. Priority Date: 07/01/2006
• Status: Active Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Todd, Stephen J.
• Application Number:
  US16/689,082
• Publication Number:
  US 20200133750A1
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/008 Reliability or availability analysis
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0629 Configuration or reconfiguration of storage systems
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: G06F 9/546 Message passing systems or structures, e.g. queues
• CPC Class Codes: G06Q 10/0635 Risk analysis of enterprise or organisation activities

Abstract

Described are methods, apparatus and computer programs for managing persistence within a data processing system such as a messaging system, database system or file system. The method for managing persistence includes a deferred evaluation (230,330,430) of at least one criterion associated with costs and/or benefits of saving to persistent storage, instead of persistence behaviour being fully predefined. The evaluation may be performed when a disk write is about to be performed (240,340,440), or at various times during processing of a data update and at various points within a data processing network. In a messaging solution, the method for managing persistence includes a dynamic evaluation (230,330,430) of costs and/or benefits of saving to persistent storage, with the evaluation being performed at various points in the messaging network after the message has been created and sent by the originating entity. The method includes determining whether message data and/or log records relating to the message require saving to persistent storage, in accordance with the costs and/or the benefits of saving to persistent storage. The benefits of persisting may be determined with reference to a message value (400).

Claims

1-7. -7. (canceled)

8. A data processing apparatus comprising:

a processor;

a volatile data store;

a non-volatile data store;

a persistence manager for managing saving of data to the non-volatile data store and executing by the processor; and

,

a messaging manager also executing by the processor;

wherein the persistence manager during execution performs;

evaluating at least one criterion of a set of criteria, the set of criteria representing costs of saving and risks associated with not saving data to persistent storage, to determine whether to save a data update to persistent storage, the evaluating comprising valuating a risk of loss of data relating to a message and evaluating a potential cost of said loss based on a value of the message; and

in response to a determination that the data update should be saved to persistent storage, initiating, by the persistence manager, a save of the data update to persistent storage;

in response to determining that the data update should not be saved to the persistent storage, setting, by the persistence manager, a timer for a next evaluation;

determining, by the persistence manager, whether to save the data update to the persistent storage based upon evaluating the at least one criterion of the set of criteria at an end of a set time period resulting from an expiration of the timer; and

,

saving the data update to the persistent storage upon a determination to save the data update to the persistent storage.

9. A data processing apparatus according to claim 8, wherein the evaluation is performed in response to a message attribute indicating that the message is a candidate for saving to the non-volatile data store.

10. A data processing apparatus according to claim 9, including a messaging API providing message persistence options including at least one message attribute value selected from the group consisting of:

persistent;

non-persistent;

persistence determined by reference to message queue definition; and

persistence determined by reference to message value.

11. A data processing apparatus according to claim 10, wherein the messaging API provides means for specifying a message value and provides at least the message persistence option persistence determined by reference to message value.

12. A data processing apparatus according to claim 8, wherein the persistence manager'"'"'s function for evaluating comprises means for evaluating a risk of loss of data if data relating to a message is not saved to the non-volatile data store, and means for evaluating a cost associated with said loss based on the value of the message.

13. A data processing apparatus according to claim 12 comprising a plurality of data processing systems connected for communication via a network, each data processing system comprising:

a processor, a volatile data store, a non-volatile data store, a messaging manager and an associated persistence manager for determining whether to save message related data to the respective non-volatile data store;

wherein a first messaging manager at a first data processing system comprises means for determining a message value for use by a first persistence manager at the first data processing system, and means for sending the determined message value to a second messaging manager at a second data processing system; and

wherein the second messaging manager comprises means for identifying the message value and providing the identified message value to a second persistence manager at the second data processing system, to enable a determination by the second persistence manager of whether to save message related data to a non-volatile data store at the second data processing system.

14. A data processing apparatus according to claim 8, wherein the persistence manager'"'"'s function for evaluating comprises means for evaluating the benefits of saving data relating to the message to the non-volatile data store based on a calculated risk of losing said data relating to the message.

15. A data processing apparatus according to claim 8, wherein the persistence manager is responsive to a message queue reaching a threshold queue depth, to initiate the functions for evaluating, determining and initiating a save to the non-volatile data store.

16. A data processing apparatus according to claim 8, wherein the persistence manager is responsive to the messaging manager performing an operation in relation to the message, to initiate the functions for evaluating, determining and initiating a save to the non-volatile data store.

17. A data processing apparatus according to claim 16, wherein the operation comprises placing the message on a queue managed by the messaging manager.

18. (canceled)

19. A method for managing persistence of data within a data processing apparatus, comprising the steps of:

evaluating, by a persistence manager executing in a memory of a computer, at least one criterion of a set of criteria, the set of criteria representing costs of saving and risks associated with not saving data to persistent storage, to determine whether to save a data update to persistent storage, the evaluating comprising valuating a risk of loss of data relating to a message and evaluating a potential cost of said loss based on a value of the message; and

in response to a determination that the data update should be saved to persistent storage, initiating, by the persistence manager, a save of the data update to persistent storage;

in response to determining that the data update should not be saved to the persistent storage, setting, by the persistence manager, a timer for a next evaluation;

determining, by the persistence manager, whether to save the data update to the persistent storage based upon evaluating the at least one criterion of the set of criteria at an end of a set time period resulting from an expiration of the timer; and

,

saving the data update to the persistent storage upon a determination to save the data update to the persistent storage.

20. The method of claim 19, for use in a messaging system, wherein the step of evaluating at least one criterion comprises evaluating a risk of loss of data relating to a message and evaluating a potential cost of said loss based on a value of the message.

21. The method of claim 20, wherein the value of the message is determined from a monetary value within the message content.

22. The method of claim 20, wherein the value of the message is inferred from attributes of a message queue to which the message is sent.

23. The method of claim 20, wherein the value of the message is an attribute of the message specified via a messaging API.

24. The method of claim 19, for use in a messaging system, wherein the step of evaluating at least one criterion comprises determining the current size of a queue of persistent messages.

25. The method of claim 19, for use in a messaging system, wherein the step of evaluating is performed in response to performance of an operation in relation to a message, to determine whether data relating to the performed operation requires saving to persistent storage.

26. The method of claim 19, for use in a messaging system, wherein the step of evaluating comprises comparing the time period for which a message has been enqueued at the messaging system with a threshold time period.

27. The method of claim 19, for use in a messaging system, wherein the step of evaluating is performed in response to identification of a message attribute indicating that the message is a candidate for saving to persistent storage.

28. The method of claim 19, wherein the step of evaluating is performed by a persistence manager that comprises:

a data collector component; and

an evaluator component for applying a stored set of rules to data collected by the data collector component, to determine whether to save a data update to persistent storage.

29. (canceled)