Adaptive transaction manager for complex transactions and business process

• US 20040068501A1
• Filed: 10/03/2002
• Published: 04/08/2004
• Est. Priority Date: 10/03/2002
• Status: Active Grant

• Current Assignee: David O. Mcgoveran
• Original Assignee: David O. Mcgoveran
• Inventors: McGoveran, David O.
• Granted Patent: US 7,103,597 B2
• Time in Patent Office: Days
• US Class Current: 707/8
• CPC Class Codes: G06F 11/1474 in transactions G06F16/20 takes precedence
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency
• CPC Class Codes: G06Q 20/10 specially adapted for electronic funds transfer [EFT] systems; specially adapted for home banking systems
• CPC Class Codes: G06Q 30/06 Buying, selling or leasing transactions
• CPC Class Codes: Y10S 707/99938 Concurrency, e.g. lock management in shared database
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

This is a method for managing and optimizing transaction processing that: (1) significantly expands the robustness of systems with respect to consistency, reliability, and recoverability; (2) provides multiple performance improvements over traditional methods; (3) enables transactions in a distributed business process to recover from errors without the cost of two-phase commit; (4) enables enhancements to resource management; and, (5) provides improved scheduling. The method further enables a high degree of collaborative and adaptive transaction management over traditional transaction managers.

Claims

1. A method for efficient transaction processing implemented as an Adaptive Transaction Manager (‘

ATM’

), said method comprising;

determining that a transaction state is consistent with a set of constraints; and

,

explicitly denoting that transaction state as a consistency point.

2. A method as in claim 1, wherein the step of determining that a transaction state is consistent with a set of constraints further comprises:

accepting an assertion that a transaction state is consistent with a set of constraints.

3. A method as in claim 2, wherein the assertion that a transaction state is consistent with a set of constraints is made by a human user.

4. A method as in claim 1, wherein the step of determining that a transaction state is consistent with a set of constraints further comprises:

establishing that consistency by validating with regards to the set of constraints.

5. A method as in claim 4, wherein the ATM automatically establishes that consistency.

6. A method as in claim 1, wherein the step of explicitly denoting that transaction state as a consistency point is made by a human user.

7. A method as in claim 1, further comprising:

recording the denotation of a transaction state as a consistency point.

8. A method as in claim 1, further comprising:

using a consistency point to recover from undesirable conditions.

9. A method as in claim 8, further comprising:

using a consistency point for automatic deadlock recovery.

10. A method as in claim 1, further comprising:

using a consistency point to automatically assert a savepoint.

11. A method as in claim 1, further comprising:

naming consistency points.

12. A method as in claim 11, wherein the naming is automatic according to a predefined naming convention.

13. A method as in claim 12, wherein the naming is by manual assertion by a human.

14. A method as in claim 1, further comprising:

using durable consistency points to optimize commit processing.

15. A method as in claim 1, further comprising:

distinguishing consistency points from synchronization points.

16. A method as in claim 1, further comprising:

transitioning the state of the software system to a consistency point on detection of a specified condition.

17. A method as in claim 1, further comprising:

creating, detecting, and denoting categories of consistency points.

18. A method as in claim 17, further comprising:

associating classes of errors with categories of consistency points.

19. A method as in claim 17, further comprising:

transitioning the state of the software system to a state belonging to a category of consistency point on detection of a condition.

20. A method as in claim 1, implementing transaction relaying, further comprising:

determining that a first transaction and a second transaction share a resource set;

establishing control over the resource set by the first transaction, such that any predefined isolation level for that first transaction is maintained; and

,

transferring at a consistency point of the resource set control over the resource set from the first transaction to the second transaction, without first requiring the first transaction to commit.

21. A method as in claim 20, wherein the control is modification authority.

22. A method as in claim 20, wherein the control is priority.

23. A method as in claim 20, wherein the first transaction belongs to a first transaction group, and the second transaction belongs to a second transaction group, with at least one transaction from each said group sharing the resource set, further comprising:

transferring at a consistency point of the resource set control over the resource set from the first transaction group to the second transaction group, without first requiring the first transaction group to commit.

24. A method as in claim 20, wherein the first transaction and second transaction belong to at least one group of transactions sharing the resource set, further comprising:

maintaining two phase locking for the resource set as if the group of transactions were part of a single transaction.

25. A method as in claim 20, wherein the first transaction and second transaction belong to at least one group of transactions sharing the resource set, further comprising:

applying lock promotion and lock demotion across the group of transactions as if they were part of a single transaction.

26. A method as in claim 20, wherein the first transaction and second transaction belong to at least one group of transactions sharing the resource set, further comprising:

applying lock scope escalation across the group of transactions as if they were part of a single transaction.

27. A method as in claim 20, wherein the first transaction and second transaction belong to at least one group of transactions sharing the resource set, further comprising:

detecting and removing redundant instructions from individual transactions involved in transaction relaying.

28. A method as in claim 20, wherein the first transaction and second transaction belong to at least one group of transactions sharing the resource set, further comprising:

automatically removing instructions from a first transaction and inserting them in a second transaction when transaction relaying spans said first and second transactions.

29. A method as in claim 20, wherein the first transaction and second transaction belong to at least one shared group of transactions sharing the resources set, further comprising:

publishing of and subscribing to intermediate transaction states by transactions.

36. A method as in claim 20, wherein the first transaction belongs to a first group of transactions, and the second transaction belongs to a second group of transactions, further comprising:

treating each of the first and second groups of transactions as if they were a first and second transaction, for the purposes of transaction relaying.

37. A method as in claim 1, implementing corrective transactions, further comprising:

detecting an exceptional condition during execution of a first transaction;

restoring the first transaction to a first consistency point; and

, executing a corrective transaction to reach a second consistency point.

38. A method as in claim 37, wherein the first consistency point and second consistency point are one and the same.

39. A method as in claim 37, further comprising, before executing a corrective transaction:

constructing the corrective transaction from a set of partially-ordered interdependent transactions.

40. A method as in claim 37, further comprising, before executing a corrective transaction:

identifying the corrective transaction according to the consistency condition it will produce.

41. A method as in claim 37, further comprising, before executing a corrective transaction:

selecting a corrective transaction.

42. A method as in claim 41, wherein selecting a corrective transaction further comprises:

identifying the acceptable sets of consistency conditions of the first transaction;

identifying at least an end state belonging to said acceptable sets; and

, invoking the selected corrective transaction to reach an end state which belongs to the acceptable sets of consistency conditions of said first transaction.

43. A method as in claim 41, wherein selecting a corrective transaction is done according to optimization criteria.

44. A method as in claim 41, wherein selecting a corrective transaction is done according to user preferences.

45. A method as in claim 41, wherein selecting a corrective transaction is done using a theorem prover.

46. A method as in claim 41, wherein selecting a corrective transaction is done using rule-based inference.

47. A method as in claim 41, wherein selecting a corrective transaction is done using back-chaining.

48. A method as in claim 41, wherein selecting a corrective transaction is done using manual input.

49. A method as in claim 41, wherein selecting a corrective transaction is done using optimization techniques and specific optimization goals.

50. A method as in claim 41, wherein selecting a corrective transaction is done using policy-driven programming implementing pre-set business rules to derive said corrective transaction.

51. A method as in claim 41, wherein the step of executing a corrective transaction is not done when the anticipated cost-benefit of other recovery methods justifies their use as an alternative.

52. A method as in claim 41, wherein selecting a corrective transaction is done using goal-oriented programming to derive said corrective transaction.

53. A method as in claim 41, wherein selecting a corrective transaction is done using genetic algorithms to derive said corrective transaction.

54. A method as in claim 41, wherein selecting a corrective transaction is done using back propagation to derive said corrective transaction.

55. A method as in claim 41, wherein selecting a corrective transaction is done using algebraic algorithms to calculate the least-cost, highest-benefit corrective transaction.

56. A method as in claim 37, further comprising:

establishing for each exceptional condition one or more classes;

establishing for each class one or more associated consistency categories; and

, requiring that the first consistency point belong to the associated category for the class of the exceptional condition.

57. A method as in claim 56, for handling a business process, further comprising, before executing a corrective transaction:

identifying that intended consistency point belonging to a consistency category which would have been reached by the first transaction but for the exceptional condition; and

, selecting a corrective transaction that transitions from the first consistency point to any second consistency point belonging to the consistency category of the intended consistency point.

58. A method as in claim 57, further comprising:

constructing a corrective transaction from a set of partially-ordered and interdependent transactions.

59. A method as in claim 57, further comprising:

using an alternative resource set than that used by the first transaction.

60. A method as in claim 57, for handling a business process, further comprising, before executing a corrective transaction and on detection of an exceptional condition wherein said exceptional condition is classified:

identifying the class of the exceptional condition;

retrieving the associated consistency category;

restoring a first consistency point of the said category;

looking up the consistency category for the intended consistency point, identifying a corrective transaction having an end state in same class of consistency conditions; and

,

selecting that corrective transaction.

61. A method as in claim 57, further comprising storing an association of classes and consistency categories.

62. A method as in claim 57, further comprising storing an association of transactions and classes.

63. A method as in claim 57, further comprising storing an association of exceptional conditions and classes.

64. A method as in claim 57, further comprising storing an association of transactions and the consistency conditions of each intended transaction end state.

65. A method as in claim 60, further comprising:

determining a minimally acceptable consistent state;

comparing at least one alternative end state resulting from at least one potential corrective transaction; and

,

selecting that corrective transaction whose alternative end state is preferred over the minimally acceptable consistent state.

66. A method as in claim 65, further comprising, after executing the corrective transaction:

reporting the minimally consistent state and the state resulting from the corrective transaction; and

,

subsequently implementing at least one manually defined step in conjunction with the corrective transaction.

67. A method as in claim 1, implementing dependency-based concurrency optimization, further comprising:

identifying resources that should be favored for subsequent processing of a transaction;

identifying a step in the transaction favoring is to be increased;

identifying a step in the transaction favoring is to be reduced; and

,

re-allocating identified resources at consistency points.

68. A method as in claim 67, further comprising:

including in a transaction definition;

identification of the data resources that should be favored for cache retention;

identification of at what step to begin such favoring; and

,

identification of at what step to reduce, possibly removing, such favoring.

69. A method as in claim 67, further comprising:

using automated identification and effectively combining it with the transaction definition.

70. A method as in claim 67, further comprising:

optimizing processing by pre-allocating resources.

71. A method as in claim 67, further comprising:

optimizing processing by pre-fetching data and placing said data in cache in anticipation of its need.

72. A method as in claim 67, further comprising:

rewriting the order of execution of transaction steps for at least one transaction in such a manner that the intended meaning of the transaction is not altered, to better coordinate favoring.

73. A method as in claim 67, further comprising:

rewriting the order of execution of transaction steps such that said transaction is optimized.

74. A method as in claim 67, further comprising:

storing in advance the transaction definitions necessary for an application.

75. A method as in claim 67, further comprising:

storing in advance the identification of all resources necessary to perform the steps of a transaction.

76. A method as in claim 67, further comprising:

storing in advance relative times associated with the use of resources.

77. A method as in claim 1 implementing dependency-based optimization for efficient transaction processing, comprising:

determining the dependencies among steps in a transaction;

identifying any group of one or more consecutive steps that have no dependencies outside the group wherein both the initial and final states of the group are consistency points; and

,

denoting that group as a consistent group.

78. A method as in claim 77, further comprising:

augmenting the definition of a transaction with at least one explicit statement as to at least one dependency among the steps of the transaction.

79. A method as in claim 77, further comprising:

augmenting the definitions of a group of transactions with dependencies among the steps of said transactions.

80. A method as in claim 77, further comprising:

maintaining said dependencies in a dependency tree.

81. A method as in claim 77, further comprising:

using said dependencies to determine which steps can be performed using a form of concurrent processing, including but not limiting such to parallel processing and asynchronous processing.

82. A method as in claim 77, further comprising:

using said dependencies to determine which groups of steps can be performed using a form of concurrent processing, including but not limiting such to parallel processing and asynchronous processing.

83. A method as in claim 77, further comprising:

executing any combination of consistent groups and mutually-independent transactions concurrently even though the members of said combination have not been submitted for concurrent execution.

84. A method as in claim 77, wherein the ATM selectively does not use any default concurrency control mechanism to enforce isolation among mutually-independent consistent groups and transactions.

85. A method as in claim 84, wherein the ATM does not use pessimistic concurrency control to enforce isolation among mutually-independent consistent groups and transactions.

86. A method as in claim 84, wherein the ATM does not use conflict detection mechanisms to enforce isolation among mutually-independent consistent groups and transactions.

87. A method as in claim 77, further comprising:

assigning consistency groups and transactions to conflict classes and scheduling said consistency groups and transactions based on said conflict classes.

88. A method as in claim 77, further comprising:

assigning transaction steps to conflict classes and scheduling said transaction steps based on said conflict classes.

89. A method as in claim 77, further comprising:

using said dependencies to determine which groups of steps can be reordered without changing the meaning of the transaction;

determining an optimized order of said groups of steps; and

,

executing the steps of the transaction in said optimized order.

30. A method as in claim 30, further comprising:

publishing states and other consistency conditions of a transaction at consistency points.

31. A method as in claim 30, further comprising:

enabling transactions to subscribe to at least one published consistency condition.

32. A method as in claim 30, further comprising:

determining, for multiple transactions that subscribe to a particular state of a particular resource set, which transaction shall have write permission for that resource.

33. A method as in claim 30, further comprising:

using a locking flag to denote dependencies across transactions on any particular resource in a transaction.

34. A method as in claim 34, further comprising:

transferring the control and responsibility over said flag to the transaction that has not yet attained a consistent state with said resource.

35. A method as in claim 34, further comprising:

permitting transactions that no longer have control over said flag to reach an acceptable state, including completion.

90. A method for efficient transaction processing and error management implemented as an Adaptive Transaction Manager (‘

ATM’

), the method being extensible to multiple business entities (related or independent) and extensible to complex transactions, said method comprising;

a coordinated set of sub-methods, extensible to and instantiable upon a distributed network of computers, each particular sub-method and any set thereof also being usable by either a unitary database management system or a distributed database management system, said sub-methods comprising steps for;

implementing consistency points;

implementing transaction relaying;

implementing corrective transactions;

implementing lookahead-based resource management; and

,

implementing dependency-based concurrency optimization.

91. A general-purpose computer incorporating specific hardware and software for manipulating at least one database when processing at least one transaction, wherein said specific hardware and software comprise:

means for implementing consistency points;

means for implementing transaction relaying;

means for implementing corrective transactions;

means for implementing lookahead-based resource management; and

,

means for implementing dependency-based concurrency optimization.

92. A general-purpose computer that includes software, dynamic and stable memory, and logical processing hardware, programmed for manipulating at least one database when processing at least one transaction and manipulating steps in at least one transaction, comprising:

means for manipulating the software, logical processing hardware, and dynamic and stable memory, to designate a set of current data values for any part of the data in the database and any particular step in a transaction, as a consistency point;

means for manipulating the software, logical processing hardware, and dynamic and stable memory, to alter the designation of a set of current data values for any part of the data in the database and any particular step in a transaction, as a consistency point;

means for manipulating the software, logical processing hardware, and dynamic and stable memory, to select at least one set of current data values for any part of the data in the database, and to manipulate any set of particular steps in at least two transactions, to effectuate transaction relaying;

means for manipulating the software, logical processing hardware, and dynamic and stable memory, upon detection of an error condition, to selectively effectuate implementation of at least one corrective transaction;

means for manipulating the software, logical processing hardware, and dynamic and stable memory, to automatically implement optimization of said logical processing hardware and dynamic and stable memory through altering the steps in said transaction using lookahead-based resource management; and

,

means for manipulating the software, logical processing hardware, and dynamic and stable memory, to automatically manipulate the steps of said transaction and software, and automatically implement optimization of said logical processing hardware and dynamic and stable memory, for the processing of said transaction, through implementation of dependency-based concurrency optimization.