System and method for conversion of JMS message data into database transactions for application to multiple heterogeneous databases

• US 8,626,778 B2
• Filed: 03/30/2011
• Issued: 01/07/2014
• Est. Priority Date: 07/23/2010
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Wilkes, Stephen, Nielsen, Michael Scott, Pora, Codin
• Primary Examiner(s): LE, HUNG D
• Application Number:
  US13/076,253
• Publication Number:
  US 20120023116A1
• Time in Patent Office: 1,014 Days
• Field of Search: None
• US Class Current: 707/756
• CPC Class Codes: G06F 16/2452 Query translation
• CPC Class Codes: G06F 16/90 Details of database functions independent of the retrieved data types
• CPC Class Codes: G06F 16/901 Indexing; Data structures therefor; Storage structures for retrieval from the web G06F16/951

Abstract

A system and method for performing real-time conversion of data which is present in the form of messages on JMS-compliant or other messaging systems into database transactions, which can then subsequently be applied to multiple heterogeneous databases or other systems. In accordance with an embodiment, the invention provides a means by which data can be read from messages, and converted to a set of database operations that are then stored as a persistent trail file (for example, as an Oracle GoldenGate trail file). The operations, as recorded in the trail file, can then be routed via a network and applied to target systems as required. In accordance with an embodiment, the data can be read in real-time from the messaging system, and written out as quickly as it can be consumed to the persistent trail files.

Claims

1. A system for performing real-time conversion of data present in the form of messages into database transactions which can subsequently be applied to a database, comprising:

a computer including a microprocessor; and

a messaging capture adaptor, which includes

a messaging connectivity component that reads messages from a message queue at a first or source system, and

a data parsing component that parses data within the messages, from a source format of the first or source system, to a target format of a target database or second system;

wherein the messaging capture adaptor

starts a local transaction,

retrieves the messages from the message queue at the first or source system,

processes data within the messages, according to a set of properties, rules and other messaging connectivity information that define how the messages should be parsed and mapped to records in the target database or second system,

generates a trail file containing the processed data, for use in applying the processed data as database operations to the target database or second system, and

commits the local transaction.

2. The system of claim 1, wherein the message queue is a Java Message Service (JMS) queue and the messages are JMS messages.

3. The system of claim 1, wherein the messaging capture adaptor communicates with an extract process to generate the trail file.

4. The system of claim 1, wherein the messaging capture adaptor includes the set of properties, rules and other messaging connectivity information that define how the messages received at the first or source system should be parsed and mapped to records in the target database or second system, to create the trail file.

5. The system of claim 4, wherein the messaging capture adaptor comprises a vendor access module or communications component or interface.

6. The system of claim 1, wherein the messaging capture adaptor operates to request messages from a designated queue at the first or source system by

starting a local messaging service transaction if not already started,

reading a message off the designated queue, and

if a message exists, then returning the contents of the message and any message header properties to the messaging capture adaptor.

7. The system of claim 6, wherein,

if the messaging capture adaptor has successfully parsed the message, has output an entire transaction to the trail, and has ensured the transaction has been flushed to disk, the local messaging service transaction is then committed, removing the messages from the queue or topic at the first or source system, and

in the event of an error, the local messaging service transaction is rolled back, leaving the message unprocessed at the first or source system.

8. The system of claim 1, wherein the messaging capture adaptor supports any one or more of fixed width message parsing, delimited message parsing, or XML message parsing.

9. A method for performing real-time conversion of data present in the form of messages into database transactions which can subsequently be applied to a database, comprising the steps of:

providing a messaging capture adaptor, which includes

a messaging connectivity component that reads messages from a message queue at a first or source system, and

a data parsing component that parses data within the messages, from a source format of the first or source system, to a target format of a target database or second system;

wherein the messaging capture adaptor

starts a local transaction,

retrieves the messages from the message queue at the first or source system,

processes data within the messages, according to a set of properties, rules and other messaging connectivity information that define how the messages should be parsed and mapped to records in the target database or second system,

generates a trail file containing the processed data, for use in applying the processed data as database operations to the target database or second system, and

commits the local transaction.

10. The method of claim 9, wherein the message queue is a Java Message Service (JMS) queue and the messages are JMS messages.

11. The method of claim 9, wherein the messaging capture adaptor communicates with an extract process to generate the trail file.

12. The method of claim 9, wherein the messaging capture adaptor includes the set of properties, rules and other messaging connectivity information that define how the messages received at the first or source system should be parsed and mapped to records in the target database or second system, to create the trail file.

13. The method of claim 12, wherein the messaging capture adaptor comprises a vendor access module or communications component or interface.

14. The method of claim 9, wherein the messaging capture adaptor operates to request messages from a designated queue at the first or source system by

starting a local messaging service transaction if not already started,

reading a message off the designated queue, and

if a message exists, then returning the contents of the message and any message header properties to the messaging capture adaptor.

15. The method of claim 14, wherein,

if the messaging capture adaptor has successfully parsed the message, has output an entire transaction to the trail, and has ensured the transaction has been flushed to disk, the local messaging service transaction is then committed, removing the messages from the queue or topic at the first or source system, and

in the event of an error, the local messaging service transaction is rolled back, leaving the message unprocessed at the first or source system.

16. The method of claim 9, wherein the messaging capture adaptor supports any one or more of fixed width message parsing, delimited message parsing, or XML message parsing.

17. A non-transitory computer readable medium, including instructions stored thereon which when read and executed by a computer cause the computer to perform the steps comprising:

providing a messaging capture adaptor, which includes

a messaging connectivity component that reads messages from a message queue at a first or source system, and

a data parsing component that parses data within the messages, from a source format of the first or source system, to a target format of a target database or second system;

wherein the messaging capture adaptor

starts a local transaction,

retrieves the messages from the message queue at the first or source system,

processes data within the messages, according to a set of properties, rules and other messaging connectivity information that define how the messages should be parsed and mapped to records in the target database or second system,

generates a trail file containing the processed data, for use in applying the processed data as database operations to the target database or second system, and

commits the local transaction.