System and method for conversion of JMS message data into database transactions for application to multiple heterogeneous databases
First Claim
1. A system for integrating a message-based system with a database-based system, comprising:
- a computer including a microprocessor;
a data parsing component, provided as a shared library to integrate a dataflow of a source system, wherein the data parsing component includes a plurality of parsers, wherein each of the plurality of parsers is configured to parse messages received from a message-based system having a particular message format; and
a properties file which indicates properties and data definitions that define how the system is to parse messages from a source format of the source system, to a target format of a target system;
wherein the data parsing component operates toread a message from a messaging topic at the source system,present values read from a header of the message to the properties file,select, using the properties file, a parser from the plurality of parsers to parse the message,parse, using the selected parser and the properties file, the message to determine transaction boundaries and operation types,store the parsed message in a persistent trail file, which is subsequently routed via a network to the target system,generate a source definition file from properties in the properties file, which is provided to the target system, andapply the parsed message as database operations at a database at the target system, wherein, if the target system needs to be recovered from an earlier point in time, the persistent trail is deciphered using the source definition file to replay the database operations.
0 Assignments
0 Petitions
Accused Products
Abstract
A system and method for performing real-time conversion of data which is present in the form of messages on JMS-compliant or other messaging systems into database transactions; which can then subsequently be applied to multiple heterogeneous databases or other systems. In accordance with an embodiment, the invention provides a means by which data can be read from messages in a messaging system, and converted to a set of database operations that are then stored as a persistent trail file. The operations can then be routed via a network and applied to target systems as required, for example to maintain a replicated set of information at one or more different or heterogeneous systems. The data in the message can be formatted in a variety of ways, for example, as fixed width, delimited, or XML data, and the system can be configured to convert this data as appropriate.
-
Citations
20 Claims
-
1. A system for integrating a message-based system with a database-based system, comprising:
-
a computer including a microprocessor; a data parsing component, provided as a shared library to integrate a dataflow of a source system, wherein the data parsing component includes a plurality of parsers, wherein each of the plurality of parsers is configured to parse messages received from a message-based system having a particular message format; and a properties file which indicates properties and data definitions that define how the system is to parse messages from a source format of the source system, to a target format of a target system; wherein the data parsing component operates to read a message from a messaging topic at the source system, present values read from a header of the message to the properties file, select, using the properties file, a parser from the plurality of parsers to parse the message, parse, using the selected parser and the properties file, the message to determine transaction boundaries and operation types, store the parsed message in a persistent trail file, which is subsequently routed via a network to the target system, generate a source definition file from properties in the properties file, which is provided to the target system, and apply the parsed message as database operations at a database at the target system, wherein, if the target system needs to be recovered from an earlier point in time, the persistent trail is deciphered using the source definition file to replay the database operations. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A method for integrating data changes from a source system to a target system, comprising:
-
providing a data parsing component as a shared library to integrate a dataflow of a source system, wherein the data parsing component includes a plurality of parsers, wherein each of the plurality of parsers is configured to parse messages received from a message-based system having a particular message format; and providing a properties file which indicates properties and data definitions that define how the system is to parse messages from a source format of the source system, to a target format of a target system; wherein the data parsing component operates to read a message from a messaging topic at the source system, present values read from a header of the message to the properties file, select, using the properties file, a parser from the plurality of parsers to parse the message, parse, using the selected parser and the properties file, the message to determine transaction boundaries and operation types, store the parsed message in a persistent trail file, which is subsequently routed via a network to the target system, generate a source definition file from properties in the properties file, which is provided to the target system, and apply the parsed message as database operations at a database at the target system, wherein, if the target system needs to be recovered from an earlier point in time, the persistent trail is deciphered using the source definition file to replay the database operations. - View Dependent Claims (9, 10, 11, 12, 13, 14)
-
-
15. A non-transitory computer readable storage medium, including instructions stored thereon which when read and executed by a computer cause the computer to perform the steps comprising:
-
providing a data parsing component as a shared library to integrate a dataflow of a source system, wherein the data parsing component includes a plurality of parsers, wherein each of the plurality of parsers is configured to parse messages received from a message-based system having a particular message format; and providing a properties file which indicates properties and data definitions that define how the system is to parse messages from a source format of the source system, to a target format of a target system; wherein the data parsing component operates to read a message from a messaging topic at the source system, present values read from a header of the message to the properties file, select, using the properties file, a parser from the plurality of parsers to parse the message, parse, using the selected parser and the properties file, the message to determine transaction boundaries and operation types, store the parsed message in a persistent trail file, which is subsequently routed via a network to the target system, generate a source definition file from properties in the properties file, which is provided to the target system, and apply the parsed message as database operations at a database at the target system, wherein, if the target system needs to be recovered from an earlier point in time, the persistent trail is deciphered using the source definition file to replay the database operations. - View Dependent Claims (16, 17, 18, 19, 20)
-
Specification