Methods and systems for real time integration services

• US 20050086360A1
• Filed: 08/24/2004
• Published: 04/21/2005
• Est. Priority Date: 08/27/2003
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: Ascential Software Corporation (International Business Machines Corporation)
• Inventors: Toum, Christophe, Cherel, Thomas, dos Santos, Cassio Souza, Scheffler, Lee, Meeks, David T., Mamou, Jean-Claude, Tinnel, Brian
• Granted Patent: US 8,307,109 B2
• Time in Patent Office: Days
• US Class Current: 709/232
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses

Abstract

Methods and systems for enterprise data integration are described. The methods and systems take elements of a data integration process typically operating in a batch-mode, transform the elements in real time, and expose the results as a service that can be accessed by a business enterprise in real time, and optionally also in batch mode. The service can be accessed through a graphical user interface, providing automatic data integration in real time without additional coding. The service can also operate with mobile devices.

Claims

1. A method for real time integration of data residing in a plurality of data sources, comprising the steps of:

processing a request for data from the plurality of data sources;

receiving data from the plurality of data sources;

integrating the received data in real time and forming at least one integrated data output configured for at least one data target; and

exposing the integrated data of the at least one data target to a user in real time as a service.

2. The method of claim 1, wherein the service is accessed through a web service protocol.

3. The method of claim 1, wherein real time integration of the data supports data integration job instances, with a job instance capable of supporting a batch topology or a real time topology, or both.

4. The method of claim 3, further comprising pipelining a series of data integration transactions for delivery to a job instance.

5. The method of claim 4, further comprising inserting an end-of-wave marker between data integration transactions to separate processing the transactions into distinct units.

6. The method of claim 1, wherein real time integration of the data includes communicating with at least one other data source where data is handled or data or other information is stored.

7. The method of claim 1, wherein real time integration includes processing the data request through a plurality of processing facilities.

8. The method of claim 7, wherein the plurality of processing facilities processes the data request concurrently.

9. The method of claim 1, wherein receiving the data inputs from the plurality of data sources intermittently.

10. The method of claim 9, wherein the intermittent reception occurs via a wireless communication channel.

11. The method of claim 1, wherein the plurality of data sources comprise a data warehouse or a data retrieval system, or both.

12. The method of claim 1, further comprising receiving the data from a facility selected from the group consisting of a mobile computing facility, a desktop computing facility, and a central computing facility.

13. The method of claim 7, wherein at least one of the plurality of processing facilities is remotely located.

14. The method of claim 13, wherein the at least one remotely located processing facility comprises a server.

15. The method of claim 1, wherein the plurality of data sources contains data from different sources.

16. The method of claim 15, wherein at least one of the different sources comprises data that is incompatible with real time integration.

17. The method of claim 1, wherein exposing the service comprises publishing the service as a web service.

18. The method of claim 1 wherein the service is defined by a WSDL description.

19. The method of claim 1, wherein the service is published in a public registry.

20. The method of claim 1, wherein the service is published in a private registry.

21. The method of claim 1, wherein the service is bound by multiple access protocols.

22. The method of claim 1, wherein the user is an enterprise device or an enterprise application.

23. The method of claim 22, wherein the enterprise application is a distribution process.

24. The method of claim 22, wherein the enterprise application is a manufacturing process.

25. The method of claim 22, wherein the enterprise application is a financial process.

26. The method of claim 1, further comprising attaching a client application to the service.

27. The method of claim 26, further comprising downloading the client application to a client.

28. The method of claim 26, wherein the client is a mobile computing device.

29. The method of claim 1, wherein integrating the data includes extracting the data from the data sources, transforming the extracted data into desired formats, and transmitting the data having the desired format to the at least one target in real time.

30. A data retrieval process for retrieving data from a plurality of data sources in real time, comprising:

generating a data request;

processing the data request, and based on the request, extracting the data from at least one of the plurality of data sources;

integrating the extracted data in real time and forming at least one integrated data output; and

exposing the integrated data output in real time as a service.

31. An enterprise real time integration platform comprising:

a plurality of data sources;

a data integration facility in communication with the plurality of data sources; and

a real time integration facility that exposes the data integration platform as a service,

processes a data request in real time through the data integration facility, and extracts from at least one of the plurality of data sources data in a format suitable to be read by a device or an application of the enterprise.

32. The platform of claim 31, wherein the service can be accessed through a web service protocol.

33. The platform of claim 31, wherein the real time integration facility supports data integration job instances, with a job instance capable of supporting a batch topology or a real time topology, or both.

34. The platform of claim 33, further comprising a pipeline facility for managing delivery of a series of data integration transactions to a job instance.

35. The platform of claim 34, further comprising an end-of-wave marker inserted between each data integration transaction of the series of data integration transactions in the job pipeline to separate processing the transactions into distinct units.

36. The platform of claim 31, wherein the real time integration facility communicates with at least one other facility where data is handled or data or other information is stored.

37. The platform of claim 31, further comprising a plurality of processing facilities, wherein the real time integration facility processes the data request through the plurality of processing facilities.

38. The platform of claim 37, wherein the plurality of processing facilities processes the data request concurrently.

39. The platform of claim 31, wherein the communication between the data integration facility and the plurality of data sources is intermittent.

40. The platform of claim 39, wherein the intermittent communication comprises a wireless communication channel.

41. The platform of claim 31, wherein the plurality of data sources comprise a data warehouse or a data retrieval system, or both.

42. The platform of claim 31, wherein the data request is generated on a facility selected from the group consisting of a mobile computing facility, a desktop computing facility, and a central computing facility.

43. The platform of claim 37, wherein at least one of the plurality of processing facilities is remotely located.

44. The platform of claim 43, wherein the at least one remotely located processing facility comprises a server.

45. The platform of claim 31, wherein the plurality of data sources contains data from different sources.

46. The platform of claim 45, wherein at least one of the different sources comprises data that is incompatible with processing by the data integration facility.

47. The platform of claim 31, wherein exposing the service comprises publishing the service as a web service.

48. The platform of claim 31 wherein the service is defined by a WSDL description.

49. The platform of claim 31, wherein the service is published in a public registry.

50. The platform of claim 31, wherein the service is published in a private registry.

51. The platform of claim 31, wherein the service is bound by multiple access protocols.

52. The platform of claim 31, wherein the enterprise application is a distribution process.

53. The platform of claim 31, wherein the enterprise application is a manufacturing process.

54. The platform of claim 31, wherein the enterprise application is a financial process.

55. The platform of claim 31, further comprising a facility for attaching a client application to the service.

56. The platform of claim 55, wherein a client can access the service to download the client application.

57. The platform of claim 56, wherein the client is a mobile computing device.