Data transfer and/or transformation system and method

• US 20040068509A1
• Filed: 07/21/2003
• Published: 04/08/2004
• Est. Priority Date: 01/19/2001
• Status: Abandoned Application

• Current Assignee: OrderWare Solutions Ltd. (Smarter Capital Pty Ltd.)
• Original Assignee: OrderWare Solutions Ltd. (Smarter Capital Pty Ltd.)
• Inventors: Garden, Peter William, Nigro, Anthony Jason, Rowley, Darren Michael
• Application Number:
  US10/466,928
• Publication Number:
  US 20040068509A1
• Time in Patent Office: Days
• US Class Current: 707/100
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database

Abstract

A method for processing data. Meta-data descriptors (15, 18) are defined to describe the data either by the user (17) or automatically using a meta-data connector (16). A meta-data descriptor describes the structure of data including field names (1, 4). A meta-data connector describes how to access the data. Different types of meta-data connectors (10, 11, 12) exist for different types of data such as JDBC and XML. An interactive user application (23) is utilised to facilitate the definition of a process. A process consists of certain operations (7) in relation to meta-data descriptors (2, 5) such as transformation of data from one field name to another. A component (8, 26) is provided to process data in accordance with the defined process. A computer system and software for implementing the method is also disclosed.

Claims

1. A computer implemented method of processing data comprising the steps of:

iv) defining meta-data descriptors to represent the data;

v) in an interactive user application defining a process associated with at least one of the meta-data descriptors; and

vi) processing the data in accordance with the defined process.

Meta-Data Connectors

2. A method as claimed in claim 1 wherein access to the data is through a meta-data connector.

3. A method as claimed in claim 2 wherein the meta-data connector comprises information about how to access the data.

4. A method as claimed in any one of claims 2 to 3 wherein the information includes database, ftp, mail server, web page or telnet logon procedures, navigation procedures, location of the data, and location of data within the data.

5. A method as claimed in any one of claims 2 to 4 wherein different types of meta-data connectors are used to access different types of data.

6. A method as claimed in any one of claims 2 to 5 wherein the meta-data connectors are defined by a user in an interactive user application.

Meta-Data Descriptors:

7. A method as claimed in one of the preceding claims wherein the meta-data descriptors represent the structure of the data.

8. A method as claimed in claim 7 wherein the structure of the data consists of entities and their elements and any relationships between the entities or between the elements.

9. A method as claimed in any one of the preceding claims wherein the meta-data descriptors describe rules to identify specific elements of data and specific types of data within the data.

10. A method as claimed in claim 9 wherein at least one rule specifies contextual criteria.

Defining Meta-Data Descriptors

11. A method as claimed in any one of the preceding claims wherein the meta-data descriptors are obtained by examination of the data.

12. A method as claimed in any of the preceding claims wherein meta-data descriptors are obtained by examination of one or more secondary sources.

13. A method as claimed in any one of the preceding claims wherein the meta-data descriptors are defined with the assistance of a user through an interactive user application.

14. A method as claimed in any one of the preceding claims wherein the meta-data descriptors are defined by any combination of the following operating in conjunction:

examination of the data, assistance of a user through an interactive user application, and examination of one or more secondary sources.

15. A method as claimed in any one of the preceding claims wherein the meta-data descriptors are defined by the examination of a database by querying the database.

16. A method as claimed in any one of the preceding claims wherein the meta-data descriptors are defined by examination of a series of keystrokes and screen captures resulting from a telnet session with the assistance of a user through an interactive user application.

17. A method as claimed in any one of the preceding claims wherein the meta-data descriptors are defined by examination of an XML structure or schema.

18. A method as claimed in any one of the preceding claims wherein the meta-data descriptors are defined by examination of an EDI or IDOC file.

19. A method as claimed in any one of the preceding claims wherein the meta-data descriptors are defined by parsing the data and automatically identifying the meta-data.

20. A method as claimed in any one of claims 2 to 19 wherein examination of the data occurs through the use of the meta-data connector.

Defining the Process

21. A method as claimed in any one of the preceding claims wherein the defined process is created in an interactive user application that displays meta-data descriptors and allows a user to define steps for processing the data represented by the meta-data descriptors.

22. A method as claimed in claim 21 wherein the process defines steps for transforming data represented by source meta-data descriptors into data represented by target metadata descriptors.

23. A method as claimed in any one of claims 21 to 22 wherein a defined step involves manipulations or transformations occurring via programmatic or arithmetic methods.

24. A method as claimed in any one of claims 21 to 23 wherein a defined step involves an arithmetic calculation, string transformation, key generation, an SQL calculation, a calendar calculation, a date/time calculation, a financial calculation or a statistical calculation.

25. A method as claimed in any one of claims 21 to 24 wherein the defined step is performed by a device that is remote from the device initiating the step.

26. A method as claimed in any one of claims 21 to 25 wherein a defined step includes the mapping of source meta-data descriptors to target meta-data descriptors.

27. A method as claimed in claim 26 wherein at least one source meta-data descriptor is mapped to a plurality of target meta-data descriptors.

28. A method as claimed in claim 26 or claim 27 wherein at least one source meta-data descriptor is mapped to target meta-data descriptors of more than one target data source.

29. A method as claimed in any one of claims 26 to 28 wherein more than one source metadata descriptor is mapped to at least one target meta-data descriptor.

30. A method as claimed in any one of claims 26 to 29 wherein meta-data descriptors of a plurality of source data sources are mapped to target meta-data descriptors of at least one target data source.

31. A method as claimed in any one of claims 26 to 30 wherein meta-data descriptors of a plurality of source data sources are mapped to target meta-data descriptors of a plurality of target data sources.

32. A method as claimed in any one of the preceding claims wherein data referenced by meta-data descriptors is remotely located.

Processing of Data

33. A method as claimed in any one of the preceding claims wherein the defined process is used to processed data.

34. A method as claimed in any of the preceding claims wherein the defined process is used to transform data extracted from one or more source data sources for supply to one or more target data sources.

Activities and Actions

35. A method as claimed in any of the preceding claims wherein an application is provided to enable the definition of an activity consisting of actions where at least one of the actions executes the defined process.

36. A method as claimed in claim 35 wherein the application is a graphical user application provided to enable a user to define the activity.

37. A method as claimed in claim 36 wherein the user defines new actions.

38. A method as claimed in any one of claims 35 to 37 wherein the activity is defined by a script.

39. A method as claimed in claim 38 wherein the script is written in a high-level programming language.

40. A method as claimed in any one of claims 35 to 39 wherein the activity contains an action which can stop the current activity until a different activity reaches a certain state.

41. A method as claimed in any one of claims 35 to 40 wherein the activity contains an action with the ability to start the execution of another activity either synchronously or asynchronously.

42. A method as claimed in any one of claims 35 to 41 wherein the activity contains an action with the ability to manipulate the environment of a computer system.

43. A method as claimed in any one of claims 35 to 42 wherein the activity contains an action which can execute defined processes, execute an email function, execute a program, pass parameters to an executing program, or set an internal flag.

44. A method as claimed in any one of claims 35 to 43 wherein the activity contains an action with the ability to control repetition of certain actions within the activity based upon programmatic manipulation of values within the data undergoing processing.

45. A method as claimed in any one of claims 35 to 44 wherein the activity is executed upon the occurrence of a trigger event.

46. A method as claimed in any one of claims 35 to 45 wherein the execution of an action is dependent on the occurrence of a trigger event.

Scheduling

47. A method as claimed in any one of the preceding claims wherein a schedule of the time or times for executing each activity is created and the activities are executed at the scheduled times.

48. A method as claimed in claim 47 wherein a user can schedule the times for execution of activities via a graphical interface.

49. A method as claimed in any one of claims 47 to 48 wherein the occurrence of time for executing each activity generates a trigger event.

Proxy Module

50. A method as claimed in any one of the preceding claims wherein a proxy module residing on a remote computer is utilised to accept data resulting from the processes and to modify data residing on the remote computer.

51. A method as claimed in any one of the preceding claims wherein a proxy module residing on a remote computer is utilised to extract data from the remote computer for the processes.

Trigger Events

52. A method as claimed in any one of claims 45 to 51 wherein the trigger event is a data and/or time.

53. A method as claimed in claim 45 to 52 wherein the trigger event is a type of data entry.

54. A method as claimed in any one of claims 45 to 53 wherein the trigger event is a specified change in status of an activity or action.

55. A method as claimed in any one of claims 45 to 54 wherein the trigger event is a specified change in status of a file system.

56. A method as claimed in any one of claims 45 to 55 wherein the trigger event is an occurrence of a specified state in a network system.

57. A method as claimed in any one of claims 45 to 56 wherein the trigger event is generated by the schedule.

58. A method as claimed in any one of the preceding claims when dependent upon claim 34 wherein the data resulting from the process and the associated meta-data descriptors are sent to the computer hosting the target data source for the host computer to update the target data source.

59. A method as claimed in any one of the preceding claims when dependent upon claim 34 wherein the data resulting from the process is sent to the computer hosting the target data source as a file to update the target data source.

Data Formats

60. A method as claimed in any preceding claim wherein the data is unstructured data.

61. A method as claimed in claim 60 wherein the data is a text file.

62. A method as claimed in claim 60 wherein the data is a telnet terminal session.

63. A method as claimed in any preceding claim wherein the data is structured data.

64. A method as claimed in claim 63 wherein the data is an HTML, EDI, IDOC, XML, CSV, text or a database file.

85. A computer system for effecting the method of any one of the claims 1 to 65.

86. Software for effecting the method or system of any one of claims 1 to 85.

87. Storage media containing software as claimed in claim 86.

88. Data produced by the method, system or software or any one of the preceding claims.

65. A computer implemented method of transforming selected data from one or more source data sources to one or more target data sources comprising the steps of:

iv) defining meta-data descriptors for the source data sources and for the target data sources;

v) in an interactive user application defining a transformation process between the source meta-data descriptors and the target meta-data descriptors; and

vi) transforming source data extracted from the source data sources in accordance with the defined transformation process to generate target data for supply to the target data sources.

66. A computer system for processing data comprising:

vi) a processor;

vii) memory for supplying data to the processor;

viii) an input device for providing user input to the processor;

ix) a display device for displaying information from the processor;

x) an application residing in memory which, when executed by the processor, is responsive to user input to define meta-data descriptors to represent data and to define a process associated with at least one of the meta-data descriptors; and

to process the data in accordance with the defined process.

67. A computer system as claimed in claim 66 wherein the application defines meta-data descriptors by examining the data.

68. A computer system as claimed in claim 66 wherein the application defines meta-data descriptors by querying a database.

69. A computer system as claimed in claim 66 wherein user input from the input device assists the application in defining the meta-data descriptors.

70. A computer system as claimed in claims 66 to 69 wherein the application defines the process by displaying meta-data descriptors on the display device and accepting user input from the input device.

71. A computer system as claimed in claim 70 wherein the defined process includes transformation or manipulation operations all relating to meta-data descriptors.

72. A computer system as claimed in claim 71 wherein the transformation operations include programmatic or arithmetic operations.

73. A computer system as claimed in claims 70 to 72 wherein the user input comprises mouse or keyboard actions.

74. A computer system as claimed in claims 66 to 73 including:

one or more data sources.

75. A computer system as claimed in claim 74 wherein one or more of the data sources reside on a remote system.

76. A computer system as claimed in any one of claims 66 to 75 including:

a proxy device.

77. A computer system as claimed in claim 76 when dependent on claim 75 wherein the proxy device resides on the remote system.

78. A computer system as claimed in claim 77 wherein the proxy device transfers data from the data sources on the remote system to the processor.

79. A computer system as claimed in claim 78 wherein the proxy device transfers data from the processor to the data sources on the remote system.

80. A computer system as claimed in any one of claims 66 to 79 including:

a timer device

81. A computer system as claimed in claim 80 wherein the application is responsive to user input to define times when an activity should occur and when times are reached as determined by the timer device executes the activities.

82. A computer system as claimed in claim 81 wherein the activity includes the defined process.

83. A computer system for processing data substantially as herein described with reference to FIGS. 7 to 25 of the accompanying drawings.

84. A computer implemented method of processing data substantially as herein described with reference to FIGS. 1 to 5 of the accompanying drawings.