System and Method of Transforming Data for Use in Data Analysis Tools

• US 20090112927A1
• Filed: 10/26/2007
• Published: 04/30/2009
• Est. Priority Date: 10/26/2007
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Chitnis, Upendra, Lingenfelder, Christoph, Pednault, Edwin Peter Dawson
• Granted Patent: US 8,655,918 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising

Abstract

A process of transforming data residing in databases, such as relational databases, into forms suitable as input to data analysis tools, such as predictive modeling tools includes the steps of defining a business process problem to be solved and identifying data requirements. For example, the business process problem may relate to predicting a customer'"'"'s propensity to make purchases in the future or a store'"'"'s requirements for inventory in the future. In the process, a computer implemented method is used for automatically transforming data for data analysis such as predictive modeling. Database metadata that describe database tables, their interrelationships, dimensional information, fact tables and measures are accessed. A mining transformation profile is created to encapsulate aggregations and transformation on data stored in relational databases in order to convert the data to forms suitable for predictive mining tools. The mining transformation profile specifies data transformations relative to the data base metadata. Executable data transformation codes is then generated from the database metadata and the mining transformation profile. Execution of this code results in aggregation and transformation of data residing in a database for input to a data analysis tool such as a predictive modeling tool. The data transformation code can be used by, for example, the predictive modeling tool to generate an output that provides a solution to a business process problem.

Claims

1. In a process of transforming data residing in databases into forms suitable as input to data analysis tools, the process including the steps of defining a business process problem to be solved and identifying data requirements, the improvement of computer implemented method for automatically transforming data for use in data analysis tools comprising the steps of:

accessing database metadata that describe at least one of database tables, their interrelationships, dimensional information, hierarchies, fact tables, and measures;

creating a mining transformation profile metadata to encapsulate at least one of aggregation and transformation on data stored in databases in order to convert the data to forms suitable for data analysis tools, said mining transformation profile specifying at least one of aggregation and transformation on data relative to said database metadata;

generating executable data transformation code from the database metadata and the mining transformation profile, execution of said code producing at least one of aggregation and transformation of data residing in a database for input to a data analysis tool; and

executing the data transformation code by said data analysis tool to generate an output that provides a solution to the business process problem.

2. The process of claim 1 wherein said data analysis tool is a predictive modeling tool.

3. The process of claim 1 wherein said database is a relational database.

4. The process of claim 1 wherein said step of accessing is performed on a plurality of tools or repositories in a relational database environment.

5. The process of claim 1 wherein said step of creating a mining transformation profile creates a mining transformation profile that refers to and/or reuses relational and multidimensional metadata.

6. The process of claim 1 wherein said business process problem is to predict future sales of each item at each store in a retail chain.

7. The process of claim 1 wherein said business process problem is to predict a customer'"'"'s likelihood of making purchases in the future.

8. A system for transforming data residing in databases into forms suitable as input to data analysis tools, comprising:

a computer;

one or more databases associated with or accessible by said computer, wherein said computer can access database metadata that describe at least one of tables, their interrelationships, dimensional information, hierarchies, fact tables and measures; and

instructions stored in a computer readable medium which can be read and performed by said computer, said instructions pertaining to defining a business process problem to be solved and identifying data requirements, the instructions coding for a computer implemented method for automatically transforming data for predictive modeling comprising the steps of;

accessing database metadata that describe at least one of database tables, their interrelationships, dimensional information, hierarchies, fact tables, and measures;

creating a mining transformation profile metadata to encapsulate at least one of aggregation and transformation on data stored in databases in order to convert the data to forms suitable for data analysis tools, said mining transformation profile specifying at least one of aggregation and transformation on data relative to said database metadata;

generating executable data transformation code from the database metadata and the mining transformation profile, execution of said code producing at least one of aggregation and transformation of data residing in a database for input to a data analysis tool; and

executing the data transformation code by a data analysis tool to generate an output that provides a solution to the business process problem.

9. The system of claim 8 wherein said business process problem is to predict future sales of each item at each store in a retail chain.

10. The system of claim 8 wherein said business process problem is to predict a customer'"'"'s likelihood of making purchases in the future.