Method and system of data warehousing and building business intelligence using a data storage model

• US 20020161778A1
• Filed: 09/27/2001
• Published: 10/31/2002
• Est. Priority Date: 02/24/2001
• Status: Abandoned Application

• Current Assignee: EROC, Inc. /CO/ (Visionary Integration Professionals LLC)
• Original Assignee: EROC, Inc. /CO/ (Visionary Integration Professionals LLC)
• Inventors: Linstedt, Daniel Eames
• Application Number:
  US09/965,343
• Publication Number:
  US 20020161778A1
• Time in Patent Office: Days
• US Class Current: 707/102
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling

Abstract

A data migration, data integration, data warehousing, and business intelligence system including a data storage model is provided that allows a business to effectively utilize its data to make business decisions. The system can be designed to include a number of data storage units including a data dock, a staging area, a data vault, a data mart, a data collection area, a metrics repository, and a metadata repository. Data received from a number of source systems moves through the data storage units and is processed along the way by a number of process areas including a profiling process area, a cleansing process area, a data loading process area, a business rules and integration process area, a propagation, aggregation and subject area breakout process area, and a business intelligence and decision support systems process area. Movement of the data is performed by metagates. The processed data is then received by corporate portals for use in making business decisions. The system may be implemented by an implementation team made up of members including a project manager, a business analyst, a system architect, a data modeler/data architect, a data migration expert, a DSS/OLAP expert, a data profiler/cleanser, and a trainer.

Claims

1. A method of building business intelligence, the method comprising:

receiving data from at least one source system of an enterprise, wherein the data is representative of business operations of the enterprise;

delivering the data to a staging area via a first metagate, wherein the staging area focuses the data into a single area on a single relational database management system;

delivering the data from the staging area to a data vault via a second metagate, wherein the data vault houses data from functional areas of the enterprise;

delivering the data from the data vault to a data mart via a third metagate, wherein the data mart stores data for a single function of the functional areas of the enterprise;

transferring data to at least one of a business intelligence and decision support systems module, a corporate portal module, and at least one of the at least one source system of the enterprise;

collecting metrics in a metrics repository; and

collecting metadata in a metadata repository.

2. A method as claimed in claim 1, further comprising profiling and cleansing the data received from the at least one source system to produce profiled and cleansed data.

3. A method as claimed in claim 2, further comprising delivering the profiled and cleansed data to a data dock.

4. A method as claimed in claim 3, wherein the act of delivering the profiled and cleansed data to the data dock is accomplished using middleware tools.

5. A method as claimed in claim 3, further comprising including operational data in the data dock.

6. A method as claimed in claim 3, further comprising positioning the data dock on a single relational database management system regardless of the data received from the at least one source system.

7. A method as claimed in claim 6, further comprising porting data between and around sources using the data dock.

8. A method as claimed in claim 3, further comprising maintaining data in the data dock for a predetermined amount of time, and then after the predetermined amount of time has elapsed, proceeding to one of a first condition and a second condition, wherein the first condition is deleting the data, and wherein the second condition is the act of delivering the data to a staging area

9. A method as claimed in claim 8, wherein the act of delivering data to the staging area is done in parallel from the data dock and the at least one source system.

10. A method as claimed in claim 1, wherein the act of delivering data to the staging area is done directly from the source systems.

11. A method as claimed in claim 1, wherein data integration and a data movement framework are provided by the first metagate.

12. A method as claimed in claim 11, wherein one of a data loading process, a near real-time load process, and a trickle feed load process are performed in the data movement framework.

13. A method as claimed in claim 1, further comprising housing snapshots of the data received from the at least one source system in the staging area.

14. A method as claimed in claim 1, wherein the staging area includes independent table structures in a parallel configuration to allow for tuning the data received from the at least one source system for speed of access.

15. A method as claimed in claim 1, wherein the staging area is refreshed with each act of delivering the data from the staging area to the data vault.

16. A method as claimed in claim 1, wherein the data vault facilitates the process of data mining.

17. A method as claimed in claim 1, wherein the second metagate improves the quality of data through integration and pre-qualification of the data using an implementation of business rules.

18. A method as claimed in claim 17, wherein data that fails to meet the implementation of the business rules is marked for one of error processing and discarding.

19. A method as claimed in claim 1, wherein the data mart is configured in a star schema.

20. A method as claimed in claim 1, wherein the data mart is split into aggregations and different subject areas.

21. A method as claimed in claim 20, wherein the data mart offers capabilities of aggregates, including at least one of drill-down, decision support systems, and on-line analytical processing support.

22. A method as claimed in claim 1, further comprising delivering the data from the data mart to a data collection area.

23. A method as claimed in claim 1, wherein the metrics are utilized for at least one of measuring a complexity of source code, measuring a reliability of source code, measuring a length of a development process, measuring a quality of a development process, measuring a performance of an application, identifying future hardware upgrades, quantifying future hardware upgrades, determining frequency of use, determining content accessed, and tracking data.

24. A method as claimed in claim 1, wherein the metadata is utilized for at least one of facilitating understanding of the cycle and flow of data, and providing knowledge of processes on the data.

26. A system as claimed in claim 25, further comprising a metadata repository.

27. A system as claimed in claim 25, further comprising a metrics repository.

28. A system as claimed in claim 25, wherein data is delivered to the profiling process area from at least one source system of an enterprise.

30. A system as claimed in claim 29, further comprising a data dock.

31. A system as claimed in claim 30, wherein data from at least one source system of an enterprise is delivered to a profiling and cleansing module which produces profiled and cleansed data.

32. A system as claimed in claim 31, wherein the profiled and cleansed data is delivered to the data dock.

33. A system as claimed in claim 32, wherein the data is delivered to the staging area from the data dock.

34. A system as claimed in claim 29, further comprising a data collection area.

35. A system as claimed in claim 34, wherein data is delivered to at least one of a business intelligence and decision support systems module, a corporate portal module, and the at least one source system from at least one of the data mart and the data collection area.

36. A system as claimed in claim 29, wherein the data vault includes at least two hubs, wherein each of the at least two hubs includes a primary key, a stamp indicating the loading time of the primary key in the hub, and a record source indicating the source of the primary key;

at least two satellites, wherein each of the at least two satellites is coupled to at least one of the at least two hubs in a parent-child relationship, further wherein each satellite includes a stamp indicating the loading time of data in the satellite and a business function;

a link to provide a one-to-many relationship between two of the at least two hubs; and

a detail table coupled to at least one of the at least two hubs, wherein the detail table includes attributes of the data from the functional areas of the enterprise.

37. A system as claimed in claim 36, wherein each of the at least two satellites further includes at least one of a primary key, business data, aggregation data, user data, a stamp indicating the time of at least one of user data insertion and user data alteration, and a record source.

38. A system as claimed in claim 36, further comprising a second satellite coupled to at least one of the at least two hubs in a parent-child relationship.

39. A system as claimed in claim 36, wherein the link includes at least two foreign keys and a stamp.

40. A system as claimed in claim 36, wherein each of the at least two hubs further includes an associated business key and a stamp includes indicating the loading time of the associated business key.

25. A data migration, data integration, data warehousing, and business intelligence system comprising:

a profiling process area;

a cleansing process area;

a data loading process area;

a business rules and integration process area;

a propagation, aggregation, and subject area breakout process area; and

a business intelligence and decision support systems process area.

29. A data migration, data integration, data warehousing, and business intelligence system comprising:

a staging area;

a data vault;

a data mart;

a metrics repository; and

a metadata repository.

41. A method of implementing a data migration, data integration, data warehousing, and business intelligence system, the method comprising:

providing an implementation team, wherein the implementation team includes

a project manager whose function is to manage the implementation of the data migration, data integration, data warehousing, and business intelligence system at client sites,

a business analyst whose function is to interface with end-users, collecting, consolidating, organizing, and prioritizing business needs of the end-users,

a systems architect whose function is to provide a blueprint for the hardware, software, and interfaces that defines the flow of data between components of the data migration, data integration, data warehousing, and business intelligence system,

a data modeler/data architect whose function is to model and document source systems and business requirements of the end-users,

a data migration expert whose function is to determine and develop the best solution to migrate and integrate data from the various sources systems, and

a DSS/OLAP expert whose function is to determine and develop the best reporting solution or DSS based on end-user business requirements and to implement any OLAP tools selected for use in the data migration, data integration, data warehousing, and business intelligence system;

allowing the members of the implementation team to perform the function they are trained to perform in a specialized manner;

providing mentoring, cross-training, and support through the course of implementing the data migration, data integration, data warehousing, and business intelligence system; and

leaving the end-users with documentation and deliverables for maintaining and expanding the data migration, data integration, data warehousing, and business intelligence system.

42. A method as claimed in claim 41, wherein the implementation team further includes a data cleanser/profiler whose function is to determine which business rules apply to which data.

43. A method as claimed in claim 41, wherein the implementation team further includes a trainer whose function is to train the end-users on the tools and methods necessary to use the data migration, data integration, data warehousing, and business intelligence system.

44. A data storage device for housing data from functional areas of an enterprise, the data storage device comprising:

at least two hubs, wherein each of the at least two hubs includes a primary key, a stamp indicating the loading time of the primary key in the hub, and a record source indicating the source of the primary key;

at least two satellites, wherein each of the at least two satellites is coupled to at least one of the at least two hubs in a parent-child relationship, further wherein each satellite includes a stamp indicating the loading time of data in the satellite and a business function;

a link to provide a one-to-many relationship between two of the at least two hubs; and

a detail table coupled to at least one of the at least two hubs, wherein the detail table includes attributes of the data from the functional areas of the enterprise.

45. A data storage device as claimed in claim 44, wherein each of the at least two satellites further includes at least one of a primary key, business data, aggregation data, user data, a stamp indicating the time of at least one of user data insertion and user data alteration, and a record source.

46. A data storage device as claimed in claim 44, and further comprising a second satellite coupled to at least one of the at least two hubs in a parent-child relationship.

47. A data storage device as claimed in claim 44, wherein the link includes at least two foreign keys and a stamp.

48. A data storage device as claimed in claim 44, wherein each of the at least two hubs further includes an associated business key and a stamp includes indicating the loading time of the associated business key.