Streaming multidimensional data by bypassing multidimensional query processor

• US 7,792,784 B2
• Filed: 06/19/2007
• Issued: 09/07/2010
• Est. Priority Date: 05/31/2007
• Status: Expired due to Fees

• Current Assignee: LinkedIn Corporation (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Gupta, Sanjeev Kumar
• Primary Examiner(s): Ali; Mohammad
• Assistant Examiner(s): Shanmugasundaram; Kannan
• Application Number:
  US11/765,277
• Publication Number:
  US 20080301086A1
• Time in Patent Office: 1,176 Days
• Field of Search: 707/2
• US Class Current: 707/602
• CPC Class Codes: G06F 16/283 Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
• CPC Class Codes: Y10S 707/957 Multidimensional

Abstract

A data warehouse comprises a multidimensional data storage area to store multidimensional data and a transactional data storage area to store underlying transactional data from which the multidimensional data is generated. The data warehouse also comprises a multidimensional query processor to provide one or more interfaces to issue a first set of queries conforming to one or more multidimensional query languages to retrieve the multidimensional data from the multidimensional data storage area. Furthermore, the data warehouse comprises a dynamic query engine to provide an additional interface to issue a second set of queries that bypass the interfaces provided by the multidimensional query processor to retrieve the transactional data from the transactional data storage area, wherein the second set of queries do not conform to any of the one or more multidimensional query languages.

Claims

1. A data warehouse comprising:

a multidimensional data storage area comprising a multidimensional data cube, the multidimensional data cube comprising a data storage schema that defines a structure of the multidimensional data cube and multidimensional data arranged according to the data storage schema;

a transactional data storage area to store underlying transactional data from which the multidimensional data of the multidimensional data cube are generated;

a computer-readable storage medium comprising instructions for a multidimensional query processor and a dynamic query engine;

one or more processors that execute the multidimensional query processor to provide one or more interfaces to issue a first set of queries conforming to one or more multidimensional query languages to retrieve metadata defining the data storage schema that defines the structure of the multidimensional data cube, without retrieving the multidimensional data of the multidimensional data cube; and

that execute the dynamic query engine to provide an additional interface to issue a second set of queries that bypass the interfaces provided by the multidimensional query processor and the multidimensional data storage area to retrieve the transactional data directly from the transactional data storage area, wherein the second set of queries do not conform to any of the one or more multidimensional query languages, and wherein the second set of queries are defined to retrieve the transactional data from which the multidimensional data of the multidimensional data cube are generated, wherein the transactional data comprises enterprise data representing enterprise activities; and

a gateway server to transmit the results of the second set of queries through a network, wherein the gateway server transmits the results as a real-time stream of events that indicate incremental changes to the transaction data, and wherein the stream of events represents real-time changes to the enterprise data in response to the enterprise activities,

wherein the data warehouse comprises a data warehouse for an enterprise software system.

2. The data warehouse of claim 1, wherein the gateway server implements a chunking protocol to transmit the results of the second set of queries as a series of discrete chunks of data.

3. The data warehouse of claim 2, wherein the gateway server implements the chunking protocol as an application layer (layer seven) application of the Open Systems Interconnection (“

OSI”

) networking model.

4. The data warehouse of claim 1, further comprising a data table, wherein the data table includes a first column to store identifiers of network devices and a second column to store queries associated with the identifiers of the network devices.

5. The data warehouse of claim 1, wherein the additional interface provided by the dynamic query engine is a Structured Query Language (“

SQL”

) interface.

6. The data warehouse of claim 5, wherein the dynamic query engine includes a query translator to translate SQL queries into OpenSQL queries.

7. The data warehouse of claim 1, wherein the data warehouse comprises the SAP Business Information Warehouse.

8. The data warehouse of claim 7, wherein the multidimensional query processor comprises a Multidimensional Expressions (“

MDX”

) processor.

9. The data warehouse of claim 7, wherein the multidimensional query processor comprises an SAP Business Explorer (“

BEX”

) processor.

10. A method comprising:

installing a dynamic query engine on a data warehouse, wherein the data warehouse includes transactional data in a transactional data storage area and a multidimensional data cube comprising multidimensional data generated from the transactional data in a multidimensional data storage area, wherein the multidimensional data cube comprises a data storage schema that defines a structure of the multidimensional data cube, wherein the multidimensional data of the multidimensional data cube is arranged according to the data storage schema, wherein the transactional data storage area comprises transactional data from which the multidimensional data of the multidimensional data cube are generated, and further wherein the data warehouse includes a multidimensional query processor that provides one or more interfaces for issuing a first set of queries conforming to the one or more multidimensional query languages to retrieve metadata defining the data storage schema that defines the structure of the multidimensional data cube, without retrieving the multidimensional data of the multidimensional data cube, and wherein the data warehouse comprises a data warehouse for an enterprise software system;

providing, with the dynamic query engine, an additional interface that issues a second set of queries that bypass the interfaces provided by the multidimensional query processor and the multidimensional data storage area, wherein the second set of queries do not conform to any of the one or more multidimensional query languages, and wherein the second set of queries are defined to retrieve the transactional data from which the multidimensional data of the multidimensional data cube are generated, wherein the transactional data comprises enterprise data representing enterprise activities;

retrieving, with the one or more interfaces of the multidimensional query processor, the metadata defining the data storage schema that defines the structure of the multidimensional data cube from the multidimensional data storage area without retrieving the multidimensional data from the multidimensional data storage area;

retrieving, with the additional interface of the dynamic query engine, the transactional data, from which the multidimensional data of the multidimensional data cube are generated, directly from the transactional data storage area according to the second set of queries; and

streaming the retrieved transactional data on a network via a gateway server, wherein the gateway server streams the retrieved transactional data as a real-time stream of events that indicate incremental changes to the transactional data, and wherein the stream of events represents real-time changes to the enterprise data in response to the enterprise activities.

11. The method of claim 10, wherein streaming includes transmitting, with a gateway server operating at an application layer (layer seven) of the Open Systems Interconnection (“

OSI”

) networking model, the results of the second set of queries through the network.

12. The method of claim 11, wherein transmitting comprises:

separating the results of the second set of queries into a plurality of chunks of application layer data; and

sending each of the chunks individually through the network according to a chunking protocol.

13. The method of claim 11, wherein transmitting comprises transmitting according to the transmission control protocol/internet protocol (“

TCP/IP”

).

14. The method of claim 10, further comprising storing, in a data table, the second set of queries and an identifier which identifies a recipient of the results of the second set of queries.

15. The method of claim 10, wherein providing an additional interface comprises providing a Structured Query Language (“

SQL”

) interface.

16. The method of claim 15, further comprising:

receiving SQL queries, wherein the SQL queries are the second set of queries; and

translating the SQL queries into OpenSQL queries.

17. A system comprising:

a data warehouse comprising a data warehouse for an enterprise software system, the data warehouse comprising;

a multidimensional data storage area comprising a multidimensional data cube comprising a data storage schema that defines a structure of the multidimensional data cube, the multidimensional data storage area further comprising multidimensional data arranged according to the data storage schema,

a transactional data storage area to store underlying transactional data from which the multidimensional data of the multidimensional data cube are generated,

a multidimensional query processor to provide one or more interfaces to issue a first set of queries conforming to one or more multidimensional query languages to retrieve metadata defining the data storage schema that defines the structure of the multidimensional data cube, without retrieving the multidimensional data of the multidimensional data cube; and

a dynamic query engine to provide an additional interface to issue a second set of queries that bypass the interfaces provided by the multidimensional query processor and the multidimensional data storage area to retrieve the transactional data directly from the transactional data storage area and metadata from the multidimensional data storage area, wherein the second set of queries do not conform to any of the one or more multidimensional query languages, and wherein the second set of queries are defined to retrieve the transactional data from which the multidimensional data of the multidimensional data cube are generated, wherein the transactional data comprises enterprise data representing enterprise activities; and

a network device comprising a dashboard device, the network device comprising;

a streaming database having a local multidimensional data storage area, a metadata driver to receive metadata from the dynamic query engine of the data warehouse, and

a data driver to create a local storage schema in the local multidimensional data storage area according to the metadata received from the dynamic query engine such that the local storage schema corresponds to the storage schema of the multidimensional data cube of the data warehouse, to generate the second set of queries for the dynamic query engine, to receive transactional data from the dynamic query engine, and to store the received transactional data in the local multidimensional storage area according to the storage schema,

wherein the dynamic query engine of the data warehouse transmits, to the dashboard device, results of the second set of queries associated with the dashboard device as a real-time stream of events that indicate incremental, real-time changes to the enterprise data in response to the enterprise activities.

18. The system of claim 17, wherein the local storage schema of the local multidimensional data storage area of the network device comprises a snowflake schema.

19. The system of claim 17, wherein the dashboard device includes a graphical user interface (“

GUI”

) module to provide a dashboard to one or more users and to receive requests for particular views of the transactional data of the data warehouse from the one or more users.

20. The system of claim 19, wherein the data driver generates the second set of queries in accordance with the requests from the one or more users.

21. The system of claim 17, wherein the data warehouse comprises an SAP®

Business Information Warehouse (“

SAP BW”

).

22. The system of claim 21, wherein the multidimensional query processor comprises a Multidimensional Expressions (“

MDX”

) processor.

23. The system of claim 17, wherein the additional interface provided by the dynamic query engine is a Structured Query Language (“

SQL”

) interface.

24. A computer-readable storage medium comprising instructions for causing a programmable processor to:

receive, from a dashboard device, a unique identifier and a set of queries, wherein the queries do not conform to a multidimensional data query language;

store the unique identifier and the set of queries in a data table;

retrieve metadata from a multidimensional data storage area of a data warehouse, wherein the metadata defines a data storage schema of a multidimensional data cube of the multidimensional data storage area, and wherein the instructions that cause the processor to retrieve the metadata from the multidimensional data storage area cause the processor to retrieve the metadata without retrieving multidimensional data of the multidimensional data cube from the multidimensional data storage area, and wherein the multidimensional data of the multidimensional data storage area are generated from transactional data of a transactional data storage area of the data warehouse, and wherein the data warehouse comprises a data warehouse for an enterprise software system;

transmit the metadata from the multidimensional data storage area to the dashboard device via a gateway server using a chunking protocol operating at an application layer (layer seven) of the Open Systems Interconnection (“

OSI”

) networking model, wherein the gateway server streams the retrieved transactional data as a real-time stream of events that indicate incremental changes to the transactional data, and wherein the stream of events represents real-time changes to the enterprise data in response to the enterprise activities;

retrieve, by bypassing the multidimensional data storage area, transactional data directly from the transactional data storage area of the data warehouse, wherein the retrieved transactional data comprises transactional data from which the multidimensional data of the multidimensional data cube are generated, wherein the transactional data comprises enterprise data representing enterprise activities; and

transmit the retrieved transactional data to the dashboard device using the chunking protocol.