Job scheduling for automatic movement of multidimensional data between live datacubes

• US 7,877,355 B2
• Filed: 09/13/2007
• Issued: 01/25/2011
• Est. Priority Date: 04/12/2005
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bowen, David S., Borgsmidt, Rasmus
• Primary Examiner(s): Breene; John E
• Assistant Examiner(s): Hershley; Mark E
• Application Number:
  US11/900,895
• Publication Number:
  US 20080077621A1
• Time in Patent Office: 1,230 Days
• Field of Search: 707/104.1, 707/999.104, 707/610, 707/616, 707/624, 707/625, 707/637, 707/638
• US Class Current: 707/616
• CPC Class Codes: G06F 16/283 Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP

Abstract

Techniques are described for sharing multidimensional data between software systems, such as enterprise software systems. Specifically, the techniques provide mechanisms for defining inter-application “links” for automatically moving data among different databases associated with the enterprise software systems. For example, a system may include a first database storing multidimensional data for a first software application, and a second database storing multidimensional data for a second software application. The system further includes an interface by which an administrator defines a link that specifies a source area of the first database and a target area of the second database. A link control module automatically moves multidimensional data from the source area to the target area in accordance with the link. The link control module may automatically move the multidimensional data directly between live portions of the respective databases.

Claims

1. A method for exporting data from a software system comprising:

defining a link from a source model of a first software application to a target model of a second software application, wherein the link specifies a source area of a multidimensional database comprising one or more multidimensional data cubes defined by the source model, the source area being associated with the first software application, and a target area of a multidimensional database comprising one or more multidimensional data cubes defined by the target model, the target area being associated with the second software application, and wherein the link defines a mapping between source items in the one or more multidimensional data cubes of the source model and target items in the one or more multidimensional data cubes of the target model;

automatically formulating, by one or more computing devices, a plurality of work elements that, when executed, move multidimensional data from the source area to the target area in accordance with the link, wherein automatically formulating the plurality of work elements comprises;

identifying groups of the target items that the mapping maps to groups of the source items;

determining overlaps of items between the groups of the source items specified by the mapping, wherein the overlaps correspond to source items that are common to two or more of the groups of the source items; and

formulating the plurality of work elements such that each of the work elements specifics one of the groups of the target items and a corresponding one of the groups of the source items based on the mapping, and such that each of the items in at least one of the overlaps of items is present in only one of the work elements; and

executing the work elements.

2. The method of claim 1, wherein identifying the groups of the target items further comprises:

estimating a number of available processors that are configured to execute the work elements; and

combining two or more of the identified groups of target items into a single group when the identified groups of target items outnumber the available processors.

3. The method of claim 1, wherein identifying the groups of the target items comprises dividing one of the identified groups of target items into two or more groups when the one of the identified groups of target items includes more target items than a maximum number of target items allowed in a group.

4. The method of claim 1, wherein identifying the groups of target items comprises dividing one of the identified groups of target items into two or more groups of target items when there are fewer of the identified groups than there are processors configured to execute the work elements.

5. The method of claim 1, wherein executing the work elements comprises:

loading that portion of the multidimensional data that is associated with the source items specified in one of the work elements from the multidimensional data cubes;

generating a data block based on the data associated with the source items; and

integrating the data block into the target model.

6. The method of claim 5,

wherein the method further comprises defining at least one data transformation;

and

wherein generating the data block comprises applying the transformation to the data associated with the source items in order to generate the data block.

7. The method of claim 1, wherein executing the work elements comprises processing a first one of the work elements as a plurality of batches when the first one of the work elements specifies more source items than a maximum permitted number of source items.

8. The method of claim 7, wherein the maximum permitted number of source items is set by an administrator.

9. The method of claim 7, wherein processing the first one of the work elements as a plurality of batches comprises:

loading multidimensional data associated with source items in a first one of the batches;

invoking a link engine that generates a data block based on the loaded multidimensional data associated with the source items in the first one of the batches; and

loading multidimensional data associated with source items in a second one of the batches after invoking the link engine.

10. The method of claim 1, wherein defining the mapping comprises defining the mapping to map a plurality of the source items to one of the target items.

11. The method of claim 1, wherein defining the mapping comprises defining the mapping to map a plurality of the source items to a plurality of the target items.

12. The method of claim 1, further comprising:

downloading multidimensional planning data to a client device of a user;

automatically moving the multidimensional data from the source area to the planning data upon manual invocation of the link by the user; and

saving the planning data to the target area from the client device.

13. The method of claim 1, wherein automatically moving multidimensional data comprises exporting a consistent set of data from the source area.

14. A system comprising:

a first database storing multidimensional data for a first software application;

a second database storing multidimensional data for a second software application;

an interface that receives input that defines a link specifying a source area of the first database and a target area of the second database, wherein the link defines a mapping between source items in one or more multidimensional data cubes of a source model for the first database and target items in one or more multidimensional data cubes of a target model for the second database;

a link control module that automatically formulates a plurality of work elements that, when executed, move multidimensional data from the source area to the target area in accordance with the link, wherein to formulate the plurality of work elements, the link control module identifies groups of the target items that the mapping maps to groups of the source items, determines overlaps of items between the groups of the source items specified by the mapping, wherein the overlaps correspond to source items that are common to two or more of the groups of the source items, and formulates the plurality of work elements such that each of the work elements specifies one of the groups of the target items and a corresponding one of the groups of the source items based on the mapping, and such that each of the items in at least one of the overlaps of items is present in only one of the work elements; and

a set of one or more processors that execute the plurality of work elements.

15. The system of claim 14,

wherein the interface includes input areas by which an administrator specifies the source model in the first database and the target model in the second database, and

wherein the interface includes input areas by which the administrator specifies the mapping between the source items in the one or more multidimensional data cubes of the source model and the target items in the one or more multidimensional data cubes of the target model.

16. The system of claim 14,

wherein the interface includes input areas by which an administrator defines the mapping.

17. The system of claim 14, wherein the job initialization process identifies the groups of the target items in part by dividing one or more of the identified groups into two or more groups of the target items when the one of the identified groups includes more of the target items than a maximum number of items in the target dimension permitted in a group of the target items.

18. The system of claim 14, wherein the job initialization process identifies the groups of the target items in part by dividing one of the identified groups into two or more groups when there are fewer of the identified groups than there are processors in the system configured to execute the work elements.

19. The system of claim 14, wherein the job initialization process identifies the groups of the target items in part by combining two or more of the identified groups where there are more of the identified groups than there are processors in the system configured to execute the work elements.

20. The system of claim 14,

wherein when the set of one or more processors executes one of the work elements, the set of one or more processors executes a job module that loads that portion of the multidimensional data that is associated with each of the source items that are specified in the one of the work elements,

wherein the set of one or more processors executes a link engine that outputs a data block based on the multidimensional data that is associated with the source items specified in the one of the work elements,

wherein the system further comprises an import queue to integrate the data block into the target model.

21. The system of claim 20,

wherein the interlace includes input areas by which an administrator defines at least one data transformation for an item in the target dimension, and

wherein when the link engine generates the data block, the link engine applies the transformation to the multidimensional data in order to output the data block.

22. The system of claim 20, wherein the job module processes a first one of the work elements as a plurality of batches when the first one of the work elements specifics more source items than a maximum permitted number of source items.

23. The system of claim 22, wherein the interface includes an input field by which an administrator enters the maximum permitted number of source items.

24. The system of claim 22,

wherein the job module loads from the first database multidimensional data associated with source items in a first one of the batches and invokes a link engine that generates a data block based on the loaded multidimensional data associated with the first one of the batches;

wherein, after invoking the link engine, the job module loads from the first database multidimensional data associated with the source items in a second one of the batches.

25. The system of claim 14, wherein the interface includes input areas by which an administrator defines the mapping to map a plurality of the source items to one of the target items.

26. The system of claim 14, wherein the interface includes input areas by which an administrator defines the mapping to map a plurality of the source items to a plurality of the target items.

27. The system of claim 14, further comprising:

a client device providing an operating environment for a planning software application,

wherein the planning software application downloads user-specific planning data from the second database to the client device, automatically retrieves the multidimensional data from the source area of the first database upon manual invocation of the link by a user, and saves the user-specific planning data and the multidimensional data to the target area from the client device.

28. A computer-readable storage medium comprising instructions that, when executed, cause a processor to:

present a user interface to receive input that specifies a mapping from one or more source items in one or more multidimensional data cubes of a source model to one or more target items in one or more multidimensional data cubes of a target model;

automatically formulate a plurality of work elements that, when executed, move multidimensional data in the source model that are associated with the source items of the source data cube to the target items of the target model wherein the instructions that cause the processor to automatically formulate the plurality of work elements comprise instructions that cause the processor to;

identify groups of the target items that the mapping maps to groups of the source items;

determine overlaps of items between the groups of the source items specified by the mapping, wherein the overlaps correspond to source items that are common to two or more of the groups of the source items; and

formulate the plurality of work elements such that each of the work elements specifies one of the groups of the target items and a corresponding one of the groups of the source items based on the mapping, and such that each of the items in at least one of the overlaps of items is present in only one of the work elements; and

cause one or more other processors to execute the plurality of work elements.

29. The computer-readable storage medium of claim 28, wherein the instructions that cause the processor to present the interface cause the processor to:

present input fields by which a user identities a source data cube in the source model and a target data cube in the target model;

present input fields by which the user identities a source dimension of the source data cube and a target dimension of the target data cube; and

present input fields by which the user defines the mapping between the source items of the source dimension and the target items of the target dimension.