Method and system for data dispatch

• US 7,865,460 B2
• Filed: 05/08/2008
• Issued: 01/04/2011
• Est. Priority Date: 05/08/2008
• Status: Expired due to Fees

• Current Assignee: LinkedIn Corporation (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Stahl, Elisabeth R., Salve, Vivek, Hamilton, Rick A. II, Sand, Anne R., Li, Jenny S.
• Primary Examiner(s): Lee; Wilson
• Assistant Examiner(s): Nguyen; Merilyn P
• Application Number:
  US12/117,285
• Publication Number:
  US 20090282090A1
• Time in Patent Office: 971 Days
• Field of Search: 707/609, 707/602, 707/636, 707/694, 707/809, 707/812
• US Class Current: 707/609
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

A method and system for migrating source data from one or more databases to a destination database, wherein the destination database is selected based on power consumption of the destination database. A data migration server determines which destination database should be selected by selecting a number of candidates and comparing the power consumed, the available space and the maximum monthly power consumption limit. A user intervention policy is created to evaluate which data should be moved to a destination database. A “payback period” is calculated to determine the amount of time that will elapse before savings are realized.

Claims

1. A method for dispatching incoming source data from one or more source databases to one or more destination databases, wherein the source and destination databases carry out one or more transactions, comprising:

receiving first, second or more incoming sets of source data from one or more source databases into a dispatch server;

determining a plurality of candidates for the one or more destination databases;

evaluating the plurality of candidates for the destination database by determining average power consumption per transaction of each candidate;

selecting a first destination database candidate having lowest power consumption per transaction;

determining if the selected first destination database candidate has available space for the first, second or more incoming sets of source data;

if space is available, determining if the selected first destination database candidate meets predefined criteria;

if the predefined criteria are met, dispatching the first, second or more incoming sets of source data to the selected first destination database candidate;

continuing to receive new sets of incoming source data; and

repeating the method for each new set of incoming source data.

2. The method of claim 1 further comprising:

removing the selected first destination database candidate from the plurality of candidates if the selected first destination database candidate does not have available space or predefined criteria are no longer met;

selecting a second destination database candidate having next lowest power consumption per transaction;

determining if the selected second destination database candidate has available space for the source data;

if space is available, determining if the selected second destination database candidate meets predefined criteria; and

if the predefined criteria are met, dispatching the source data to the selected second destination database candidate.

3. The method of claim 2 further comprising:

removing the selected second destination database candidate from the plurality of candidates if the selected second destination database candidate does not have available space or predefined criteria are no longer met;

continuing the method with a third, fourth or more destination database candidates until a third, forth or more selected destination database candidate has available space for the source data; and

if space is available, determining if the selected third, fourth or more destination database candidate meets predefined criteria; and

if the predefined criteria are met, dispatching the source data to the selected third, forth or more destination database candidate.

4. The method of claim 1 further comprising:

identifying the first, second or more incoming sets of source data as (i) real time data;

(ii) near-real-time data;

or (iii) non-real-time data;

determining if the first, second or more incoming sets of source data should be dispatched immediately to a final destination database or be dispatched to a temporary storage location; and

determining duration of time and a temporary storage location if the data should not be dispatched immediately.

5. The method of claim 1 further comprising:

setting a threshold for a total cost of dispatch of the incoming sets of source data;

identifying a second cost of dispatch of the incoming sets of source data to each candidate destination;

determining if the second cost of dispatch to the candidate destination exceeds the threshold of the total cost of dispatch for the incoming sets of source data;

eliminating the candidate destination if the second cost of dispatch exceeds the threshold of the total cost of dispatch for the incoming sets of source data.

6. The method of claim 1 further comprising;

creating an index for each set of source data to respective destination database selected therefore for future retrieval.

7. The method of claim 1 further comprising:

predicting the receipt of future incoming data to the dispatch server;

including the prediction of the future incoming data in the selection process of the selecting the first destination database candidate;

selecting an alternate destination database for the first, second or more source data if the prediction of the future incoming data requires dispatch to the first selected destination database candidate.

8. The method of claim 1 further comprising:

prior to receiving incoming data, proactively checking the plurality of destination database candidates for available space;

determining the candidates having the lowest power consumption per transaction; and

requesting an increase in available space in the candidates having the lowest power consumption per transaction.

9. The method of claim 3 further comprising:

wherein one or more of the each new set of incoming source data belongs with first, second or more incoming source data;

determining if there is available space in the first, second, third, fourth or more destination database where the first, second or more incoming source data is located;

dispatching the one or more new sets of incoming source data to the location of the first, second or more incoming data if space is available;

if space is not available, moving the one or more new sets of incoming source data to the next available destination database candidate having the next lowest power consumption;

creating an index for each of the first, second, or more incoming source data and the one or more of each new set of incoming data of the selected destination database for future retrieval of all data that belongs together.

10. The method of claim 1 further comprising:

if the selected first destination database candidate does not have available space for the first, second, or more incoming source data;

segregating the first, second or more incoming data into one or more segregated data sets;

moving as many as the one or more segregated data sets to the selected first destination database candidate as can fit;

leaving in the dispatch server the one or more segregated data sets that cannot fit in the selected first destination database;

moving the one or more segregated data sets left in the dispatch server to one or more next available destination database candidates having the next lowest power consumption.

11. A system for dispatching incoming data from one or more source databases to one or more destination databases, wherein the source and destination databases carry out one or more transactions, comprising:

one or more source databases;

a plurality of target databases;

a data dispatch server having a decision system, an incoming data receipt facility and a temporary storage device;

wherein the incoming data is received by the incoming data receipt facility and is moved to the temporary storage device;

wherein the decision system evaluates the target databases for power consumption properties and selects first most efficient power consumption target database;

wherein the decision system determines if the first most efficient power consumption target database has available space;

wherein the decision system determines if the first most efficient power consumption target database meets predefined criteria;

wherein the decision system dispatches the incoming source data from the temporary storage device to the first most efficient power consumption target database if there is available space and the predefined criteria are met.

12. The system of claim 11 wherein if the first efficient power consumption target database does not have available space for the incoming source data and/or does not meet the predefined criteria, the decision system determines second most efficient power consumption target database and checks for available space and predefined criteria requirements;

and moves to the second most efficient power consumption target database.

13. The system of claim 12 wherein if the available space and predefined criteria requirements are not met, the system continues to evaluate the target databases for third, fourth, or more target databases, until the space availability and predefined criteria requirements are met.

14. The system of claim 13 wherein the data dispatch server creates an index of locations of all dispatched data for easy retrieval.

15. The system of claim 11 wherein the decision system identifies the incoming source data as (i) real time data;

(ii) near-real-time data;

or (iii) non-real-time data;

determines if the incoming source data should be dispatched immediately to a final destination database or be dispatched to a temporary storage location; and

determines duration of time and a temporary storage location if the data should not be dispatched immediately.

16. The system of claim 11 wherein the decision system sets a threshold for total cost of dispatching the incoming source data;

identifies a second cost of dispatching the incoming source data to each target destination;

determines if the second cost of dispatching to the target database exceeds the threshold of the total cost of dispatching for the incoming source data;

eliminates the target database if the second cost of dispatching exceeds the threshold of the total cost of dispatching for the incoming source data.

17. The system of claim 11 wherein the decision system predicts the receipt of future incoming data to the dispatch server;

includes the prediction of the future incoming data in the selection process of the selecting the first target database candidate;

selects an alternate target database for the source data if the prediction of the future incoming data requires dispatch to the first selected target database candidate.

18. The system of claim 11 wherein the decision system, prior to receiving incoming data, proactively checks the plurality of target databases for available space;

determines the target databases having the lowest power consumption per transaction; and

requests an increase in available space in the target databases having the lowest power consumption per transaction.

19. The system of claim 11 wherein new source data from the one or more source databases continues to enter the incoming data receipt facility;

wherein the decision system determines if the new source data belongs with incoming source data that has been moved to a target database;

wherein the decision system determines if there is available space in the target database wherein the incoming source data has been moved;

wherein the decision system dispatches new source data to the location of the moved source data if space is available;

wherein if space is not available, the decision system moves the new source data to the next available target database having the next lowest power consumption; and

wherein the data dispatch server creates an index for each new source data for future retrieval of all data that belongs together.

20. The system of claim 11 wherein if the selected first most efficient power consumption target database does not have available space for new source data;

the data dispatch server segregates the incoming source data into one or more segregated data sets, moves as many as the one or more segregated data sets to the selected first most efficient power consumption target database that can fit, leaves in the dispatch server the one or more segregated data sets that cannot fit in the selected first target database, and moves the one or more segregated data sets left in the dispatch server to one or more next available target databases having the next lowest power consumption.

21. A computer program product encoded in a computer readable storage medium for instructing a data dispatch server to dispatch incoming source data to a destination database comprising:

instructing the data dispatch sever to receive first, second or more incoming source data;

instructing the data dispatch server to determine a plurality of candidates for the destination database;

instructing the data dispatch server to evaluate the plurality of destination database candidates for power consumption properties to determine most efficient power consumption destination database;

instructing the data dispatch server to determine if the most efficient power consumption destination database has available space;

instructing the data dispatch server to determine if the most efficient power consumption destination database has met predefined criteria;

instructing the data dispatch server to dispatch data to the most efficient power consumption destination database if space is available and predefined criteria has been met.