SYSTEM AND METHOD FOR FLEXIBLE DATA TRANSFER

• US 20130074079A1
• Filed: 11/13/2012
• Published: 03/21/2013
• Est. Priority Date: 07/30/2004
• Status: Active Grant

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Clark, Nicholas James, Woods, William S.
• Granted Patent: US 8,918,524 B2
• Time in Patent Office: Days
• US Class Current: 718/101
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems
• CPC Class Codes: Y10S 707/99953 Recoverability
• CPC Class Codes: Y10S 707/99954 Version management

Abstract

A method and system for flexibly transferring data from one or more data sources to one or more data destinations within an information network where each of the one or more data sources have data in a particular source format and each of the one or more data destinations have data in the same or another particular destination format using a parameter database that includes parameters to control the transfer of data, a scheduler that initiates the transfer of data, and a data loader in communications with the parameter database and scheduler that, upon initiation by the scheduler, extracts data from the one or more data sources, manipulates the extracted source data into one or more destination formats associated with the one or more data destinations, and inserts the data into one or more data destinations according to the parameters within the parameter database.

Claims

1-17. -17. (canceled)

18. A method, comprising:

assigning, by a data loader, batch numbers to a plurality of data loading operations;

launching concurrently a first subset of the plurality of data loading operations having a first batch number; and

responsive to detecting completion of the first subset of data loading operations, launching concurrently a second set of data loading operations having a second batch number.

19. The method of claim 18, wherein launching the first subset includes launching a plurality of threads concurrently wherein each of the plurality of threads is associated with a corresponding one of the first subset of data loading operations.

20. The method of claim 18, further comprising:

storing the batch numbers assigned in a parameter database; and

accessing the batch numbers stored in the parameter database to determine the first subset of the plurality of data loading operations.

21. The method of claim 18, wherein each of the plurality of data loading operations includes:

extracting source information from a data source;

manipulating the source information into destination information having a destination format; and

inserting the destination information into a data destination.

22. The method of claim 21, wherein manipulating the source information includes invoking plug-ins to manipulate the source information.

23. The method of claim 22, wherein invoking the plug-ins includes determining a name and location of a plug-in from a parameter database.

24. The method of claim 21, wherein manipulating the source information includes converting source information content based on at least one of a computation, an algorithm, and a formula.

25. A system, comprising:

a processor;

a computer readable medium, accessible to the processor, including stored program instructions that, when executed by a processor, perform program operations including;

assigning, by a data loader, batch numbers to a plurality of data loading operations;

launching concurrently a first subset of the plurality of data loading operations having a first batch number; and

responsive to detecting completion of the first subset of data loading operations, launching concurrently a second set of data loading operations having a second batch number.

26. The system of claim 25, wherein launching the first set includes launching a plurality of threads concurrently wherein each of the plurality of threads is associated with a corresponding one of the first subset of data loading operations.

27. The system of claim 25, wherein the program operations include:

storing the batch numbers assigned in a parameter database; and

accessing the batch numbers stored in the parameter database to determine the first subset of the plurality of data loading operations.

28. The system of claim 25, wherein each of the plurality of data loading operations includes:

extracting source information from a data source;

manipulating the source information into destination information having a destination format; and

inserting the destination information into a data destination.

29. The system of claim 28, wherein manipulating the source information includes invoking plug-ins to manipulate the source information.

30. The system of claim 29, wherein invoking the plug-ins includes determining a name and location of a plug-in from a parameter database.

31. The system of claim 28, wherein manipulating the source information includes converting source information content based on at least one of a computation, an algorithm, and a formula.

32. A computer readable storage medium including stored program instructions that, when executed by a processor, perform program operations, comprising:

assigning, by a data loader, batch numbers to a plurality of data loading operations;

launching concurrently a first subset of the plurality of data loading operations having a first batch number; and

responsive to detecting completion of the first subset of data loading operations, launching concurrently a second set of data loading operations having a second batch number.

33. The computer readable medium of claim 32, wherein launching the first subset includes launching a plurality of threads concurrently wherein each of the plurality of threads is associated with a corresponding one of the first subset of data loading operations.

34. The computer readable storage medium of claim 32, wherein the program operations include:

storing the batch numbers assigned in a parameter database; and

accessing the batch numbers stored in the parameter database to determine the first subset of the plurality of data loading operations.

35. The computer readable storage medium of claim 32, wherein each of the plurality of data loading operations includes:

extracting source information from a data source;

manipulating the source information into destination information having a destination format; and

inserting the destination information into a data destination.

36. The computer readable storage medium of claim 35, wherein manipulating the source information includes invoking plug-ins to manipulate the source information and wherein invoking the plug-ins includes determining a name and location of a plug-in from a parameter database.

37. The computer readable storage medium of claim 35, wherein manipulating the source information includes converting source information content based on at least one of a computation, an algorithm, and a formula.