Transforming data structures and data objects for migrating data between databases having different schemas

• US 10,540,366 B2
• Filed: 03/09/2017
• Issued: 01/21/2020
• Est. Priority Date: 03/09/2017
• Status: Active Grant

• Current Assignee: Bank of America Corp.
• Original Assignee: Bank of America Corp.
• Inventors: Huang, Tao, Jana, Sriharsha
• Primary Examiner(s): Wu, Yicun
• Application Number:
  US15/454,471
• Publication Number:
  US 20180260458A1
• Time in Patent Office: 1,048 Days
• CPC Class Codes: G06F 16/214 Database migration support
• CPC Class Codes: G06F 16/24554 Unary operations; Data partitioning operations
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database

Abstract

Aspects of the disclosure relate to transforming data structures and data objects. A computing platform may query a logical view of a data table associated with a first database maintained by a first database server in a first schema and may receive source data associated with the logical view. Subsequently, the computing platform may split the source data into a plurality of data chunks associated with the logical view. Next, the computing platform may move the plurality of data chunks to a plurality of nodes configured to receive and transform the plurality of data chunks from the first schema to a second schema different from the first schema. Then, the computing platform may command the plurality of nodes to transform the plurality of data chunks and may move the transformed data from the plurality of nodes to a second database maintained by a second database server in the second schema.

Claims

1. A computing platform, comprising:

at least one processor;

a communication interface communicatively coupled to the at least one processor; and

memory storing computer-readable instructions that, when executed by the at least one processor, cause the computing platform to;

query a logical view of a data table associated with a first database maintained by a first database server in a first schema;

responsive to querying the logical view of the data table associated with the first database maintained by the first database server in the first schema, receive, via the communication interface, from the first database server, source data associated with the logical view;

split the source data associated with the logical view to produce a plurality of data chunks associated with the logical view;

move the plurality of data chunks associated with the logical view to a plurality of nodes associated with the computing platform, wherein each node of the plurality of nodes associated with the computing platform is configured to receive a particular data chunk of the plurality of data chunks associated with the logical view and transform the particular data chunk of the plurality of data chunks associated with the logical view from the first schema to a second schema different from the first schema;

command the plurality of nodes associated with the computing platform to transform the plurality of data chunks associated with the logical view to produce transformed data; and

move the transformed data from the plurality of nodes associated with the computing platform to a second database maintained by a second database server in the second schema,

wherein splitting the source data associated with the logical view to produce the plurality of data chunks associated with the logical view comprises identifying one or more available nodes and determining a uniform size of the plurality of data chunks associated with the logical view based on the one or more available nodes,

wherein a numerical column included in the logical view of the data table associated with the first database maintained by the first database server provides an index indicative of a total size of the source data associated with the logical view, and

wherein splitting the source data associated with the logical view to produce the plurality of data chunks associated with the logical view comprises splitting the source data associated with the logical view based on the numerical column to balance processing load across the one or more available nodes.

2. The computing platform of claim 1, wherein the memory stores additional computer-readable instructions that, when executed by the at least one processor, cause the computing platform to:

prior to querying the logical view of the data table associated with the first database maintained by the first database server in the first schema;

receive, via the communication interface, from an administrative computing device, a data migration command;

responsive to receiving the data migration command from the administrative computing device, generate a view command directing the first database server to create the logical view of the data table associated with the first database maintained by the first database server; and

send, via the communication interface, to the first database server, the view command directing the first database server to create the logical view of the data table associated with the first database maintained by the first database server.

3. The computing platform of claim 2, wherein the view command directing the first database server to create the logical view of the data table associated with the first database maintained by the first database server is configured to cause the first database server to execute a script that creates the logical view of the data table associated with the first database maintained by the first database server.

4. The computing platform of claim 1, wherein the first schema is a relational database schema.

5. The computing platform of claim 4, wherein the second schema is a non-relational database schema.

6. The computing platform of claim 1, wherein the plurality of nodes associated with the computing platform are configured to process the plurality of data chunks associated with the logical view in parallel.

7. The computing platform of claim 1, wherein commanding the plurality of nodes associated with the computing platform to transform the plurality of data chunks associated with the logical view to produce the transformed data comprises initiating a data transformation process at the plurality of nodes associated with the computing platform and directing the plurality of nodes associated with the computing platform to transfer the transformed data to the second database maintained by the second database server when the data transformation process is completed.

8. The computing platform of claim 1, wherein moving the transformed data from the plurality of nodes associated with the computing platform to the second database maintained by the second database server comprises moving the transformed data from the plurality of nodes associated with the computing platform to the second database maintained by the second database server using at least one application programming interface function call provided by the second database server.

9. A method, comprising:

at a computing platform comprising at least one processor, memory, and a communication interface;

querying, by the at least one processor, a logical view of a data table associated with a first database maintained by a first database server in a first schema;

responsive to querying the logical view of the data table associated with the first database maintained by the first database server in the first schema, receiving, by the at least one processor, via the communication interface, from the first database server, source data associated with the logical view;

splitting, by the at least one processor, the source data associated with the logical view to produce a plurality of data chunks associated with the logical view;

moving, by the at least one processor, the plurality of data chunks associated with the logical view to a plurality of nodes associated with the computing platform, wherein each node of the plurality of nodes associated with the computing platform is configured to receive a particular data chunk of the plurality of data chunks associated with the logical view and transform the particular data chunk of the plurality of data chunks associated with the logical view from the first schema to a second schema different from the first schema;

commanding, by the at least one processor, the plurality of nodes associated with the computing platform to transform the plurality of data chunks associated with the logical view to produce transformed data; and

moving, by the at least one processor, the transformed data from the plurality of nodes associated with the computing platform to a second database maintained by a second database server in the second schema,

wherein splitting the source data associated with the logical view to produce the plurality of data chunks associated with the logical view comprises identifying one or more available nodes and determining a uniform size of the plurality of data chunks associated with the logical view based on the one or more available nodes,

wherein a numerical column included in the logical view of the data table associated with the first database maintained by the first database server provides an index indicative of a total size of the source data associated with the logical view, and

wherein splitting the source data associated with the logical view to produce the plurality of data chunks associated with the logical view comprises splitting the source data associated with the logical view based on the numerical column to balance processing load across the one or more available nodes.

10. The method of claim 9, comprising:

prior to querying the logical view of the data table associated with the first database maintained by the first database server in the first schema;

receiving, by the at least one processor, via the communication interface, from an administrative computing device, a data migration command;

responsive to receiving the data migration command from the administrative computing device, generating, by the at least one processor, a view command directing the first database server to create the logical view of the data table associated with the first database maintained by the first database server; and

sending, by the at least one processor, via the communication interface, to the first database server, the view command directing the first database server to create the logical view of the data table associated with the first database maintained by the first database server.

11. The method of claim 10, wherein the view command directing the first database server to create the logical view of the data table associated with the first database maintained by the first database server is configured to cause the first database server to execute a script that creates the logical view of the data table associated with the first database maintained by the first database server.

12. The method of claim 9, wherein the first schema is a relational database schema.

13. The method of claim 12, wherein the second schema is a non-relational database schema.

15. The method of claim 9, wherein the plurality of nodes associated with the computing platform are configured to process the plurality of data chunks associated with the logical view in parallel.

16. The method of claim 9, wherein commanding the plurality of nodes associated with the computing platform to transform the plurality of data chunks associated with the logical view to produce the transformed data comprises initiating a data transformation process at the plurality of nodes associated with the computing platform and directing the plurality of nodes associated with the computing platform to transfer the transformed data to the second database maintained by the second database server when the data transformation process is completed.

17. The method of claim 9, wherein moving the transformed data from the plurality of nodes associated with the computing platform to the second database maintained by the second database server comprises moving the transformed data from the plurality of nodes associated with the computing platform to the second database maintained by the second database server using at least one application programming interface function call provided by the second database server.

14. One or more non-transitory computer-readable media storing instructions that, when executed by a computing platform comprising at least one processor, memory, and a communication interface, cause the computing platform to:

query a logical view of a data table associated with a first database maintained by a first database server in a first schema;

responsive to querying the logical view of the data table associated with the first database maintained by the first database server in the first schema, receive, via the communication interface, from the first database server, source data associated with the logical view;

split the source data associated with the logical view to produce a plurality of data chunks associated with the logical view;

move the plurality of data chunks associated with the logical view to a plurality of nodes associated with the computing platform, wherein each node of the plurality of nodes associated with the computing platform is configured to receive a particular data chunk of the plurality of data chunks associated with the logical view and transform the particular data chunk of the plurality of data chunks associated with the logical view from the first schema to a second schema different from the first schema;

command the plurality of nodes associated with the computing platform to transform the plurality of data chunks associated with the logical view to produce transformed data; and

move the transformed data from the plurality of nodes associated with the computing platform to a second database maintained by a second database server in the second schema,

wherein splitting the source data associated with the logical view to produce the plurality of data chunks associated with the logical view comprises identifying one or more available nodes and determining a uniform size of the plurality of data chunks associated with the logical view based on the one or more available nodes,

wherein a numerical column included in the logical view of the data table associated with the first database maintained by the first database server provides an index indicative of a total size of the source data associated with the logical view, and

wherein splitting the source data associated with the logical view to produce the plurality of data chunks associated with the logical view comprises splitting the source data associated with the logical view based on the numerical column to balance processing load across the one or more available nodes.

18. The one or more non-transitory computer-readable media of claim 14, storing additional instructions that, when executed by the computing platform, cause the computing platform to:

prior to querying the logical view of the data table associated with the first database maintained by the first database server in the first schema;

receive, via the communication interface, from an administrative computing device, a data migration command;

responsive to receiving the data migration command from the administrative computing device, generate a view command directing the first database server to create the logical view of the data table associated with the first database maintained by the first database server; and

send, via the communication interface, to the first database server, the view command directing the first database server to create the logical view of the data table associated with the first database maintained by the first database server.

19. The one or more non-transitory computer-readable media of claim 18, wherein the view command directing the first database server to create the logical view of the data table associated with the first database maintained by the first database server is configured to cause the first database server to execute a script that creates the logical view of the data table associated with the first database maintained by the first database server.

20. The one or more non-transitory computer-readable media of claim 14, wherein the first schema is a relational database schema, and wherein the second schema is a non-relational database schema.