Digital data processing system for efficiently storing, moving, and/or processing data across a plurality of computing clusters

• US 10,678,936 B2
• Filed: 12/01/2017
• Issued: 06/09/2020
• Est. Priority Date: 12/01/2017
• Status: Active Grant

• Current Assignee: Bank of America Corp.
• Original Assignee: Bank of America Corp.
• Inventors: Yarlagadda, Sitaram C., Anusuri, Vijaya M.
• Primary Examiner(s): Doan, Huan V
• Application Number:
  US15/828,876
• Publication Number:
  US 20190171828A1
• Time in Patent Office: 921 Days
• CPC Class Codes: G06F 21/31 User authentication
• CPC Class Codes: G06F 21/6209 to a single file or object, e.g. in a secure envelope, encrypted and accessed using a key, or with access control rules appended to the object itself
• CPC Class Codes: H04L 63/08 for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal

Abstract

A method, apparatus, computer-readable medium, and/or system described herein may be used to efficiently store, move, and/or process data across a plurality of computing clusters. For example, a computing device may receive an indication of one or more data storage locations within a first cluster of servers and/or an indication of one or more data storage locations within a second cluster of servers. The computing device may generate a data file comprising the indication of the one or more data storage locations within the first cluster of servers and/or the indication of one or more data storage locations within the second cluster of servers. Based on the generated data file, the computing device may generate a job to move data stored at the one or more data storage locations within the first cluster of servers to the one or more data storage locations within the second cluster of servers. Based on the job, the computing device may transmit, e.g., to the first cluster of servers and/or the second cluster of servers, instructions to move data stored at the one or more data storage locations within the first cluster of servers to the one or more data storage locations within the second cluster of servers.

Claims

1. A system comprising:

a computing device comprising;

one or more processors; and

memory storing computer-executable instructions that, when executed by the one or more processors of the computing device, cause the computing device to;

authenticate a user of a user device;

receive, from the user device, an indication of one or more data storage locations within a first cluster of servers;

determine, based on the one or more data storage locations within the first cluster of servers, available environments to which the data can be moved, wherein the available environments comprise a plurality of clusters of servers;

transmit, to the user device, an indication of the plurality of clusters of servers to which the data can be moved, wherein the indication of the plurality of clusters of servers to which the data can be moved comprises a destination file at the destination for which to move the data;

receive, from the user device, an indication of one or more data storage locations within a second cluster of servers of the plurality of clusters of servers;

determine whether the user of the user device is authorized to access one or more of the first cluster of servers or the second cluster of servers,

generate a data file comprising the indication of the one or more data storage locations within the first cluster of servers and the indication of one or more data storage locations within the second cluster of servers, wherein generating the data file is performed after authenticating the user and after determining that the user is authorized to access one or more of the first cluster of servers or the second cluster of servers;

based on the generated data file, generate a job to move data stored at the one or more data storage locations within the first cluster of servers to the one or more data storage locations within the second cluster of servers; and

based on the job, transmit, to the first cluster of servers and the second cluster of servers, instructions to move data stored at the one or more data storage locations within the first cluster of servers to the one or more data storage locations within the second cluster of servers;

the first cluster of servers comprising;

one or more processors; and

memory storing computer-executable instructions that, when executed by the one or more processors of the first cluster of servers, cause the first cluster of servers to;

receive, from the computing device, the instructions to move the data;

based on the instructions, identify the data stored at the one or more data storage locations within the first cluster of servers; and

initiate transfer of the data stored at the one or more data storage locations within the first cluster of servers to the one or more data storage locations within the second cluster of servers; and

the second cluster of servers comprising;

one or more processors; and

memory storing computer-executable instructions that, when executed by the one or more processors of the second cluster of servers, cause the second cluster of servers to;

receive, from the computing device, the instructions to move the data;

based on the instructions, identify the one or more data storage locations within the second cluster of servers; and

store, at the one or more data storage locations within the second cluster of servers, the data transferred from the first cluster of servers.

2. The system of claim 1, wherein the memory of the computing device stores computer-executable instructions that, when executed by the one or more processors of the computing device, cause the computing device to:

determine a format of the data stored at the one or more data storage locations,

wherein generating the data file comprises generating the data file comprising the indication of the one or more data storage locations within the first cluster of servers, the indication of the one or more data storage locations within the second cluster of servers, and the format of the data.

3. The system of claim 1, wherein the one or more data storage locations within the first cluster of servers comprises one or more files within the first cluster of servers, and wherein the one or more data storage locations within the second cluster of servers comprises one or more files within the second cluster of servers.

4. The system of claim 1, wherein receiving the indication of one or more data storage locations within the first cluster of servers comprises receiving, from the user device, a plurality of source paths associated with the one or more data storage locations within the first cluster of servers, and wherein receiving the indication of one or more data storage locations within the second cluster of servers comprises receiving, from the user device, a plurality of destination paths associated with the one or more data storage locations within the second cluster of servers.

5. The system of claim 1, wherein the memory of the computing device stores computer-executable instructions that, when executed by the one or more processors of the computing device, cause the computing device to:

generate one or more identifiers for the job to move data stored at the one or more data storage locations within the first cluster of servers to the one or more data storage locations within the second cluster of servers; and

transmit, to the user device, the one or more identifiers.

6. The system of claim 1, wherein generating the job is based on a data type of the data stored at the one or more data storage locations within the first cluster of servers.

7. The system of claim 1, wherein the first cluster of servers runs in a higher level environment, and wherein the second cluster of servers runs in a lower level environment.

8. The system of claim 7, wherein the higher level environment comprises a production environment, and wherein the lower level environment comprises one or more of an acceptance testing environment, a development environment, or a system integration testing environment.

9. The system of claim 1, wherein the memory of the computing device stores computer-executable instructions that, when executed by the one or more processors of the computing device, cause the computing device to:

based on the generated data file, generate parallel jobs to simultaneously move data stored at the one or more data storage locations within the first cluster of servers to the one or more data storage locations within the second cluster of servers.

10. A method comprising:

authenticating a user of a user device;

receiving, by a computing device and from the user device, an indication of one or more data storage locations within a first cluster of servers;

determining, based on the one or more data storage locations within the first cluster of servers, available environments to which the data can be moved, wherein the available environments comprise a plurality of clusters of servers;

transmitting, to the user device, an indication of the plurality of clusters of servers to which the data can be moved, wherein the indication of the plurality of clusters of servers to which the data can be moved comprises a destination folder at the destination for which to move the data;

receiving, by the computing device and from the user device, an indication of one or more data storage locations within a second cluster of servers of the plurality of clusters of servers;

determining whether the user of the user device is authorized to access one or more of the first cluster of servers or the second cluster of servers;

generating a data file comprising the indication of the one or more data storage locations within the first cluster of servers and the indication of one or more data storage locations within the second cluster of servers, wherein generating the data file is performed after authenticating the user and after determining that the user is authorized to access one or more of the first cluster of servers or the second cluster of servers;

based on the generated data file, generating a job to move data stored at the one or more data storage locations within the first cluster of servers to the one or more data storage locations within the second cluster of servers; and

based on the job, transmitting, by the computing device and to the first cluster of servers and the second cluster of servers, instructions to move data stored at the one or more data storage locations within the first cluster of servers to the one or more data storage locations within the second cluster of servers.

11. The method of claim 10, further comprising:

determining a format of the data stored at the one or more data storage locations,

wherein generating the data file comprises generating the data file comprising the indication of the one or more data storage locations within the first cluster of servers, the indication of the one or more data storage locations within the second cluster of servers, and the format of the data.

12. The method of claim 10, wherein the one or more data storage locations within the first cluster of servers comprises one or more files within the first cluster of servers, and wherein the one or more data storage locations within the second cluster of servers comprises one or more files within the second cluster of servers.

13. The method of claim 10, wherein receiving the indication of one or more data storage locations within the first cluster of servers comprises receiving, from the user device, a plurality of source paths associated with the one or more data storage locations within the first cluster of servers, and wherein receiving the indication of one or more data storage locations within the second cluster of servers comprises receiving, from the user device, a plurality of destination paths associated with the one or more data storage locations within the second cluster of servers.

14. The method of claim 10, further comprising:

generating one or more identifiers for the job to move data stored at the one or more data storage locations within the first cluster of servers to the one or more data storage locations within the second cluster of servers; and

transmitting, from the computing device and to the user device, the one or more identifiers.

15. The method of claim 10, wherein generating the job is based on a data type of the data stored at the one or more data storage locations within the first cluster of servers.

16. The method of claim 10, further comprising:

based on the generated data file, generating parallel jobs to simultaneously move data stored at the one or more data storage locations within the first cluster of servers to the one or more data storage locations within the second cluster of servers.

17. A non-transitory computer readable medium storing instructions thereon that, when read by a computing device, causes the computing device to:

authenticate a user of a user device;

receive, from the user device, an indication of one or more data storage locations within a first cluster of servers;

determine, based on the one or more data storage locations within the first cluster of servers, available environments to which the data can be moved, wherein the available environments comprise a plurality of clusters of servers;

transmit, to the user device, an indication of the plurality of clusters of servers to which the data can be moved, wherein the indication of the plurality of clusters of servers to which the data can be moved comprises a file or destination folder at the destination for which to move the data;

receive, from the user device, an indication of one or more data storage locations within a second cluster of servers of the plurality of clusters of servers;

determine whether the user of the user device is authorized to access one or more of the first cluster of servers and the second cluster of servers;

generate a data file comprising the indication of the one or more data storage locations within the first cluster of servers and the indication of one or more data storage locations within the second cluster of servers, wherein generating the data file is performed after authenticating the user and after determining that the user is authorized to access one or more of the first cluster of servers and the second cluster of servers;

based on the generated data file, generate a job to move data stored at the one or more data storage locations within the first cluster of servers to the one or more data storage locations within the second cluster of servers; and

based on the job, transmit, to the first cluster of servers and the second cluster of servers, instructions to move data stored at the one or more data storage locations within the first cluster of servers to the one or more data storage locations within the second cluster of servers.