Data locality in data integration applications

• US 9,607,062 B1
• Filed: 11/19/2015
• Issued: 03/28/2017
• Est. Priority Date: 11/19/2015
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bonagiri, Krishna K., Jacobson, Eric A.
• Primary Examiner(s): Arjomandi, Noosha
• Application Number:
  US14/945,476
• Time in Patent Office: 495 Days
• Field of Search: 707/602
• US Class Current: 1/1
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/183 Provision of network file services by network file servers, e.g. by using NFS, CIFS network file access protocols H04L67/1097
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A computer-implemented method includes identifying a logical node. The logical node is associated with one or more source stages. The computer-implemented method further includes identifying one or more file block components. The one or more file block components include a retrieval target for at least one of the one or more source stages. The computer-implemented method further includes identifying one or more physical nodes and determining, for each of the one or more physical nodes, a degree value. A corresponding computer program product and computer system are also disclosed.

Claims

1. A computer-implemented method comprising:

identifying a logical node, said logical node being associated with one or more source stages;

identifying one or more file block components, said one or more file block components comprising a retrieval target for at least one of said one or more source stages;

identifying one or more physical nodes;

determining, for each of said one or more physical nodes, a degree value;

identifying one or more qualified physical nodes from said one or more physical nodes having said degree value of one or more;

creating a preferred physical node table, said preferred physical node table comprising, for each of said one or more qualified physical nodes, an identifying indication and an indication of said degree value; and

sorting said preferred physical node table based on said degree value associated with each of said one or more qualified physical nodes.

2. A computer-implemented method comprising:

identifying a logical node, said logical node being associated with one or more source stages;

identifying one or more file block components, said one or more file block components comprising a retrieval target for at least one of said one or more source stages;

identifying one or more physical nodes;

determining, for each of said one or more physical nodes, a degree value;

determining a candidate preferred physical node based on each said degree value;

determining whether said candidate preferred physical node is available for allocation to said logical node; and

responsive to said candidate preferred physical node being available for allocation to said logical node, allocating said candidate preferred physical node to said logical node.

3. The computer-implemented method of claim 2, further comprising:

responsive to said candidate preferred physical node not being available for allocation to said logical node;

marking said candidate preferred physical node as unavailable for allocation to said logical node; and

determining an alternative candidate preferred physical node based on each said degree value.

4. A computer program product comprising one or more computer readable storage media and program instructions stored on said one or more computer readable storage media, said program instructions comprising instructions to:

identify a logical node, said logical node being associated with one or more source stages;

identify one or more file block components, said one or more file block components comprising a retrieval target for at least one of said one or more source stages;

identify one or more physical nodes;

determine, for each of said one or more physical nodes, a degree value;

identify one or more qualified physical nodes from said one or more physical nodes having said degree value of one or more;

create a preferred physical node table, said preferred physical node table comprising, for each of said one or more qualified physical nodes, an identifying indication and an indication of said degree value; and

sort said preferred physical node table based on said degree value associated with each of said one or more qualified physical nodes.

5. The computer program product of claim 4, wherein said program instructions further comprise instructions to:

determine a candidate preferred physical node based on each said degree value.

6. The computer program product of claim 5, wherein said program instructions further comprise instructions to:

determine whether said candidate preferred physical node is available for allocation to said logical node; and

responsive to said candidate preferred physical node being available for allocation to said logical node, allocate said candidate preferred physical node to said logical node.

7. The computer program product of claim 6, wherein said program instructions further comprise instructions to:

responsive to said candidate preferred physical node not being available for allocation to said logical node;

mark said candidate preferred physical node as unavailable for allocation to said logical node; and

determine an alternative candidate preferred physical node based on each said degree value.