Search-based detection, link, and acquisition of data

• US 10,318,527 B2
• Filed: 01/27/2015
• Issued: 06/11/2019
• Est. Priority Date: 01/27/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Rais-Ghasem, Mohsen, Grosset, Robin N.
• Primary Examiner(s): Burke, Jeff A
• Assistant Examiner(s): Nguyen, Cindy
• Application Number:
  US14/606,844
• Publication Number:
  US 20160217179A1
• Time in Patent Office: 1,596 Days
• CPC Class Codes: G06F 16/243 Natural language query formulation
• CPC Class Codes: G06F 16/24575 using context

Abstract

A computing device may receive a natural language query. The computing device may determine, based on the natural language query, a set of search criteria. The computing device may determine, from a plurality of data sets, two or more data sets that are relevant to the natural language query based at least in part on matches between the set of search criteria and attributes of the plurality of data sets. The computing device may join the two or more data sets, including joining one or more rows and one or more columns of the two or more data sets to produce a joint data set. The computing device may output a representation of the joint data set.

Claims

1. A computer system comprising:

one or more processors, one or more computer-readable memories, and one or more computer readable, tangible storage devices;

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to receive a natural language query;

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to determine, based on the natural language query, a set of search criteria;

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to;

determine a respective coverage score for each of a plurality of data sets, wherein the plurality of data sets is stored in one or more data sources, and

determine a respective coverage score for each of one or more combinations of two or more data sets, wherein the respective coverage score for each of the one or more combinations of two or more data sets is separate from the coverage score for each of the plurality of data sets,

wherein the respective coverage score for each of the one or more combinations of two or more data sets is based at least in part on matches between the set of search criteria and attributes of the plurality of data sets and of the one or more combinations of two or more data sets, wherein the respective coverage score for a respective one of the plurality of data sets corresponds to a percentage of the natural language query that is bound to one or more attributes of the respective one of the plurality of data sets, and wherein the respective coverage score for a respective one of the one or more combinations of two or more data sets corresponds to the percentage of the natural language query that is bound to one or more attributes of the respective one of the one or more combinations of two or more data sets;

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to, in response to determining a combination of two or more data sets out of the one or more combinations of two or more data sets that are associated with the natural language query based at least in part on the respective coverage score of the combination of two or more data sets, build an on-the-fly data source query to join two or more data sets making up the combination of two or more data sets, including joining at least one of;

two or more rows and two or more columns of the two or more data sets to produce a joint data set;

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to execute the on-the-fly data source query to retrieve the joint data set from the one or more data sources; and

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to output a representation of the joint data set for display at a display device.

2. The computer system of claim 1, wherein the matches between the set of search criteria and the attributes of the plurality of data sets include one or more of:

a literal match, a value match, a semantic match, and a partial name match, and wherein the attributes of the plurality of data sets include one or more of;

semantic metadata, lexical clues, sample values, and numeric statistics.

3. The computer system of claim 2, further comprising:

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to associate an indication of matching strength with each of the matches; and

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to rank the matches based on the indication of matching strength associated with the respective matches.

4. The computer system of claim 3, further comprising:

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to determine the respective coverage score for each of the one or more combinations of two or more data sets of the plurality of data sets based at least in part on the ranking of the matches.

5. The computer system of claim 4, wherein:

the program instructions to build the on-the-fly data source query to join the two or more data sets making up the combination of two or more data sets further comprise program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to determine one or more join paths of data items between the two or more data sets; and

the program instructions to build the on-the-fly data source query to join the two or more data sets making up the combination of two or more data sets further comprise program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to build the on-the-fly data source query to join the two or more data sets based at least in part on the one or more join paths.

12. The computer system of claim 1, wherein the one or more data sources are external to the computer system.

6. A computer product comprising a computer-readable storage medium having program code embodied therewith, the program code executable by a computing device to:

receive a natural language query;

determine, based on the natural language query, a set of search criteria;

determine a respective coverage score for each of a plurality of data sets, wherein the plurality of data sets is stored in one or more data sources,

determine a respective coverage score for each of one or more combinations of two or more data sets, wherein;

the respective coverage score for each of the one or more combinations of two or more data sets is separate from the coverage score for each of the plurality of data sets,

the respective coverage score for each of the one or more combinations of two or more data sets is based at least in part on matches between the set of search criteria and attributes of the plurality of data sets and of the one or more combinations of two or more data sets, wherein the respective coverage score for a respective one of the plurality of data sets corresponds to a percentage of the natural language query that is bound to one or more attributes of the respective one of the plurality of data sets, and wherein the respective coverage score for a respective one of the one or more combinations of two or more data sets corresponds to the percentage of the natural language query that is bound to one or more attributes of the respective one of the one or more combinations of two or more data sets;

in response to determining a combination of two or more data sets out of the one or more combinations of two or more data sets that are associated with the natural language query based at least in part on the respective coverage score of the combination of two or more data sets, build an on-the-fly data source query to join two or more data sets making up the combination of two or more data sets, including joining at least one of;

two or more rows and two or more columns of the two or more data sets to produce a joint data set;

execute the on-the-fly data source query to retrieve the joint data set from the one or more data sources; and

output a representation of the joint data set.

7. The computer product of claim 6, wherein the one or more data sources are external to the computing device.

8. The computer product of claim 6, wherein the matches between the set of search criteria and the attributes of the plurality of data sets include one or more of:

a literal match, a value match, a semantic match, and a partial name match, and wherein the attributes of the plurality of data sets include one or more of;

semantic metadata, lexical clues, sample values, and numeric statistics.

9. The computer product of claim 8, wherein the program code is further executable by the computing device to:

associate an indication of matching strength with each of the matches; and

rank the matches based on the indication of matching strength associated with the respective matches.

10. The computer product of claim 9, wherein the program code is further executable by the computing device to:

determine the respective coverage score for each of the one or more combinations of two or more data sets of the plurality of data sets based at least in part on the ranking of the matches.

11. The computer product of claim 10, wherein:

the program code executable by the computing device to build the on-the-fly data source query to join the two or more data sets making up the combination of two or more data sets further comprises program code executable by the computing device to determine one or more join paths of data items between the two or more data sets; and

the program code executable by the computing device to build the on-the-fly data source query to join the two or more data sets making up the combination of two or more data sets further comprises program code executable by the computing device to build the on-the-fly data source query to join the two or more data sets based at least in part on the one or more join paths.