Finding data in connected corpuses using examples

• US 8,983,954 B2
• Filed: 04/10/2012
• Issued: 03/17/2015
• Est. Priority Date: 04/10/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Platt, John C., Chaudhuri, Surajit, Novik, Lev, Meijer, Henricus Johannes Maria, Hudis, Efim, Mukerjee, Kunal, Hays, Christopher Alan
• Primary Examiner(s): Corrielus, Jean M
• Application Number:
  US13/443,681
• Publication Number:
  US 20130268531A1
• Time in Patent Office: 1,071 Days
• Field of Search: 707/736, 707/752, 707/756, 707/802, 707/E17.005, 707/E17.014, 707/E17.044, 707/737, 707/723, 707/748, 707/690, 707/776
• US Class Current: 707/736
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/2425 Iterative querying; Query formulation based on the results of a preceding query
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/2465 Query processing support for facilitating data mining operations in structured databases
• CPC Class Codes: G06F 16/284 Relational databases
• CPC Class Codes: G06F 16/38 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/634 Query by example, e.g. query by humming
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation

Abstract

In one embodiment, datasets are stored in a catalog. The datasets are enriched by establishing relationships among the domains in different datasets. A user searches for relevant datasets by providing examples of the domains of interest. The system identifies datasets corresponding to the user-provided examples. The system them identifies connected subsets of the datasets that are directly linked or indirectly linked through other domains. The user provides known relationship examples to filter the connected subsets and to identify the connected subsets that are most relevant to the user'"'"'s query. The selected connected subsets may be further analyzed by business intelligence/analytics to create pivot tables or to process the data.

Claims

1. A data processing system, comprising:

a processor; and

a memory coupled to the processor, the memory configured to store program instructions executable by the processor to cause the data processing system to;

receive a collection of values from a user;

identify a data type for each of the values;

identify distinct datasets that correspond to the data types, each of the distinct datasets data set having one or more of the data types;

identify relationships among the distinct datasets, the relationships corresponding to links between similar data types in the distinct datasets;

provide, to the user, a list of proposed groups of datasets, wherein the datasets within each proposed group are linked to each other through one or more relationships;

receive an example value set from the user, the example value set corresponding to a known relationship between two or more data types, and the example value set including at least two values;

re-interpret the one or more relationships based upon the example value set; and

provide, to the user, a second list of a second proposed group of datasets based upon the re-interpretation, wherein the datasets within the second proposed group include the example value set.

2. The data processing system of claim 1, further comprising:

receive a user selection of one of the proposed dataset groups.

3. The data processing system of claim 2, further comprising:

combine the datasets within the selected proposed dataset group into a new dataset.

4. The data processing system of claim 2, further comprising:

combine subsets of the datasets within the selected proposed dataset group into a new dataset.

5. The data processing system of claim 1, wherein the list of proposed groups of datasets are linked to each other through the relationships to intermediate datasets that do not have data types corresponding to the values received from the user.

6. The data processing system of claim 1, wherein the list of proposed groups of datasets comprises datasets that have overlapping data types corresponding to the values received from the user.

7. The data processing system of claim 1, further comprising:

rank the list of proposed groups of datasets, the ranking based upon weights assigned to one or more of the data types, datasets, and values received from the user.

8. A method, comprising:

performing, by a processor in a computer system;

identifying a collection of domains corresponding to a collection of values;

identifying distinct datasets corresponding to at least one of the domains;

identifying relationships among the distinct datasets, the relationships corresponding to links between similar domains in the distinct datasets;

identifying groups of datasets among the distinct datasets, wherein the datasets within each proposed group are linked to each other through one or more relationships;

receiving an example value set, the example value set corresponding to a known relationship between two or more domains, the example value set including at least two values;

re-interpreting the one or more relationships based upon the example value set; and

identifying one or more proposed groups of datasets based upon the re-interpretation, wherein the at least two values of the example value set are found within the datasets of the proposed groups.

9. The method of claim 8, wherein all of the values of the example value set are found within the datasets of the proposed groups.

10. The method of claim 8, wherein at least one of the values of the example value set is found within the datasets of the proposed groups.

11. The method of claim 8, further comprising:

receiving a user selection of one of the proposed dataset groups.

12. The method of claim 11, further comprising:

combining the datasets within the selected dataset group into a new dataset.

13. The method of claim 11, further comprising:

combining subsets of the datasets within the selected dataset group into a new dataset.

14. The method of claim 8, wherein the datasets in the proposed groups are linked to each other through the relationships to intermediate datasets that do not include the domains in the collection of domains.

15. The method of claim 8, wherein the datasets in the proposed groups comprise datasets that have overlapping domains corresponding to the collection of values.

16. The method of claim 8, further comprising:

ranking the proposed groups of datasets.

17. An article of manufacture having computer-executable instructions stored thereon that, upon execution by at least one processor of a computer system, cause the computer system to:

identify a collection of data types corresponding to a collection of values;

identify distinct datasets corresponding to at least one of the data types;

identify relationships among the distinct datasets, the relationships corresponding to links between similar data types in the distinct datasets;

identify groups of datasets among the distinct datasets, wherein the datasets within each proposed group are linked to each other through one or more relationships;

receive an example value set from the user, the example value set corresponding to a known relationship between two or more data types, the example value set including at least two values;

re-interpret the one or more relationships based upon the example value set; and

identify one or more proposed groups of datasets based upon the re-interpretation, wherein the at least two values of the example value set are found within the datasets of the proposed groups.

18. The article of manufacture of claim 17, wherein the computer-executable instructions, upon execution, further cause the computer system to:

receive a user selection of one of the proposed dataset groups; and

combining subsets of the datasets within the selected dataset group into a new dataset.