Finding Data in Connected Corpuses Using Examples

• US 20130268531A1
• Filed: 04/10/2012
• Published: 10/10/2013
• Est. Priority Date: 04/10/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Platt, John C., Chaudhuri, Surajit, Novik, Lev, Meijer, Henricus Johannes Maria, Hudis, Efim, Mukerjee, Kunal, Hays, Christopher Alan
• Granted Patent: US 8,983,954 B2
• Time in Patent Office: Days
• US Class Current: 707/737
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/2425 Iterative querying; Query formulation based on the results of a preceding query
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/2465 Query processing support for facilitating data mining operations in structured databases
• CPC Class Codes: G06F 16/284 Relational databases
• CPC Class Codes: G06F 16/38 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/634 Query by example, e.g. query by humming
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation

Abstract

In one embodiment, datasets are stored in a catalog. The datasets are enriched by establishing relationships among the domains in different datasets. A user searches for relevant datasets by providing examples of the domains of interest. The system identifies datasets corresponding to the user-provided examples. The system them identifies connected subsets of the datasets that are directly linked or indirectly linked through other domains. The user provides known relationship examples to filter the connected subsets and to identify the connected subsets that are most relevant to the user'"'"'s query. The selected connected subsets may be further analyzed by business intelligence/analytics to create pivot tables or to process the data.

Claims

1. A data processing system, comprising:

a processor; and

a memory coupled to the processor, the memory configured to store program instructions executable by the processor to cause the data processing system to;

receive a collection of values from a user;

identify a data type for each of the values;

identify datasets that correspond to the data types, each data set having one or more of the data types;

identify relationships among the datasets, the relationships corresponding to links between similar data types in different datasets; and

provide a list of proposed groups of datasets, wherein the datasets within each proposed group linked to each other through one or more relationships.

2. The data processing system of claim 1, further comprising:

receive an example value set from the user, the example value set corresponding to a known relationship between two or more data types.

3. The data processing system of claim 2, further comprising:

list a second proposed group of datasets for the user, wherein the datasets within the second proposed group include the example value set.

4. The data processing system of claim 1, further comprising:

receive a user selection of one of the proposed dataset groups.

5. The data processing system of claim 4, further comprising:

combine the datasets within the selected proposed dataset group into a new dataset.

6. The data processing system of claim 4, further comprising:

combine subsets of the datasets within the selected proposed dataset group into a new dataset.

7. The data processing system of claim 1, wherein the list of proposed groups of datasets are linked to each other through the relationships to intermediate datasets that do not have data types corresponding to the values received from the user.

8. The data processing system of claim 1, wherein the list of proposed groups of datasets comprises datasets that have overlapping data types corresponding to the values received from the user.

9. The data processing system of claim 1, further comprising:

rank the list of proposed groups of datasets, the ranking based upon weights assigned to one or more of the data types, datasets, and values received from the user.

10. A method, comprising:

performing, by a processor in a computer system, identifying a collection of domains corresponding to a collection of values;

identifying datasets corresponding to at least one of the domains;

identifying relationships among the datasets, the relationships corresponding to links between similar domains in different datasets;

identifying groups of datasets, wherein the datasets within each proposed group are linked to each other through one or more relationships;

receiving an example value set, the example value set corresponding to a known relationship between two or more domains; and

identifying one or more proposed groups of datasets, wherein values of the example value set are found within the datasets of the proposed groups.

11. The method of claim 10, wherein all of the values of the example value set are found within the datasets of the proposed groups.

12. The method of claim 10, wherein at least one of the values of the example value set is found within the datasets of the proposed groups.

13. The method of claim 10, further comprising:

receiving a user selection of one of the proposed dataset groups.

14. The method of claim 13, further comprising:

combining the datasets within the selected dataset group into a new dataset.

15. The method of claim 13, further comprising:

combining subsets of the datasets within the selected dataset group into a new dataset.

16. The method of claim 10, wherein the datasets in the proposed groups are linked to each other through the relationships to intermediate datasets that do not include the domains in the collection of domains.

17. The method of claim 10, wherein the datasets in the proposed groups comprise datasets that have overlapping domains corresponding to the collection of values.

18. The method of claim 10, further comprising:

ranking the proposed groups of datasets.

19. A computer-readable storage medium storing computer-executable instructions that when executed by at least one processor cause the at least one processor to perform a method for identifying groups of datasets, the method comprising:

identify a collection of data types corresponding to a collection of values;

identify datasets corresponding to at least one of the data types;

identify relationships among the datasets, the relationships corresponding to links between similar data types in different datasets;

identify groups of datasets, wherein the datasets within each proposed group are linked to each other through one or more relationships;

receive an example value set from the user, the example value set corresponding to a known relationship between two or more data types; and

identify one or more proposed groups of datasets, wherein values of the example value set are found within the datasets of the proposed groups.

20. The computer-readable storage medium of claim 19, wherein the method further comprises:

receive a user selection of one of the proposed dataset groups; and

combining subsets of the datasets within the selected dataset group into a new dataset.