SEMANTIC ANALYSIS OF INFORMATION

• US 20120101975A1
• Filed: 10/20/2010
• Published: 04/26/2012
• Est. Priority Date: 10/20/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Khosravy, Moe
• Granted Patent: US 9,076,152 B2
• Time in Patent Office: Days
• US Class Current: 706/55
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: G06F 16/36 Creation of semantic tools, e.g. ontology or thesauri
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence
• CPC Class Codes: G06F 16/907 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06Q 10/063 Operations research, analysis or management
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data

Abstract

Semantic information that describes data sets is inferred based upon a semantic analysis performed on data sets retained within a data repository. The semantic analysis can include a determination of formats associated with fields of the data sets and a comparison of values of the fields against reference data sets having predetermined semantic types. Correlations are inferred between data sets based upon respective semantic information. The correlations are incorporated into visualizations displayed in connection with a graphical user interface.

Claims

1. A method, comprising:

extracting a subset of data from a data set stored in a data repository;

identifying a format corresponding to the subset of data;

selecting at least one reference data set based upon the format, wherein the at least one reference data set is associated with a known semantic type;

comparing values of the subset of data against the at least one reference data set; and

inferring semantic information of the data set based upon at least one result of the comparing, wherein the semantic information specifies at least a semantic type of data in the data set.

2. The method of claim 1, wherein the extracting comprises extracting an entirety of the data set.

3. The method of claim 1, wherein the extracting the subset of data comprises selecting, at random, values of the data set.

4. The method of claim 1, wherein the extracting the subset of data comprises extracting a pre-determined number of values from at least one of a beginning of the data set, an end of the data set, or a middle of the data set.

5. The method of claim 1, further comprising:

validating the values of the subset of data against one or more domain rules, wherein the one or more domain rules correspond to the semantic type of data in the data set including presenting invalid values including values disallowed by the one or more domain rules.

6. The method of claim 1, further comprising:

extracting a field name from the data set; and

comparing the field name to a set of potential valid tags.

7. The method of claim 6, further comprising:

identifying when the field name matches a tag in the set of potential valid tags;

identifying a type associated with the tag in response to a match; and

associating the data set with the type as the semantic type of the data set.

8. The method of claim 1, wherein the identifying the format comprises evaluating at least one structure of the values of the subset of data against a plurality of pre-configured patterns.

9. The method of claim 8, further comprising:

identifying when the at least one structure of the values match a pre-configured pattern; and

selecting the at least one reference data set, wherein the known semantic type associated with the at least one reference data set corresponds to the pre-configured pattern.

10. The method of claim 9, further comprising:

adopting the known semantic type as the semantic type of data of the data set when a number of structures matching the pre-configured pattern meets at least one pre-defined criterion.

11. The method of claim 1, wherein the comparing the values of the subset of data against the at least one reference data set comprises:

selecting a value from the values;

querying the at least one reference data set with the value; and

counting a valid hit when a result of the query with the value is returned.

12. The method of claim 1, further comprising:

identifying a second data set including at least one field corresponding to the semantic type of data of the data set; and

generating relationship information that associates the data set with the second data set, wherein the relationship information specifies a correlation between the data set and the second data set.

13. The method of claim 1, further comprising:

obtaining relationship information correlating a plurality of data sets in the data repository; and

displaying a visualization of the relationship information as part of a graphical user interface.

14. The method of claim 13, further comprising:

receiving input via the graphical user interface; and

updating the relationship information based upon the input.

15. The method of claim 1, further comprising outputting the semantic information for external confirmation.

16. A semantic analysis system, comprising:

a semantic analysis component configured to infer semantic information regarding data of a data set stored by a system providing information as a service, the semantic analysis component comprising;

a format check component configured to identify a format of sample data extracted from the data set and to select one or more reference data sets based upon the format; and

a value check component configured to check values of the sample data against the one or more reference data sets to identify a semantic type of the sample data,

wherein the semantic analysis component is further configured to infer the semantic information based upon the semantic type identified.

17. The semantic analysis system of claim 16, further comprising:

a domain check component configured to determine whether values of the sample data satisfy a set of domain rules,

wherein the set of domain rules establish valid data values for a particular data type.

18. The semantic analysis system of claim 16, further comprising:

a linking component configured to correlate a plurality of disparate data sets stored by the information-as-a-service system based upon respective semantic information.

19. The semantic analysis system of claim 16, further comprising:

a visualization component configured to generate a visualization of correlations between data sets stored by the information-as-a-service system,

wherein the visualization comprises a graph display where nodes represent the data sets and edges represent correlations between the data sets.

20. A computer-readable storage medium comprising computer-executable instructions that, in response to execution, cause a computing device to perform operations, comprising:

identifying a format corresponding to a subset of data extracted from a data set stored in a data repository;

selecting at least one reference data set based upon the format, wherein the at least one reference data set is associated with a known semantic type;

comparing values of the subset of data with the at least one reference data set; and

inferring semantic information about the data set based upon an output of the comparing, wherein the semantic information specifies at least a semantic type of data in the data set.