AUTOMATED ENTITY CORRELATION AND CLASSIFICATION ACROSS HETEROGENEOUS DATASETS

• US 20160092475A1
• Filed: 09/24/2015
• Published: 03/31/2016
• Est. Priority Date: 09/26/2014
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Stojanovic, Alexander Sasha, Ogren, Philip, Markey, Kevin L., Kreider, Mark
• Granted Patent: US 10,915,233 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/211 Schema design and management
• CPC Class Codes: G06F 16/2219 Large Object storage; Management thereof
• CPC Class Codes: G06F 16/2379 Updates performed during online database operations; commit processing
• CPC Class Codes: G06F 16/9038 Presentation of query results
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]

Abstract

The present disclosure describes techniques for entity classification and data enrichment of data sets. A data enrichment system is disclosed that can extract, repair, and enrich datasets, resulting in more precise entity resolution and classification for purposes of subsequent indexing and clustering. Disclosed techniques may include performing entity recognition to identify segments of interest that relate to an entity. Related data may be analyzed for classification, which can be used to transform the data for enrichment to its users.

Claims

1. A method comprising:

receiving, by a computing system, a data set from one or more data sources;

creating, by the computing system, a normalized data set by normalizing the data set based on a format of the data set;

identifying, by the computing system, a set of patterns for a set of entities in the normalized data set;

extracting, by the computing system, based on the set of patterns, entity information corresponding to the set of entities in the normalized data set;

determining, by the computing system, a classification of the set of entities using the entity information;

generating, by the computing system, based on the classification and the entity information, a transformed data set for the set of entities in the normalized data set; and

rendering a graphical interface that displays the transformed data set and information indicating a transformation to the normalized data set to generate the transformed data set.

2. The method of claim 1, further comprising:

computing a set of pattern metrics, wherein each of the set of pattern metrics is computed for a different pattern in the set of patterns, and wherein the set of patterns includes a plurality of different patterns identified for the set of entities;

determining a difference amongst the set of pattern metrics; and

selecting, based on the difference amongst the set of pattern metrics, a pattern in the set of patterns, wherein the classification is determined based on the selected pattern.

3. The method of claim 1, further comprising:

identifying text data in the normalized data set, the text data corresponding to each entity of the set of entities;

determining a set of classifications for the set of entities; and

computing a set of classification metrics, each of the set of classification metrics computed for a different classification in the set of classifications; and

wherein the classification is determined based on determining a difference amongst the set of classifications.

4. The method of claim 1, wherein normalizing the data set to create the normalized data set includes modifying the data set having the format to an adjusted format of the normalized data set, the adjusted format being different from the format.

5. The method of claim 1, wherein identifying the set of patterns includes comparing column data amongst a plurality of columns in the normalized data set, and determining that each of a set of columns in the plurality of columns has an attribute, and wherein the set of patterns for the set of entities is identified based on the normalized data for each column having the attribute.

6. The method of claim 1, wherein the transformed data set may be generated from the normalized data set by modifying the normalized data set to include data about the classification for the set of entities.

7. The method of claim 1, further comprising:

rendering, by the computing system, an additional graphical interface that displays the normalized data, that identifies the entity information, and that indicates the classification; and

receiving input indicating selection of the classification in the additional graphical interface;

wherein the transformed data set is generated upon receiving the input.

8. A data enrichment system comprising:

a plurality of data sources; and

a cloud computing infrastructure system comprising;

one or more processors communicatively coupled to the plurality of data sources over at least one communication network; and

a memory coupled to the one or more processors, the memory storing instructions to provide a data enrichment service, wherein the instructions, when executed by the one or more processors, cause the one or more processors to;

receive a data set from one or more data sources of the plurality of data sources;

create a normalized data set by normalizing the data set based on a format of the data set;

identify a set of patterns for a set of entities in the normalized data set;

extract, based on the set of patterns, entity information corresponding to the set of entities in the normalized data set;

determine a classification of the set of entities using the entity information;

generate, based on the classification and the entity information, a transformed data set for the set of entities in the normalized data set; and

render a graphical interface that displays the transformed data set and information indicating a transformation to the normalized data set to generate the transformed data set.

9. The data enrichment system of claim 8, wherein normalizing the data set to create the normalized data includes modifying the data set having the format to an adjusted format of the normalized data set, the adjusted format being different from the format.

10. The data enrichment system of claim 8, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:

compute a set of pattern metrics, wherein each of the set of pattern metrics is computed for a different pattern in the set of patterns, and wherein the set of patterns includes a plurality of different patterns identified for the set of entities;

determine a difference amongst the set of pattern metrics; and

select, based on the difference amongst the set of pattern metrics, a pattern in the set of patterns, wherein the classification is determined based on the selected pattern.

11. The data enrichment system of claim 8, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:

identify text data in the normalized data set, the text data corresponding to each entity of the set of entities;

determine a set of classifications for the set of entities; and

compute a set of classification metrics, each of the set of classification metrics computed for a different classification in the set of classifications; and

wherein the classification is determined based on determining a difference amongst the set of classifications.

12. The data enrichment system of claim 8, wherein the transformed data set may be generated from the normalized data set by modifying the normalized data set to include data about the classification for the set of entities.

13. The data enrichment system of claim 8, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:

render an additional graphical interface that displays the normalized data, that identifies the entity information, and that indicates the classification; and

receive input indicating selection of the classification in the additional graphical interface;

wherein the transformed data set is generated upon receiving the input.

14. A non-transitory computer readable storage medium including instructions stored thereon which, when executed by one or more processors, cause the one or more processors to:

receive, by a computing system, a data set from one or more data sources;

create, by the computing system, a normalized data set by normalizing the data set based on a format of the data set;

identify, by the computing system, a set of patterns for a set of entities in the normalized data set;

extract, by the computing system, based on the set of patterns, entity information corresponding to the set of entities in the normalized data set;

determine, by the computing system, a classification of the set of entities using the entity information;

generate, by the computing system, based on the classification and the entity information, a transformed data set for the set of entities in the normalized data set; and

render a graphical interface that displays the transformed data set and information indicating a transformation to the normalized data set to generate the transformed data set.

15. The non-transitory computer readable storage medium of claim 14, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:

compute a set of pattern metrics, wherein each of the set of pattern metrics is computed for a different pattern in the set of patterns, and wherein the set of patterns includes a plurality of different patterns identified for the set of entities;

determine a difference amongst the set of pattern metrics; and

select, based on the difference amongst the set of pattern metrics, a pattern in the set of patterns, wherein the classification is determined based on the selected pattern.

16. The non-transitory computer readable storage medium of claim 14,

wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:

identify text data in the normalized data set, the text data corresponding to each entity of the set of entities;

determine a set of classifications for the set of entities; and

compute a set of classification metrics, each of the set of classification metrics computed for a different classification in the set of classifications; and

wherein the classification is determined based on determining a difference amongst the set of classifications.

17. The non-transitory computer readable storage medium of claim 14, wherein normalizing the data set to create the normalized data includes modifying the data set having the format to an adjusted format of the normalized data set, the adjusted format being different from the format.

18. The non-transitory computer readable storage medium of claim 14, wherein identifying the one or more patterns includes comparing column data amongst a plurality of columns in the normalized data set, and determining that each of a set of columns in the plurality of columns has an attribute, and wherein the one or more patterns for the set of entities is identified based on the normalized data for each column having the attribute.

19. The non-transitory computer readable storage medium of claim 14, wherein the transformed data set may be generated from the normalized data set by modifying the normalized data set to include data about the classification for the set of entities.

20. The non-transitory computer readable storage medium of claim 14, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:

render an additional graphical interface that displays the normalized data, that identifies the entity information, and that indicates the classification; and

receive input indicating selection of the classification in the additional graphical interface;

wherein the transformed data set is generated upon receiving the input.