DECLARATIVE LANGUAGE AND VISUALIZATION SYSTEM FOR RECOMMENDED DATA TRANSFORMATIONS AND REPAIRS

• US 20160092474A1
• Filed: 09/24/2015
• Published: 03/31/2016
• Est. Priority Date: 09/26/2014
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Stojanovic, Alexander Sasha, Ogren, Philip, Rivas, Luis E., Murray, Glenn Allen
• Granted Patent: US 10,891,272 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/211 Schema design and management
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/2379 Updates performed during online database operations; commit processing
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence

Abstract

The present disclosure relates generally to a data enrichment service that extracts, repairs, and enriches datasets, resulting in more precise entity resolution and correlation for purposes of subsequent indexing and clustering. As the data enrichment service can include a visual recommendation engine and language for performing large-scale data preparation, repair, and enrichment of heterogeneous datasets. This enables the user to select and see how the recommended enrichments (e.g., transformations and repairs) will affect the user'"'"'s data and make adjustments as needed. The data enrichment service can receive feedback from users through a user interface and can filter recommendations based on the user feedback.

Claims

1. A method comprising:

identifying, by a computer system, patterns in data from one or more data sources;

matching the patterns to entity information from a knowledge service;

generating one or more transformation scripts for the data based on the entity information;

generating one or more recommendations corresponding to the one or more transformation scripts;

causing the one or more recommendations to be displayed in a user interface;

receiving transformation instructions based on the one or more recommendations;

transforming the data based on the transformation instructions; and

publishing the transformed data to one or more data targets based on the transformation instructions.

2. The method of claim 1, further comprising:

receiving a selection of at least one column of data from at least one of the one or more data sources; and

causing a data profile of the at least one column of data to be displayed, wherein the data profile indicates a plurality of patterns identified in the at least one column of data and at least one data visualization associated with the at least one column of data.

3. The method of claim 1, wherein the transformation instructions include a transformation instruction to rename at least one column of data based on the entity information.

4. The method of claim 3, further comprising:

receiving a further transformation instruction to rename the at least one column of data to a default name.

5. The method of claim 1, wherein the transformation instructions include a transformation instruction to reformat at least one column of data based on the entity information.

6. The method of claim 5, wherein the transformation instructions include a further transformation instruction to obfuscate at least one column of data based on the entity information.

7. The method of claim 1, wherein the transformation instructions include an enrichment instruction to add one or more columns of data obtained from the knowledge service based on the entity information.

8. A system comprising:

a plurality of data sources;

a plurality of data targets; and

a cloud computing infrastructure system comprising;

one or more processors communicatively coupled to the plurality of data sources and communicatively coupled to the plurality of data targets, over at least one communication network; and

a memory coupled to the one or more processors, the memory storing instructions to provide a data enrichment service, wherein the instructions, when executed by the one or more processors, cause the one or more processors to;

receive a data enrichment request from a client device;

identify patterns in data from at least one data source specified in the data enrichment request;

match the patterns to entity information from a knowledge service;

generate one or more transformation scripts for the data based on the entity information;

generate one or more recommendations corresponding to the one or more transformation scripts;

cause the one or more recommendations to be displayed in a user interface on the client device;

receive transformation instructions based on the one or more recommendations;

transform the data based on the transformation instructions; and

publish the transformed data to one or more data targets based on the transformation instructions.

9. The system of claim 8, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:

receive a selection of at least one column of data from at least one of the plurality of data sources; and

cause a data profile of the at least one column of data to be displayed in the user interface on the client device, wherein the data profile indicates a plurality of patterns identified in the at least one column of data and at least one data visualization associated with the at least one column of data.

10. The system of claim 8, wherein the transformation instructions include a transformation instruction to rename at least one column of data based on the entity information.

11. The system of claim 10, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:

receive a further transformation instruction to rename the at least one column of data to a default name.

12. The system of claim 8, wherein the transformation instructions include a transformation instruction to reformat at least one column of data based on the entity information.

13. The system of claim 12, wherein the transformation instructions include a further transformation instruction to obfuscate at least one column of data based on the entity information.

14. The system of claim 8, wherein the transformation instructions include an enrichment instruction to add one or more columns of data obtained from the knowledge service based on the entity information.

15. A non-transitory computer readable storage medium including instructions stored thereon which, when executed by a processor, cause the processor to perform operations comprising:

identifying, by a computer system, patterns in data from one or more data sources;

matching the patterns to entity information from a knowledge service;

generating one or more transformation scripts for the data based on the entity information;

generating one or more recommendations corresponding to the one or more transformation scripts;

causing the one or more recommendations to be displayed in a user interface;

receiving transformation instructions based on the one or more recommendations;

transforming the data based on the transformation instructions; and

publishing the transformed data to one or more data targets based on the transformation instructions.

16. The non-transitory computer readable storage medium of claim 15, wherein the operations further comprise:

receiving a selection of at least one column of data from at least one of the one or more data sources; and

causing a data profile of the at least one column of data to be displayed, wherein the data profile indicates a plurality of patterns identified in the at least one column of data and at least one data visualization associated with the at least one column of data.

17. The non-transitory computer readable storage medium of claim 15, wherein the transformation instructions include a transformation instruction to rename at least one column of data based on the entity information.

18. The non-transitory computer readable storage medium of claim 17, wherein the operations further comprise:

receiving a further transformation instruction to rename the at least one column of data to a default name.

19. The non-transitory computer readable storage medium of claim 15, wherein the transformation instructions include:

a transformation instruction to reformat at least one column of data based on the entity information; and

a further transformation instruction to obfuscate at least one column of data based on the entity information.

20. The non-transitory computer readable storage medium of claim 15, wherein the transformation instructions include an enrichment instruction to add one or more columns of data obtained from the knowledge service based on the entity information.