Declarative language and visualization system for recommended data transformations and repairs
First Claim
1. A method comprising:
- identifying, automatically by a data enrichment system comprising a processor and a memory, patterns in data from one or more data sources by parsing fields in a structure of the data;
matching, automatically by the data enrichment system, the patterns in data from the one or more data sources to entity information from a knowledge service;
generating, automatically by the data enrichment system, one or more transformation scripts for the data based on the entity information, wherein the one or more transformation scripts comprises a script for transforming the data to correspond to the patterns matching the entity information, and wherein the one or more transformation scripts are written in a declarative language;
generating, automatically by the data enrichment system based on a history of user activity, one or more transform recommendations corresponding to the one or more transformation scripts generated for the data based on the entity information;
displaying, by the data enrichment system, the one or more transform recommendations and the data to be transformed on an interactive user interface, wherein the one or more transform recommendations displayed on the interactive user interface are interactive and wherein the one or more transform recommendations that are displayed are based on the identified patterns in the data from the one or more data sources;
filtering, by the data enrichment system, the one or more transform recommendations in accordance with user input on the interactive user interface;
receiving, by the data enrichment system, transformation instructions based on a selection of the one or more transform recommendations on the interactive user interface;
transforming, automatically by the data enrichment system, the data displayed on the interactive user interface by applying the transformation instructions, corresponding to the one or more transformation scripts, to the data displayed on the interactive user interface in order to process large data sets;
displaying the transformed data and the transformation scripts applied to the transformed data on the interactive user interface; and
publishing, by the data enrichment system, the transformed data to one or more data targets based on the transformation instructions.
1 Assignment
0 Petitions
Accused Products
Abstract
The present disclosure relates generally to a data enrichment service that extracts, repairs, and enriches datasets, resulting in more precise entity resolution and correlation for purposes of subsequent indexing and clustering. As the data enrichment service can include a visual recommendation engine and language for performing large-scale data preparation, repair, and enrichment of heterogeneous datasets. This enables the user to select and see how the recommended enrichments (e.g., transformations and repairs) will affect the user'"'"'s data and make adjustments as needed. The data enrichment service can receive feedback from users through a user interface and can filter recommendations based on the user feedback.
51 Citations
24 Claims
-
1. A method comprising:
-
identifying, automatically by a data enrichment system comprising a processor and a memory, patterns in data from one or more data sources by parsing fields in a structure of the data; matching, automatically by the data enrichment system, the patterns in data from the one or more data sources to entity information from a knowledge service; generating, automatically by the data enrichment system, one or more transformation scripts for the data based on the entity information, wherein the one or more transformation scripts comprises a script for transforming the data to correspond to the patterns matching the entity information, and wherein the one or more transformation scripts are written in a declarative language; generating, automatically by the data enrichment system based on a history of user activity, one or more transform recommendations corresponding to the one or more transformation scripts generated for the data based on the entity information; displaying, by the data enrichment system, the one or more transform recommendations and the data to be transformed on an interactive user interface, wherein the one or more transform recommendations displayed on the interactive user interface are interactive and wherein the one or more transform recommendations that are displayed are based on the identified patterns in the data from the one or more data sources; filtering, by the data enrichment system, the one or more transform recommendations in accordance with user input on the interactive user interface; receiving, by the data enrichment system, transformation instructions based on a selection of the one or more transform recommendations on the interactive user interface; transforming, automatically by the data enrichment system, the data displayed on the interactive user interface by applying the transformation instructions, corresponding to the one or more transformation scripts, to the data displayed on the interactive user interface in order to process large data sets; displaying the transformed data and the transformation scripts applied to the transformed data on the interactive user interface; and publishing, by the data enrichment system, the transformed data to one or more data targets based on the transformation instructions. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
-
-
12. A system comprising:
-
a plurality of data sources; a plurality of data targets; and a cloud computing infrastructure system comprising; one or more processors communicatively coupled to the plurality of data sources and communicatively coupled to the plurality of data targets, over at least one communication network; and a memory coupled to the one or more processors, the memory storing instructions to provide a data enrichment service, wherein the instructions, when executed by the one or more processors, cause the one or more processors to; receive a data enrichment request from a client device; automatically identify patterns in data from at least one data source specified in the data enrichment request by parsing fields in a structure of the data; automatically match the patterns in data from the one or more data sources to entity information from a knowledge service; automatically generate one or more transformation scripts for the data based on the entity information, wherein the one or more transformation scripts comprises a script for transforming the data to correspond to the patterns matching the entity information, and wherein the one or more transformation scripts are written in a declarative language; automatically generate based on a history of user activity one or more transform recommendations corresponding to the one or more transformation scripts generated for the data based on the entity information; display the one or more transform recommendations and the data to be transformed on an interactive user interface on the client device, wherein the one or more transform recommendations displayed on the interactive user interface are interactive and wherein the one or more transform recommendations that are displayed are based on the identified patterns in the data from the one or more data sources; filter the one or more transform recommendations in accordance with user input on the interactive user interface; receive transformation instructions based on a selection of the one or more transform recommendations on the interactive user interface; automatically transform the data displayed on the interactive user interface by applying the transformation instructions, corresponding to the one or more transformation scripts, to the data displayed on the interactive user interface in order to process large data sets; displaying the transformed data and the transformation scripts applied to the transformed data on the interactive user interface; and publish the transformed data to one or more data targets based on the transformation instructions. - View Dependent Claims (13, 14, 15, 16, 17, 18)
-
-
19. A non-transitory computer readable storage medium including instructions stored thereon which, when executed by a processor, cause the processor to perform operations comprising:
-
identifying, automatically by a data enrichment system, patterns in data from one or more data sources by parsing fields in a structure of the data; matching, automatically by the data enrichment system, the patterns in data from the one or more data sources to entity information from a knowledge service; generating, automatically by the data enrichment system, one or more transformation scripts for the data based on the entity information, wherein the one or more transformation scripts comprises a script for transforming the data to correspond to the patterns matching the entity information; generating, automatically by the data enrichment system based on a history of user activity, one or more transform recommendations corresponding to the one or more transformation scripts generated for the data based on the entity information, and wherein the one or more transformation scripts are written in a declarative language; displaying, by the data enrichment system, the one or more transform recommendations and the data to be transformed on an interactive user interface, wherein the one or more transform recommendations displayed on the interactive user interface are interactive and wherein the one or more transform recommendations that are displayed are based on the identified patterns in the data from the one or more data sources; filtering, by the data enrichment system, the one or more transform recommendations in accordance with user input on the interactive user interface; receiving, by the data enrichment system, transformation instructions based on a selection of the one or more transform recommendations on the interactive user interface; transforming, automatically by the data enrichment system, the data displayed on the interactive user interface by applying the transformation instructions, corresponding to the one or more transformation scripts, to the data displayed on the interactive user interface in order to process large data sets; displaying the transformed data and the transformation scripts applied to the transformed data on the interactive user interface; and publishing, by the data enrichment system, the transformed data to one or more data targets based on the transformation instructions. - View Dependent Claims (20, 21, 22, 23, 24)
-
Specification