RESOLVING DATABASE ENTITY INFORMATION

• US 20090106242A1
• Filed: 10/18/2007
• Published: 04/23/2009
• Est. Priority Date: 10/18/2007
• Status: Active Grant

• Current Assignee: Palantir Technologies Incorporated
• Original Assignee: Palantir Technologies Incorporated
• Inventors: Cohen, Stephen, McGrew, Robert J.
• Granted Patent: US 8,554,719 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/219 Managing data history or versioning querying versioned data G06F16/2474; querying temporal data G06F16/2477
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/24556 Aggregation; Duplicate elimination
• CPC Class Codes: G06F 16/252 between a Database Management System and a front-end application
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/285 Clustering or classification

Abstract

Entity resolution in a database comprises receiving imported data comprising imported data entities each having properties each having values; receiving first user input that selects the imported data entities for resolution to existing data entities in a database; receiving second user input that specifies matching criteria for matching the imported data entities to the existing data entities, wherein each of the matching criteria comprises a matching technique; matching the imported data entities to the existing data entities using the matching criteria, resulting in creating and storing matched entity information, wherein the matched entity information is organized in matched entity data sets associated with subsets of the matching criteria that were matched; consolidating the imported data entities into the existing data entities; storing the first user input and second user input as a named criteria set for use in subsequent entity resolution operations.

Claims

1. A computer-readable data storage medium comprising one or more stored sequences of instructions which, when executed by one or more processors, cause the one or more processors to perform:

receiving imported data comprising one or more imported data entities each having one or more properties each having one or more values;

receiving first user input that selects one or more of the imported data entities for resolution to one or more existing data entities in a database;

receiving second user input that specifies one or more matching criteria for matching the imported data entities to the existing data entities, wherein each of the matching criteria comprises a matching technique;

matching the imported data entities to the existing data entities using the matching criteria, resulting in creating and storing matched entity information, wherein the matched entity information is organized in one or more matched entity data sets associated with one or more subsets of the matching criteria that were matched;

consolidating the imported data entities into the existing data entities;

storing the first user input and second user input as a named criteria set for use in subsequent entity resolution operations.

2. The computer-readable medium of claim 1, wherein the matching technique specifies no conflicts between values of properties of the imported data entities and the existing data entities.

3. The computer-readable medium of claim 1, further comprising sequences of instructions which, when executed by the processor, cause the processor to perform:

receiving additional imported data for resolution to the existing data entities;

receiving third user input that specifies a name of a stored criteria set;

loading the stored criteria set;

automatically performing the matching based on the stored criteria set and without receiving the second user input.

4. The computer-readable medium of claim 1, further comprising sequences of instructions which, when executed by the processor, cause the processor to perform:

receiving the imported data;

receiving the first user input as a selection of one of the imported data entities and one of the existing data entities in a database;

receiving third user input requesting an entity resolution of the selected imported data entity and the selected existing data entity;

loading the stored criteria set;

automatically performing the matching based on the stored criteria set and performing the consolidating without receiving the second user input.

5. The computer-readable medium of claim 1, further comprising sequences of instructions which, when executed by the processor, cause the processor to perform receiving third user input that specifies one or more target filters for types of the existing data entities, and performing the matching only for corresponding ones of the existing data entities in the database.

6. The computer-readable medium of claim 1, further comprising sequences of instructions which, when executed by the processor, cause the processor to perform displaying the imported data entities in a histogram display that identifies the imported data entities and corresponding counts of the values of the properties of the imported data entities;

receiving the first user input as selections of one or more of the imported data entities in the histogram display.

7. The computer-readable medium of claim 1, further comprising sequences of instructions which, when executed by the processor, cause the processor to perform:

displaying, in a first panel of a graphical user interface, the subsets of the matching criteria that were matched, and counts of the one or more matched entity data sets associated with the one or more subsets of the matching criteria that were matched;

receiving third user input that selects one of the subsets;

displaying, in response to the third user input, in a second panel of the graphical user interface, each matching entity represented in the count of the selected subset.

8. The computer-readable medium of claim 1, wherein the consolidating comprises merging the imported data entities into the existing data entities.

9. The computer-readable medium of claim 1, further comprising sequences of instructions which, when executed by the processor, cause the processor to perform displaying, in a graphical user interface, results of consolidating the imported data entities into the existing data entities, the results comprising:

a first list of one or more associations of resolved entities each comprising one of the imported data entities, one of the existing data entities, and an associated subset of the matching criteria that was matched for that association;

a second list of one or more new entities to create in the database based on one or more of the imported data entities that did not match any of the existing data entities.

10. The computer-readable medium of claim 1, further comprising sequences of instructions which, when executed by the processor, cause the processor to perform:

displaying, in a graphical user interface, four visually expandable and collapsible graphical regions associated with steps in an entity resolution process;

wherein a first graphical region when expanded comprises a first list of the one or more imported data entities and one or more of the properties and one or more of the values;

wherein a second graphical region when expanded comprises a first panel comprising the one or more matching criteria for matching the imported data entities to the existing data entities and the matching technique for each of the matching criteria, and a second panel comprising one or more target filters for types of the existing data entities;

wherein a third graphical region when expanded comprises a third panel comprising the subsets of the matching criteria that were matched, and counts of the one or more matched entity data sets associated with the one or more subsets of the matching criteria that were matched, and a fourth panel comprising each matching entity represented in the count of a selected subset;

wherein a fourth graphical region when expanded comprises a fifth panel comprising a second list of one or more associations of resolved entities each comprising one of the imported data entities, one of the existing data entities, and an associated subset of the matching criteria that was matched for that association; and

a sixth panel comprising a third list of one or more new entities to create in the database based on one or more of the imported data entities that did not match any of the existing data entities.

11. The computer-readable medium of claim 1, further comprising instructions which when executed cause determining whether all the data entities were resolved, receiving modified matching criteria applicable to one or more unresolved data entities, and repeating the matching and consolidating steps for the unresolved data entities.

12. The computer-readable medium of claim 1, wherein the first user input comprises a semantic criterion referencing one or more properties of the data entities.

13. A machine-implemented method, comprising:

receiving imported data comprising one or more imported data entities each having one or more properties each having one or more values;

receiving first user input that selects one or more of the imported data entities for resolution to one or more existing data entities in a database;

receiving second user input that specifies one or more matching criteria for matching the imported data entities to the existing data entities, wherein each of the matching criteria comprises a matching technique;

matching the imported data entities to the existing data entities using the matching criteria, resulting in creating and storing matched entity information, wherein the matched entity information is organized in one or more matched entity data sets associated with one or more subsets of the matching criteria that were matched;

consolidating the imported data entities into the existing data entities;

storing the first user input and second user input as a named criteria set for use in subsequent entity resolution operations.

14. The method of claim 13, wherein the matching technique specifies no conflicts between values of properties of the imported data entities and the existing data entities.

15. The method of claim 13, further comprising:

receiving additional imported data for resolution to the existing data entities;

receiving third user input that specifies a name of a stored criteria set;

loading the stored criteria set;

automatically performing the matching based on the stored criteria set and without receiving the second user input.

16. The method of claim 13, further comprising:

receiving the imported data;

receiving the first user input as a selection of one of the imported data entities and one of the existing data entities in a database;

receiving third user input requesting an entity resolution of the selected imported data entity and the selected existing data entity;

loading the stored criteria set;

automatically performing the matching based on the stored criteria set and performing the consolidating without receiving the second user input.

17. The method of claim 13, further comprising displaying the imported data entities in a histogram display that identifies the imported data entities and corresponding counts of the values of the properties of the imported data entities;

receiving the first user input as selections of one or more of the imported data entities in the histogram display.

18. The method of claim 13, further comprising:

displaying, in a graphical user interface, four visually expandable and collapsible graphical regions associated with steps in an entity resolution process;

wherein a first graphical region when expanded comprises a first list of the one or more imported data entities and one or more of the properties and one or more of the values;

wherein a second graphical region when expanded comprises a first panel comprising the one or more matching criteria for matching the imported data entities to the existing data entities and the matching technique for each of the matching criteria, and a second panel comprising one or more target filters for types of the existing data entities;

wherein a third graphical region when expanded comprises a third panel comprising the subsets of the matching criteria that were matched, and counts of the one or more matched entity data sets associated with the one or more subsets of the matching criteria that were matched, and a fourth panel comprising each matching entity represented in the count of a selected subset;

wherein a fourth graphical region when expanded comprises a fifth panel comprising a second list of one or more associations of resolved entities each comprising one of the imported data entities, one of the existing data entities, and an associated subset of the matching criteria that was matched for that association; and

a sixth panel comprising a third list of one or more new entities to create in the database based on one or more of the imported data entities that did not match any of the existing data entities.

19. The method of claim 13, further comprising determining whether all the data entities were resolved, receiving modified matching criteria applicable to one or more unresolved data entities, and repeating the matching and consolidating steps for the unresolved data entities.

20. The method of claim 13, wherein the first user input comprises a semantic criterion referencing one or more properties of the data entities.

21. An apparatus, comprising:

a database;

entity resolution logic coupled to the database and which when executed by one or more processors causes the one or more processors to perform;

receiving imported data comprising one or more imported data entities each having one or more properties each having one or more values;

receiving first user input that selects one or more of the imported data entities for resolution to one or more existing data entities in a database;

receiving second user input that specifies one or more matching criteria for matching the imported data entities to the existing data entities, wherein each of the matching criteria comprises a matching technique;

matching the imported data entities to the existing data entities using the matching criteria, resulting in creating and storing matched entity information, wherein the matched entity information is organized in one or more matched entity data sets associated with one or more subsets of the matching criteria that were matched;

consolidating the imported data entities into the existing data entities;

storing the first user input and second user input as a named criteria set for use in subsequent entity resolution operations.

22. The apparatus of claim 21, wherein the matching technique specifies no conflicts between values of properties of the imported data entities and the existing data entities.

23. The apparatus of claim 21, the logic further comprising sequences of instructions which, when executed by the processor, cause the processor to perform:

receiving additional imported data for resolution to the existing data entities;

receiving third user input that specifies a name of a stored criteria set;

loading the stored criteria set;

automatically performing the matching based on the stored criteria set and without receiving the second user input.

24. The apparatus of claim 21, the logic further comprising sequences of instructions which, when executed by the processor, cause the processor to perform:

receiving the imported data;

receiving the first user input as a selection of one of the imported data entities and one of the existing data entities in a database;

receiving third user input requesting an entity resolution of the selected imported data entity and the selected existing data entity;

loading the stored criteria set;

automatically performing the matching based on the stored criteria set and performing the consolidating without receiving the second user input.

25. The apparatus of claim 21, the logic further comprising sequences of instructions which, when executed by the processor, cause the processor to perform receiving third user input that specifies one or more target filters for types of the existing data entities, and performing the matching only for corresponding ones of the existing data entities in the database.

26. The apparatus of claim 21, the logic further comprising sequences of instructions which, when executed by the processor, cause the processor to perform displaying the imported data entities in a histogram display that identifies the imported data entities and corresponding counts of the values of the properties of the imported data entities;

receiving the first user input as selections of one or more of the imported data entities in the histogram display.

27. The apparatus of claim 21, the logic further comprising sequences of instructions which, when executed by the processor, cause the processor to perform:

displaying, in a first panel of a graphical user interface, the subsets of the matching criteria that were matched, and counts of the one or more matched entity data sets associated with the one or more subsets of the matching criteria that were matched;

receiving third user input that selects one of the subsets;

displaying, in response to the third user input, in a second panel of the graphical user interface, each matching entity represented in the count of the selected subset.

28. The apparatus of claim 21, the logic further comprising sequences of instructions which, when executed by the processor, cause the processor to perform displaying, in a graphical user interface, results of consolidating the imported data entities into the existing data entities, the results comprising:

a first list of one or more associations of resolved entities each comprising one of the imported data entities, one of the existing data entities, and an associated subset of the matching criteria that was matched for that association;

a second list of one or more new entities to create in the database based on one or more of the imported data entities that did not match any of the existing data entities.

29. The apparatus of claim 21, the logic further comprising sequences of instructions which, when executed by the processor, cause the processor to perform:

displaying, in a graphical user interface, four visually expandable and collapsible graphical regions associated with steps in an entity resolution process;

wherein a first graphical region when expanded comprises a first list of the one or more imported data entities and one or more of the properties and one or more of the values;

wherein a second graphical region when expanded comprises a first panel comprising the one or more matching criteria for matching the imported data entities to the existing data entities and the matching technique for each of the matching criteria, and a second panel comprising one or more target filters for types of the existing data entities;

wherein a third graphical region when expanded comprises a third panel comprising the subsets of the matching criteria that were matched, and counts of the one or more matched entity data sets associated with the one or more subsets of the matching criteria that were matched, and a fourth panel comprising each matching entity represented in the count of a selected subset;

wherein a fourth graphical region when expanded comprises a fifth panel comprising a second list of one or more associations of resolved entities each comprising one of the imported data entities, one of the existing data entities, and an associated subset of the matching criteria that was matched for that association; and

a sixth panel comprising a third list of one or more new entities to create in the database based on one or more of the imported data entities that did not match any of the existing data entities.

30. The apparatus of claim 21, further comprising logic which when executed cause determining whether all the data entities were resolved, receiving modified matching criteria applicable to one or more unresolved data entities, and repeating the matching and consolidating steps for the unresolved data entities.

31. The apparatus of claim 21, wherein the first user input comprises a semantic criterion referencing one or more properties of the data entities.

32. An apparatus, comprising:

means for receiving imported data comprising one or more imported data entities each having one or more properties each having one or more values;

means for receiving first user input that selects one or more of the imported data entities for resolution to one or more existing data entities in a database;

means for receiving second user input that specifies one or more matching criteria for matching the imported data entities to the existing data entities, wherein each of the matching criteria comprises a matching technique;

means for matching the imported data entities to the existing data entities using the matching criteria, resulting in creating and storing matched entity information, wherein the matched entity information is organized in one or more matched entity data sets associated with one or more subsets of the matching criteria that were matched;

means for consolidating the imported data entities into the existing data entities;

means for storing the first user input and second user input as a named criteria set for use in subsequent entity resolution operations.

33. The apparatus of claim 32, further comprising:

means for receiving additional imported data for resolution to the existing data entities;

means for receiving third user input that specifies a name of a stored criteria set;

means for loading the stored criteria set;

means for automatically performing the matching based on the stored criteria set and without receiving the second user input.

34. The apparatus of claim 32, further comprising:

means for receiving the imported data;

means for receiving the first user input as a selection of one of the imported data entities and one of the existing data entities in a database;

means for receiving third user input requesting an entity resolution of the selected imported data entity and the selected existing data entity;

means for loading the stored criteria set;

means for automatically performing the matching based on the stored criteria set and performing the consolidating without receiving the second user input.

35. The apparatus of claim 32, further comprising means for displaying the imported data entities in a histogram display that identifies the imported data entities and corresponding counts of the values of the properties of the imported data entities;

means for receiving the first user input as selections of one or more of the imported data entities in the histogram display.

36. The apparatus of claim 32, further comprising means for displaying, in a graphical user interface, four visually expandable and collapsible graphical regions associated with steps in an entity resolution process;

wherein a first graphical region when expanded comprises a first list of the one or more imported data entities and one or more of the properties and one or more of the values;

wherein a second graphical region when expanded comprises a first panel comprising the one or more matching criteria for matching the imported data entities to the existing data entities and the matching technique for each of the matching criteria, and a second panel comprising one or more target filters for types of the existing data entities;

wherein a third graphical region when expanded comprises a third panel comprising the subsets of the matching criteria that were matched, and counts of the one or more matched entity data sets associated with the one or more subsets of the matching criteria that were matched, and a fourth panel comprising each matching entity represented in the count of a selected subset;

wherein a fourth graphical region when expanded comprises a fifth panel comprising a second list of one or more associations of resolved entities each comprising one of the imported data entities, one of the existing data entities, and an associated subset of the matching criteria that was matched for that association; and

a sixth panel comprising a third list of one or more new entities to create in the database based on one or more of the imported data entities that did not match any of the existing data entities.