Resolving database entity information

• US 8,554,719 B2
• Filed: 10/18/2007
• Issued: 10/08/2013
• Est. Priority Date: 10/18/2007
• Status: Active Grant

• Current Assignee: Palantir Technologies Incorporated
• Original Assignee: Palantir Technologies Incorporated
• Inventors: McGrew, Robert J., Cohen, Stephen
• Primary Examiner(s): Leroux, Etienne
• Assistant Examiner(s): DAYE, CHELCIE L
• Application Number:
  US11/874,644
• Publication Number:
  US 20090106242A1
• Time in Patent Office: 2,182 Days
• Field of Search: 707/602, 707/748
• US Class Current: 707/602
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/219 Managing data history or versioning querying versioned data G06F16/2474; querying temporal data G06F16/2477
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/24556 Aggregation; Duplicate elimination
• CPC Class Codes: G06F 16/252 between a Database Management System and a front-end application
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/285 Clustering or classification

Abstract

Entity resolution in a database comprises receiving imported data comprising imported data entities each having properties each having values; receiving first user input that selects the imported data entities for resolution to existing data entities in a database; receiving second user input that specifies matching criteria for matching the imported data entities to the existing data entities, wherein each of the matching criteria comprises a matching technique; matching the imported data entities to the existing data entities using the matching criteria, resulting in creating and storing matched entity information, wherein the matched entity information is organized in matched entity data sets associated with subsets of the matching criteria that were matched; consolidating the imported data entities into the existing data entities; storing the first user input and second user input as a named criteria set for use in subsequent entity resolution operations.

Claims

1. A non-transitory computer-readable storage medium comprising one or more stored sequences of instructions which, when executed by one or more processors, cause the one or more processors to perform:

receiving imported data comprising imported data entities each having one or more properties each having one or more values;

matching the imported data entities to existing data entities in a database using matching criteria, resulting in creating and storing matched entity information, wherein the matched entity information indicates that a subset of the imported data entities matched a subset of the existing data entities for a subset of the matching criteria, and at least another, different subset of the imported data entities matched another, different subset of the existing data entities for another, different subset of the matching criteria;

causing display of;

the matched entity information, and

for each entity of the imported data entities, options comprising;

a first option to select the entity for consolidation when the entity is matched correctly;

wherein consolidation of the entity comprises adding one or more values from the entity into a matched existing data entity;

a second option to select the entity for further resolution when the entity is to be further resolved;

receiving user inputs comprising a first user input and a second user input;

wherein the first user input selects, for consolidation, a first one or more data entities of the imported data entities that are matched correctly;

wherein the second user input selects, for further resolution, a second one or more data entities of the imported data entities that are to be further resolved;

after receiving at least the first user input, consolidating at least the first one or more data entities but not the second one or more data entities.

2. The non-transitory computer-readable storage medium of claim 1, wherein one of the plurality of matching techniques specifies no conflicts between values of properties of the imported data entities and the existing data entities.

3. The non-transitory computer-readable storage medium of claim 1, further comprising sequences of instructions which, when executed by the one or more processors, cause the one or more processors to perform:

receiving additional imported data for resolution to the existing data entities;

receiving a third user input that specifies a name of a stored criteria set;

loading the stored criteria set;

automatically performing the matching based on the stored criteria set.

4. The non-transitory computer-readable storage medium of claim 1, further comprising sequences of instructions which, when executed by the one or more processors, cause the one or more processors to perform:

receiving a third user input as a selection of one of the imported data entities and one of the existing data entities in the database;

receiving a fourth user input requesting an entity resolution of the selected imported data entity and the selected existing data entity;

loading a named criteria set;

automatically performing the matching based on the named criteria set.

5. The non-transitory computer-readable storage medium of claim 1, further comprising sequences of instructions which, when executed by the one or more processors, cause the one or more processors to perform receiving a third user input that specifies one or more target filters for types of the existing data entities, wherein the matching is not performed for other types of data entities.

6. The non-transitory computer-readable storage medium of claim 1, further comprising sequences of instructions which, when executed by the one or more processors, cause the one or more processors to perform displaying the imported data entities in a histogram display that identifies the imported data entities and corresponding counts of the values of the properties of the imported data entities;

receiving a third user input as selections of one or more of the imported data entities in the histogram display.

7. The non-transitory computer-readable storage medium of claim 1, further comprising sequences of instructions which, when executed by the one or more processors, cause the one or more processors to perform:

displaying, in a first panel of a graphical user interface, the multiple subsets of the matching criteria that were matched, and counts of matched imported data entities in the multiple matched subsets of imported data entities associated with the multiple subsets of the matching criteria;

displaying, in response to a third user input, in a second panel of the graphical user interface, each matched imported data entity represented in the count of a selected subset of imported data entities.

8. The non-transitory computer-readable storage medium of claim 1, wherein the instructions that cause consolidating comprise instructions which, when executed by the one or more processors, cause the one or more processors to perform merging at least the first one or more data entities into a set of matched existing data entities from the existing data entities.

9. The non-transitory computer-readable storage medium of claim 1, further comprising sequences of instructions which, when executed by the one or more processors, cause the one or more processors to perform displaying, in a graphical user interface, results of consolidating the at least the first one or more data entities into a set of matched existing data entities from the existing data entities, the results comprising:

a first list of one or more associations of resolved entities each comprising one of at least the first one or more data entities, one of the set of matched existing data entities, and an associated subset of the matching criteria that was matched for the one or more associations;

a second list of one or more new entities to create in the database based on those of the imported data entities that did not match any of the existing data entities.

10. The non-transitory computer-readable storage medium of claim 1, further comprising sequences of instructions which, when executed by the one or more processors, cause the one or more processors to perform:

displaying, in a graphical user interface, four visually expandable and collapsible graphical regions associated with steps in an entity resolution process;

wherein a first graphical region when expanded comprises a first list of the imported data entities and one or more of the properties and one or more of the values;

wherein a second graphical region when expanded comprises a first panel comprising the matching criteria for matching the imported data entities to the existing data entities and a particular matching technique for each of the matching criteria, and a second panel comprising one or more target filters for types of the existing data entities;

wherein a third graphical region when expanded comprises a third panel comprising the multiple subsets of the matching criteria that were matched, and counts of matched imported data entities of the multiple matched subsets of imported data entities associated with the multiple subsets of the matching criteria, and a fourth panel comprising each matched imported data entity represented in the count of a selected subset of the imported data entities;

wherein a fourth graphical region when expanded comprises a fifth panel comprising a second list of one or more associations of resolved entities each comprising one of the imported data entities, one of the existing data entities, and an associated subset of the matching criteria that was matched for the one or more associations; and

a sixth panel comprising a third list of one or more new entities to create in the database based on those of the imported data entities that did not match any of the existing data entities.

11. The non-transitory computer-readable storage medium of claim 1, further comprising instructions which when executed by the one or more processors cause the one or more processors to perform determining whether all the imported data entities were resolved, receiving modified matching criteria applicable to one or more unresolved data entities, and repeating the matching and consolidating steps for the one or more unresolved data entities.

12. The non-transitory computer-readable storage medium of claim 1, further comprising instructions which when executed by the one or more processors cause the one or more processors to perform receiving a third user input that selects the imported data entities from the imported data for resolution, wherein the third user input comprises a semantic criterion referencing one or more properties of the imported data entities, and wherein the imported data comprises one or more other data entities that are not selected for resolution.

21. The non-transitory computer-readable storage medium of claim 1, wherein the options further comprise a third option to select the entity for removal when the entity is matched incorrectly, and wherein the user inputs further comprise a third user input that selects, for removal, a third one or more data entities, of the imported data entities, that are matched incorrectly.

22. The non-transitory computer-readable storage medium of claim 1, further comprising instructions which when executed by the one or more processors cause the one or more processors to perform, after receiving the user inputs, resolving the second one or more entities that are selected for further resolution.

23. The non-transitory computer-readable storage medium of claim 1, wherein the matched entity information is a first matched entity information, further comprising instructions which when executed by the one or more processors cause the one or more processors to perform, after receiving the user inputs, resolving the second one or more entities that are selected for further resolution at least in part by:

matching the second one or more entities to the existing data entities using at least one matching criteria that is different from the multiple subsets of matching criteria, resulting in creating and storing a second matched entity information, wherein the second matched entity information indicates that a subset of the second one or more entities matched a subset of the existing data entities for the at least one matching criteria;

causing display of;

the second matched entity information, and

one or more options to select one or more correctly matched entities of the second one or more entities for consolidation;

receiving one or more user inputs selecting the one or more correctly matched entities of the second one or more entities for consolidation;

after receiving the one or more user inputs, consolidating the one or more correctly matched entities of the second one or more entities for consolidation.

24. The non-transitory computer-readable storage medium of claim 1, further comprising instructions which when executed by the one or more processors cause the one or more processors to perform:

receiving a third user input that selects the imported data entities from the imported data for resolution, wherein the imported data comprises one or more other data entities that are not selected for resolution.

25. The non-transitory computer-readable storage medium of claim 1, further comprising instructions which when executed by the one or more processors cause the one or more processors to perform:

receiving a third user input that specifies the matching criteria for matching the imported data entities to the existing data entities, wherein each of the matching criteria comprises a particular matching technique from a plurality of matching techniques.

26. The non-transitory computer-readable storage medium of claim 1, further comprising instructions which when executed by the one or more processors cause the one or more processors to perform:

receiving a third user input that selects the imported data entities from the imported data for resolution, wherein the imported data comprises one or more other data entities that are not selected for resolution;

receiving a fourth user input that specifies the matching criteria for matching the imported data entities to the existing data entities, wherein each of the matching criteria comprises a particular matching technique from a plurality of matching techniques

based on the third user input and the fourth user input, storing a named criteria set for use in subsequent entity resolution operations.

27. The non-transitory computer-readable storage medium of claim 1, wherein the matched entity information is organized in multiple matched entity data sets respectively associated with multiple different subsets of the matching criteria that were matched.

13. A machine-implemented method, comprising:

receiving imported data comprising imported data entities each having one or more properties each having one or more values;

matching the imported data entities to existing data entities in a database using matching criteria, resulting in creating and storing matched entity information, wherein the matched entity information indicates that a subset of the imported data entities matched a subset of the existing data entities for a subset of the matching criteria, and at least another, different subset of the imported data entities matched another, different subset of the existing data entities for another, different subset of the matching criteria;

causing display of;

the matched entity information, and

for each entity of the imported data entities, options comprising;

a first option to select the entity for consolidation when the entity is matched correctly;

wherein consolidation of the entity comprises adding one or more values from the entity into a matched existing data entity;

a second option to select the entity for further resolution when the entity is to be further resolved;

receiving user inputs comprising a first user input and a second user input;

wherein the first user input selects, for consolidation, a first one or more data entities of the one or more imported data entities that are matched correctly;

wherein the second user input selects, for further resolution, a second one or more data entities of the one or more imported data entities that are to be further resolved;

after receiving at least the first user input, consolidating at least the first one or more data entities but not the second one or more data entities;

wherein the method is performed by one or more computer systems.

14. The method of claim 13, wherein one of the plurality of matching techniques specifies no conflicts between values of properties of the imported data entities and the existing data entities.

15. The method of claim 13, further comprising:

receiving additional imported data for resolution to the existing data entities;

receiving a third user input that specifies a name of a stored criteria set;

loading the stored criteria set;

automatically performing the matching based on the stored criteria set.

16. The method of claim 13, further comprising:

receiving a third user input as a selection of one of the imported data entities and one of the existing data entities in the database;

receiving a fourth user input requesting an entity resolution of the selected imported data entity and the selected existing data entity;

loading a named criteria set;

automatically performing the matching based on the named criteria set.

17. The method of claim 13, further comprising displaying the imported data entities in a histogram display that identifies the imported data entities and corresponding counts of the values of the properties of the imported data entities;

receiving a third user input as selections of one or more of the imported data entities in the histogram display.

18. The method of claim 13, further comprising:

displaying, in a graphical user interface, four visually expandable and collapsible graphical regions associated with steps in an entity resolution process;

wherein a first graphical region when expanded comprises a first list of the imported data entities and one or more of the properties and one or more of the values;

wherein a second graphical region when expanded comprises a first panel comprising the matching criteria for matching the imported data entities to the existing data entities and a particular matching technique for each of the matching criteria, and a second panel comprising one or more target filters for types of the existing data entities;

wherein a third graphical region when expanded comprises a third panel comprising the multiple subsets of the matching criteria that were matched, and counts of matched imported data entities of the multiple matched subsets of imported data entities associated with the multiple subsets of the matching criteria, and a fourth panel comprising each matched imported data entity represented in the count of a selected subset of the imported data entities;

wherein a fourth graphical region when expanded comprises a fifth panel comprising a second list of one or more associations of resolved entities each comprising one of the imported data entities, one of the existing data entities, and an associated subset of the matching criteria that was matched for the one or more associations; and

a sixth panel comprising a third list of one or more new entities to create in the database based on those of the imported data entities that did not match any of the existing data entities.

19. The method of claim 13, further comprising determining whether all the imported data entities were resolved, receiving modified matching criteria applicable to one or more unresolved data entities, and repeating the matching and consolidating steps for the one or more unresolved data entities.

20. The method of claim 13, further comprising instructions which when executed by the one or more processors cause the one or more processors to perform receiving a third user input that selects the imported data entities for resolution, wherein the third user input comprises a semantic criterion referencing one or more properties of the imported data entities, and wherein the imported data comprises one or more other imported data entities that are not selected for resolution.

28. The method of claim 13, wherein the options further comprise a third option to select the entity for removal when the entity is matched incorrectly, and

wherein the user inputs further comprise a third user input that selects, for removal, a

third one or more data entities of the imported data entities that are matched incorrectly.

29. The method of claim 13, further comprising, after receiving the user inputs, resolving the second one or more entities that are selected for further resolution.

30. The method of claim 13, wherein the matched entity information is a first matched entity information, further comprising, after receiving the user inputs, resolving the second one or more entities that are selected for further resolution at least in part by:

matching the second one or more entities to the existing data entities using at least one matching criteria that is different from the multiple subsets of matching criteria, resulting in creating and storing a second matched entity information, wherein the second matched entity information indicates that a subset of the second one or more entities matched a subset of the existing data entities for the at least one matching criteria;

causing display of;

the second matched entity information, and

one or more options to select one or more correctly matched entities of the second one or more entities for consolidation;

receiving one or more user inputs selecting the one or more correctly matched entities of the second one or more entities for consolidation;

after receiving the one or more user inputs, consolidating the one or more correctly matched entities of the second one or more entities for consolidation.

31. The method of claim 13, further comprising:

receiving a third user input that selects the imported data entities in the imported data for resolution, wherein the imported data comprises one or more other data entities that are not selected for resolution.

32. The method of claim 13, further comprising:

receiving a third user input that specifies the matching criteria for matching the imported data entities to the existing data entities, wherein each of the matching criteria comprises a particular matching technique from a plurality of matching techniques.

33. The method of claim 13, further comprising:

receiving a third user input that selects the imported data entities in the imported data for resolution, wherein the imported data comprises one or more other data entities that are not selected for resolution;

receiving a fourth user input that specifies the matching criteria for matching the imported data entities to the existing data entities, wherein each of the matching criteria comprises a particular matching technique from a plurality of matching techniques based on the third user input and the fourth user input, storing a named criteria set for use in subsequent entity resolution operations.

34. The method of claim 13, wherein the matched entity information is organized in multiple matched entity data sets respectively associated with multiple different subsets of the matching criteria that were matched.

35. The method of claim 13, further comprising receiving a third user input that specifies one or more target filters for types of the existing data entities, wherein the matching is not performed for other types of data entities.

36. The method of claim 13, further comprising:

displaying, in a first panel of a graphical user interface, the multiple subsets of the matching criteria that were matched, and counts of matched imported data entities in the multiple matched subsets of imported data entities associated with the multiple subsets of the matching criteria;

displaying, in response to a third user input, in a second panel of the graphical user interface, each matched imported data entity represented in the count of a selected subset of imported data entities.

37. The method of claim 13, wherein consolidating comprises merging at least the first one or more data entities into a set of matched existing data entities from the existing data entities.

38. The method of claim 13, further comprising displaying, in a graphical user interface, results of consolidating at least the first one or more data entities into a set of matched existing data entities from the existing data entities, the results comprising:

a first list of one or more associations of resolved entities each comprising one of at least the first one or more data entities, one of the set of matched existing data entities, and an associated subset of the matching criteria that was matched for the one or more associations;

a second list of one or more new entities to create in the database based on those of the imported data entities that did not match any of the existing data entities.