Systems and methods for node deduplication based on a node merging policy

• US 10,671,612 B2
• Filed: 05/21/2019
• Issued: 06/02/2020
• Est. Priority Date: 05/24/2018
• Status: Active Grant

• Current Assignee: People.ai, Inc.
• Original Assignee: People.ai, Inc.
• Inventors: Rogynskyy, Oleg, Markovsky, Yury, Brunets, Yurii, Kysylov, Maksym
• Primary Examiner(s): Pham, Michael
• Application Number:
  US16/418,539
• Publication Number:
  US 20190361851A1
• Time in Patent Office: 378 Days
• Field of Search: 707602, 707661
• CPC Class Codes: G06F 11/3024 where the computing system component is a central processing unit [CPU]
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/212 with details for data modelling support
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/219 Managing data history or versioning querying versioned data G06F16/2474; querying temporal data G06F16/2477
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/221 Column-oriented storage; Management thereof
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/2264 Multidimensional index structures
• CPC Class Codes: G06F 16/2272 Management thereof
• CPC Class Codes: G06F 16/23 Updating
• CPC Class Codes: G06F 16/235 Update request formulation
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/2379 Updates performed during online database operations; commit processing
• CPC Class Codes: G06F 16/2386 : Bulk updating operations data conversion details G06F16/258
• CPC Class Codes: G06F 16/245 : Query processing
• CPC Class Codes: G06F 16/24558 : Binary matching operations
• CPC Class Codes: G06F 16/24564 : Applying rules; Deductive queries
• CPC Class Codes: G06F 16/2457 : with adaptation to user needs
• CPC Class Codes: G06F 16/24575 : using context
• CPC Class Codes: G06F 16/24578 : using ranking
• CPC Class Codes: G06F 16/2477 : Temporal data queries
• CPC Class Codes: G06F 16/254 : Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 16/256 : in federated or virtual databases
• CPC Class Codes: G06F 16/26 : Visual data mining; Browsing structured data
• CPC Class Codes: G06F 16/27 : Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/273 : Asynchronous replication or reconciliation
• CPC Class Codes: G06F 16/28 : Databases characterised by their database models, e.g. relational or object models
• CPC Class Codes: G06F 16/285 : Clustering or classification
• CPC Class Codes: G06F 16/288 : Entity relationship models
• CPC Class Codes: G06F 16/289 : Object oriented databases
• CPC Class Codes: G06F 16/29 : Geographical information databases
• CPC Class Codes: G06F 16/313 : Selection or weighting of terms for indexing
• CPC Class Codes: G06F 16/337 : Profile generation, learning or modification
• CPC Class Codes: G06F 16/355 : Class or cluster creation or modification
• CPC Class Codes: G06F 16/901 : Indexing; Data structures therefor; Storage structures for retrieval from the web G06F16/951
• CPC Class Codes: G06F 16/9024 : Graphs; Linked lists G06F16/9027 takes precedence
• CPC Class Codes: G06F 16/90344 : by using string matching techniques
• CPC Class Codes: G06F 16/9035 : Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/906 : Clustering; Classification
• CPC Class Codes: G06F 16/93 : Document management systems
• CPC Class Codes: G06F 16/9535 : Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 21/6218 : to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 21/6245 : Protecting personal data, e.g. for financial or medical purposes
• CPC Class Codes: G06F 40/20 : Natural language analysis semantic analysis of natural language G06F40/30
• CPC Class Codes: G06F 40/205 : Parsing
• CPC Class Codes: G06F 40/237 : Lexical tools
• CPC Class Codes: G06F 40/295 : Named entity recognition
• CPC Class Codes: G06F 40/30 : Semantic analysis
• CPC Class Codes: G06F 7/14 : Merging, i.e. combining at least two sets of record carriers each arranged in the same ordered sequence to produce a single set having the same ordered sequence
• CPC Class Codes: G06F 9/542 : Event management; Broadcasting; Multicasting; Notifications
• CPC Class Codes: G06N 20/00 : Machine learning
• CPC Class Codes: G06N 3/08 : Learning methods
• CPC Class Codes: G06N 5/022 : Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 5/025 : Extracting rules from data
• CPC Class Codes: G06N 5/04 : Inference or reasoning models
• CPC Class Codes: G06N 7/02 : using fuzzy logic computing arrangements based on biological models G06N3/00; computing arrangements using knowledge-based models G06N5/00
• CPC Class Codes: G06Q 10/04 : Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem" market predictions or forecasting for commercial activities G06Q30/0202
• CPC Class Codes: G06Q 10/063114 : Status monitoring or status determination for a person or group
• CPC Class Codes: G06Q 10/06312 : Adjustment or analysis of established resource schedule, e.g. resource or task levelling, or dynamic rescheduling
• CPC Class Codes: G06Q 10/06315 : Needs-based resource requirements planning or analysis
• CPC Class Codes: G06Q 10/06393 : Score-carding, benchmarking or key performance indicator [KPI] analysis
• CPC Class Codes: G06Q 10/06398 : Performance of employee with respect to a job function
• CPC Class Codes: G06Q 10/10 : Office automation; Time management
• CPC Class Codes: G06Q 10/107 : Computer-aided management of electronic mailing [e-mailing]
• CPC Class Codes: G06Q 10/109 : Time management, e.g. calendars, reminders, meetings or time accounting
• CPC Class Codes: G06Q 10/1091 : Recording time for administrative or management purposes
• CPC Class Codes: G06Q 10/1095 : Meeting or appointment
• CPC Class Codes: G06Q 30/0201 : Market modelling; Market analysis; Collecting market data
• CPC Class Codes: G06Q 50/01 : Social networking
• CPC Class Codes: G06Q 50/22 : Social work or social welfare, e.g. community support activities or counselling services
• CPC Class Codes: G16H 15/00 : ICT specially adapted for medical reports, e.g. generation or transmission thereof
• CPC Class Codes: G16H 50/20 : for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16H 50/30 : for calculating health indices; for individual health risk assessment
• CPC Class Codes: H04L 12/14 : Charging , metering or billing arrangements for data wireline or wireless communications
• CPC Class Codes: H04L 12/1407 : Policy-and-charging control [PCC] architecture
• CPC Class Codes: H04L 12/1831 : Tracking arrangements for later retrieval, e.g. recording contents, participants activities or behavior, network status
• CPC Class Codes: H04L 2101/00 : Indexing scheme associated with group H04L61/00
• CPC Class Codes: H04L 2101/35 : containing special prefixes
• CPC Class Codes: H04L 2101/37 : E-mail addresses
• CPC Class Codes: H04L 41/082 : the condition being updates or upgrades of network functionality
• CPC Class Codes: H04L 41/14 : Network analysis or design
• CPC Class Codes: H04L 43/026 : using flow identification
• CPC Class Codes: H04L 43/045 : for graphical visualisation of monitoring data
• CPC Class Codes: H04L 43/062 : related to network traffic
• CPC Class Codes: H04L 43/065 : related to network devices
• CPC Class Codes: H04L 43/067 : using time frame reporting
• CPC Class Codes: H04L 43/08 : Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
• CPC Class Codes: H04L 43/0876 : Network utilisation, e.g. volume of load or congestion level
• CPC Class Codes: H04L 43/14 : using software, i.e. software packages network security related monitoring H04L63/1408
• CPC Class Codes: H04L 51/046 : Interoperability with other network applications or services
• CPC Class Codes: H04L 51/212 : using filtering or selective blocking
• CPC Class Codes: H04L 51/234 : for tracking messages
• CPC Class Codes: H04L 51/42 : Mailbox-related aspects, e.g. synchronisation of mailboxes
• CPC Class Codes: H04L 61/45 : Network directories; Name-to-address mapping
• CPC Class Codes: H04L 61/4594 : Address books, i.e. directories containing contact information about correspondents telephone directories in user terminals H04M1/27453
• CPC Class Codes: H04L 67/125 : involving control of end-device applications over a network
• CPC Class Codes: H04L 67/30 : Profiles
• CPC Class Codes: H04L 67/303 : Terminal profiles
• CPC Class Codes: H04L 67/306 : User profiles
• CPC Class Codes: H04L 67/535 : Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04M 15/00 : Arrangements for metering, time-control or time indication ; Metering, charging or billing arrangements for voice wireline or wireless communications, e.g. VoIP
• CPC Class Codes: H04M 15/41 : Billing record details, i.e. parameters, identifiers, structure of call data record [CDR]
• CPC Class Codes: H04M 15/50 : for cross-charging network operators
• CPC Class Codes: H04M 15/58 : based on statistics of usage or network monitoring
• CPC Class Codes: H04M 15/60 : based on actual use of network resources
• CPC Class Codes: H04M 15/66 : Policy and charging system
• CPC Class Codes: H04M 15/755 : Account identification
• CPC Class Codes: H04M 2203/556 : Statistical analysis and interpretation
• CPC Class Codes: H04M 3/2218 : Call detail recording
• CPC Class Codes: H04M 3/436 : Arrangements for screening incoming calls , i.e. evaluating the characteristics of a call before deciding whether to answer it based on the calling party profile H04M3/42059; based on location H04M3/42348; based on presence H04M3/42365; diversion H04M3/54
• CPC Class Codes: H04M 3/5175 : Call or contact centers supervision arrangements
• CPC Class Codes: H04M 3/56 : Arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities video conference systems H04N7/15
• CPC Class Codes: H04W 4/24 : Accounting or billing

Abstract

The present disclosure relates to node deduplication based on a node merging policy. A plurality of node profiles may be maintained. Each node profile may include plurality of fields. Each field of the plurality of fields may include one or more values. For a node profile including one or more field-value pairs, each field-value pair may be identified. Each field-value pair may include a value of a field. A respective weight may be assigned to each field-value pair based on a number of field-value pairs or a confidence score. Based on the respective weights assigned to each field-value pair, it may be determined that node profiles satisfy a node profile merging policy. The node profiles may be merged based on the node profiles satisfying the node profile merging policy. Field-value pairs including a value of a field may be identified.

Claims

1. A method, comprising:

maintaining, by one or more processors, a first node profile including a plurality of first field-value pairs, each first field-value pair of the plurality of first field-value pairs identifying a corresponding first field and a corresponding first value;

maintaining, by the one or more processors, a plurality of second node profiles, each second node profile of the plurality of second node profiles including a plurality of second field-value pairs, each second field-value pair of the plurality of second field-value pairs identifying a corresponding second field and a corresponding second value;

assigning, by the one or more processors, to each first field-value pair of the first node profile, a respective weight based on i) a number of second field-value pairs included in the plurality of second node profiles that match the first field-value pair of the first node profile or ii) a confidence score of the first value of the first field-value pair, the confidence score based on a number of data points that i) identify the first field-value pair and ii) are stored in association with the first node profile;

determining, by the one or more processors, using the respective weight assigned to each first field-value pair of the first node profile, that the first node profile and a candidate second node profile of the plurality of second node profiles satisfy a node profile merging policy; and

merging, by the one or more processors, the first node profile and the candidate second node profile responsive to determining that the first node profile and the candidate second node profile satisfy the node profile merging policy.

2. The method of claim 1, wherein the number of second field-value pairs included in the plurality of second node profiles that match the first field-value pair is a first number of field-value pairs;

and

wherein assigning, to each first field-value pair of the first node profile, the respective weight comprises;

determining a second number of second field-value pairs included in the plurality of second node profiles that identifies a respective field that matches the field of the first field-value pair;

determining a ratio between the first number and the second number; and

using the ratio between the first number and the second number to assign the weight.

3. The method of claim 1, wherein assigning the respective weight based on the confidence score of the first value of the first field-value pair comprises determining the confidence score of the first value based on a contribution score of each data point of the number of data points that is identified in a value data structure of the first field-value pair.

4. The method of claim 3, wherein the contribution score of each data point is based on a timestamp of the data point or a trust score of a source of the data point.

5. The method of claim 3, wherein the data point identifies an electronic activity linked to the first node profile based on comparing activity field-value pairs of the electronic activity to first field-value pairs of the first node profile.

6. The method of claim 3, wherein the data point identifies a record object linked to the first node profile based on comparing object field-value pairs of the record object to first field-value pairs of the first node profile.

7. The method of claim 1, wherein determining that the first node profile and the second node profile satisfy the node profile merging policy includes assigning, by the one or more processors, at least one of a completeness score or a recency score to at least one of the first node profile or the candidate second node profile.

8. The method of claim 1, wherein determining that the first node profile and the candidate second node profile satisfy the node profile merging policy comprises selecting the candidate second node profile based on an event detected using electronic activities linked to the candidate second node profile within a predetermined time period of generating the first node profile.

9. The method of claim 1, wherein determining that the first node profile and the candidate second node profile satisfy the node profile merging policy comprises selecting the candidate second node profile based on determining that a contact identifier of the candidate second node profile has changed from a first status to a second status.

10. The method of claim 1, further comprising:

parsing, by the one or more processors, the one or more first field-value pairs of the first node profile to identify an email address of the first node profile;

identifying, by the one or more processors from the email address, at least one of a first name or a last name corresponding to the email address; and

matching, by the one or more processors, the at least one of the first name or the last name to corresponding field-value pairs of the second node profile to determine whether the first node profile and the second node profile satisfy the node profile merging policy.

11. The method of claim 1, further comprising:

identifying, by the one or more processors, a first time stamp of a latest electronic activity transmitted or received by a first electronic account associated with the first node profile;

identifying, by the one or more processors, a second time stamp of an earliest electronic activity transmitted or received by a second electronic account associated with the candidate second node profile; and

determining, by the one or more processors, that the first node profile and the second node profile satisfy the node profile merging policy based on the first time stamp being within a predetermined period of time of the second time stamp.

12. A system comprising:

one or more processors configured by machine-readable instructions to;

maintain a first node profile including a plurality of first field-value pairs, each first field-value pair of the plurality of first field-value pairs identifying a corresponding first field and a corresponding first value;

maintain a plurality of second node profiles, each second node profile of the plurality of second node profiles including a plurality of second field-value pairs, each second field-value pair of the plurality of second field-value pairs identifying a corresponding second field and a corresponding second value;

assign, to each first field-value pair of the first node profile, a respective weight based on i) a number of second field-value pairs included in the plurality of second node profiles that match the first field-value pair of the first node profile or ii) a confidence score of the first value of the first field-value pair, the confidence score based on a number of data points that i) identify the first field-value pair and ii) are stored in association with the first node profile;

determine, using the respective weight assigned to each first field-value pair of the first node profile, that the first node profile and candidate second node profile of the plurality of second node profiles satisfy a node profile merging policy; and

merge the first node profile and the candidate second node profile responsive to determining that the first node profile and the candidate second node profile satisfy the node profile merging policy.

13. The system of claim 12, wherein the number of second field-value pairs included in the plurality of second node profiles that match the first field-value pair is a first number of field-value pairs;

and

wherein the one or more processors are configured to assign, to each first field-value pair of the first node profile, the respective weight by;

determining a second number of second field-value pairs included in the plurality of second node profiles that identifies a respective field that matches the field of the first field-value pair;

determining a ratio between the first number and the second number; and

using the ratio between the first number and the second number to assign the weight.

14. The system of claim 12, wherein to assign the respective weight based on the confidence score of the first value of the first field-value pair, the one or more processors are configured to determine the confidence score of the first value based on a contribution score of each data point of the number of data points that is identified in a value data structure of the first field-value pair.

15. The system of claim 14, wherein the contribution score of each data point is based on a timestamp of the data point or a trust score of a source of the data point.

16. The system of claim 14, wherein the data point identifies an electronic activity linked to the first node profile based on comparing activity field-value pairs of the electronic activity to first field-value pairs of the first node profile.

17. The system of claim 14, wherein the data point identifies a record object linked to the first node profile based on comparing object field-value pairs of the record object to first field-value pairs of the first node profile.

18. The system of claim 12, wherein the one or more processors are configured to determine that the first node profile and the second node profile satisfy the node profile merging policy by assigning at least one of a completeness score or a recency score to at least one of the first node profile or the candidate second node profile.

19. The system of claim 12, wherein the one or more processors are configured to determine that the first node profile and the second node profile satisfy the node profile merging policy by selecting the candidate second node profile based on an event detected using electronic activities linked to the candidate second node profile within a predetermined time period of generating the first node profile.

20. A non transitory computer-readable storage medium having instructions embodied thereon, the instructions being executable by one or more processors to:

maintain a first node profile including a plurality of first field-value pairs, each first field-value pair of the plurality of first field-value pairs identifying a corresponding first field and a corresponding first value;

maintain a plurality of second node profiles, each second node profile of the plurality of second node profiles including a plurality of second field-value pairs, each second field-value pair of the plurality of second field-value pairs identifying a corresponding second field and a corresponding second value;

assign, to each first field-value pair of the first node profile, a respective weight based on i) a number of second field-value pairs included in the plurality of second node profiles that match the first field-value pair of the first node profile or ii) a confidence score of the first value of the first field-value pair, the confidence score based on a number of data points that i) identify the first field-value pair and ii) are stored in association with the first node profile;

determine, using the respective weight assigned to each first field-value pair of the first node profile, that the first node profile and candidate second node profile of the plurality of second node profiles satisfy a node profile merging policy; and

merge the first node profile and the candidate second node profile responsive to determining that the first node profile and the candidate second node profile satisfy the node profile merging policy.