System for determining and optimizing for relevance in match-making systems

• US 10,380,158 B2
• Filed: 08/08/2016
• Issued: 08/13/2019
• Est. Priority Date: 07/01/2010
• Status: Active Grant

• Current Assignee: Match Group, LLC (Match Group, Inc.)
• Original Assignee: Match Group, LLC (Match Group, Inc.)
• Inventors: Diaz, Fernando, Metzler, Donald, Amer-Yahia, Sihem
• Primary Examiner(s): Owyang, Michelle N
• Application Number:
  US15/231,181
• Publication Number:
  US 20160342684A1
• Time in Patent Office: 1,100 Days
• Field of Search: 707705-780
• CPC Class Codes: G06F 16/337 Profile generation, learning or modification
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/048 Fuzzy inferencing
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

Disclosed are methods and apparatus for automatically determining the relevance of matches between entities. A set of one or more indicators of relevance for each of a plurality of matches may be detected, where each of the plurality of matches exists between a first entity and a different one of a plurality of entities. Each set of one or more indicators of relevance indicates a degree of two-way interest for a corresponding one of the plurality of matches, the degree of two-way interest indicating both a degree of interest of the first entity in the corresponding one of the plurality of entities and a degree of interest of the corresponding one of the plurality of entities in the first entity. A probability of relevance of each of the plurality of matches may be determined based at least in part upon a corresponding set of one or more indicators of relevance. Each of the plurality of matches may be ranked based at least in part on the corresponding probability of relevance. A ranking function may be trained based upon the probability of relevance of each of the plurality of matches. The ranking function may subsequently be applied to identify and rank matches (e.g., in the absence of indicators of relevance).

Claims

1. A method comprising:

receiving, from a mobile device associated with a dating profile of a dating website, a query identifying constraints on attributes of a target dating profile;

retrieving, from the dating website, unlabeled matches based on the query, wherein each of the unlabeled matches comprises a pairing between the dating profile and a candidate dating profile that satisfies the constraints on the attributes, wherein each of the unlabeled matches lacks behavioral features, which indicate whether a match is relevant, wherein the behavioral features comprise;

click feedback from either of the dating profile and the candidate dating profile,

a first duration of profile views initiated by the dating profile toward the candidate dating profile, and

a second duration of profile views initiated by the candidate dating profile toward the dating profile; and

identifying relevant ones of the unlabeled matches utilizing ranking features of each of the unlabeled matches as a proxy for the behavioral features by;

retrieving labeled matches from the dating website, wherein each of the labeled matches comprises a pairing between two dating profiles from the dating website, wherein each of the labeled matches is labeled with a probability of relevance based on the behavioral features of the two dating profiles, wherein;

the probability of relevance identifies a relevant match as ones of the labeled matches in which the two dating profiles have exchanged contact info, and

the probability of relevance identifies a non-relevant match as;

ones of the labeled matches in which a first dating profile of the two dating profiles accessed a second dating profile of the two dating profiles but did not send a message to the second dating profile, or

ones of the labeled matches in which the first dating profile sent a message to the second dating profile but the second dating profile did not send a message to the first dating profile;

generating, from the labeled matches, a ranking model that correlates the probability of relevance to ranking features of each of the labeled matches;

wherein the ranking model is configured to receive, as input, the ranking features of each of the labeled matches and generate, as output, the probability of relevance;

calculating the probability of relevance of each of the unlabeled matches by inputting, into the ranking model, ranking features of each of the unlabeled matches and retrieving, from the ranking model, the probability of relevance of each of the unlabeled matches, wherein the probability of relevance of each of the unlabeled matches is calculated due at least in part to the each of the unlabeled matches lacking the behavioral features;

calculating a rank for each of the unlabeled matches based on the probability of relevance of each of the unlabeled matches to generate a set of ranked matches; and

identifying, to the dating profile, the relevant ones of the unlabeled matches by transmitting at least a portion of the set of ranked matches to the mobile device associated with the dating profile.

2. The method of claim 1, further comprising:

generating the ranking features of each of the labeled matches from the two dating profiles, wherein the ranking features comprise a vector of attributes from each of the two dating profiles.

3. The method of claim 1, further comprising:

generating the ranking features of each of the unlabeled matches from the dating profile and the candidate dating profile, wherein the ranking features comprise a vector of attributes from the dating profile and the candidate dating profile.

4. The method of claim 1, wherein the ranking model comprises a ranking function generated based on one or more regression trees.

5. The method of claim 1, wherein the probability of relevance is one for each relevant match and the probability of relevance is zero for each non-relevant match.

6. A server comprising:

a memory storing data associated with a dating website; and

at least one of processor that performs operations comprising;

receiving, from a mobile device associated with a dating profile of a dating website, a query identifying constraints on attributes of a target dating profile;

retrieving, from the memory, unlabeled matches based on the query, wherein each of the unlabeled matches comprises a pairing between the dating profile and a candidate dating profile that satisfies the constraints on the attributes, wherein each of the unlabeled matches lacks behavioral features, which indicate whether a match is relevant, wherein the behavioral features comprise;

click feedback from either of the dating profile and a candidate dating profile,

a first duration of profile views initiated by the dating profile toward the candidate dating profile, and

a second duration of profile views initiated by the candidate dating profile toward the dating profile; and

identifying relevant ones of the unlabeled matches utilizing ranking features of each of the unlabeled matches as a proxy for the behavioral features by;

retrieving labeled matches from the memory, wherein each of the labeled matches comprises a pairing between two dating profiles from the dating website, wherein each of the labeled matches is labeled with a probability of relevance based on the behavioral features of the two dating profiles, wherein;

the probability of relevance identifies a relevant match as ones of the labeled matches in which the two dating profiles have exchanged contact info, and

the probability of relevance identifies a non-relevant match as;

 

ones of the labeled matches in which a first dating profile of the two dating profiles accessed a second dating profile of the two dating profiles but did not send a message to the second dating profile, or

 

ones of the labeled matches in which the first dating profile sent a message to the second dating profile but the second dating profile did not send a message to the first dating profile;

generating, from the labeled matches, a ranking model that correlates the probability of relevance to ranking features of each of the labeled matches;

wherein the ranking model is configured to receive, as input, the ranking features of each of the labeled matches and generate, as output, the probability of relevance;

calculating the probability of relevance of each of the unlabeled matches by inputting, into the ranking model, ranking features of each of the unlabeled matches and retrieving, from the ranking model, the probability of relevance of each of the unlabeled matches, wherein the probability of relevance of each of the unlabeled matches is calculated due at least in part to the each of the unlabeled matches lacking the behavioral features;

calculating a rank for each of the unlabeled matches based on the probability of relevance of each of the unlabeled matches to generate a set of ranked matches; and

identifying, to the dating profile, the relevant ones of the unlabeled matches by transmitting at least a portion of the set of ranked matches to the mobile device associated with the dating profile.

7. The server of claim 6, wherein the operations further comprise:

generating the ranking features of each of the labeled matches from the two dating profiles, wherein the ranking features comprise a vector of attributes from each of the two dating profiles.

8. The server of claim 6, wherein the operations further comprise

generating the ranking features of each of the unlabeled matches from the dating profile and the candidate dating profile, wherein the ranking features comprise a vector of attributes from the dating profile and the candidate dating profile.

9. The server of claim 6, wherein the ranking model comprises a ranking function generated based on one or more regression trees.

10. The server of claim 6, wherein the probability of relevance is one for each relevant match and the probability of relevance is zero for each non-relevant match.

11. A non-transitory computer-readable medium comprising instructions, that when executed by at least one processor, configure the at least one processor to perform operations comprising:

receiving, from a mobile device associated with a dating profile of a dating website, a query identifying constraints on attributes of a target dating profile;

retrieving, from the dating website, unlabeled matches based on the query, wherein each of the unlabeled matches comprises a pairing between the dating profile and a candidate dating profile that satisfies the constraints on the attributes, wherein each of the unlabeled matches lacks behavioral features, which indicate whether a match is relevant, wherein the behavioral features comprise;

click feedback from either of the dating profile and a candidate dating profile,

a first duration of profile views initiated by the dating profile toward the candidate dating profile, and

a second duration of profile views initiated by the candidate dating profile toward the dating profile; and

identifying relevant ones of the unlabeled matches utilizing ranking features of each of the unlabeled matches as a proxy for the behavioral features by;

retrieving labeled matches from the dating website, wherein each of the labeled matches comprises a pairing between two dating profiles from the dating website, wherein each of the labeled matches is labeled with a probability of relevance based on the behavioral features of the two dating profiles, wherein;

the probability of relevance identifies a relevant match as ones of the labeled matches in which the two dating profiles have exchanged contact info, and

the probability of relevance identifies a non-relevant match as;

ones of the labeled matches in which a first dating profile of the two dating profiles accessed a second dating profile of the two dating profiles but did not send a message to the second dating profile, or

ones of the labeled matches in which the first dating profile sent a message to the second dating profile but the second dating profile did not send a message to the first dating profile;

generating, from the labeled matches, a ranking model that correlates the probability of relevance to ranking features of each of the labeled matches;

wherein the ranking model is configured to receive, as input, the ranking features of each of the labeled matches and generate, as output, the probability of relevance;

calculating the probability of relevance of each of the unlabeled matches by inputting, into the ranking model, ranking features of each of the unlabeled matches and retrieving, from the ranking model, the probability of relevance of each of the unlabeled matches, wherein the probability of relevance of each of the unlabeled matches is calculated due at least in part to the each of the unlabeled matches lacking the behavioral features;

calculating a rank for each of the unlabeled matches based on the probability of relevance of each of the unlabeled matches to generate a set of ranked matches; and

identifying, to the dating profile, the relevant ones of the unlabeled matches by transmitting at least a portion of the set of ranked matches to the mobile device associated with the dating profile.

12. The non-transitory computer-readable medium of claim 11, wherein the operations further comprise:

generating the ranking features of each of the labeled matches from the two dating profiles, wherein the ranking features comprise a vector of attributes from each of the two dating profiles.

13. The non-transitory computer-readable medium of claim 11, wherein the operations further comprise:

generating the ranking features of each of the unlabeled matches from the dating profile and the candidate dating profile, wherein the ranking features comprise a vector of attributes from the dating profile and the candidate dating profile.

14. The non-transitory computer-readable medium of claim 11, wherein the ranking model comprises a ranking function generated based on one or more regression trees.