Systems and methods for training and employing a machine learning system in evaluating entity pairs

• US 9,679,259 B1
• Filed: 01/24/2014
• Issued: 06/13/2017
• Est. Priority Date: 01/25/2013
• Status: Active Grant

• Current Assignee: Plentyoffish Media ULC (Match Group, Inc.)
• Original Assignee: Plentyoffish Media ULC (Match Group, Inc.)
• Inventors: Frind, Markus, Li, Sa, Oldridge, Steve
• Primary Examiner(s): Hill, Stanley K
• Assistant Examiner(s): Traktovenko, Ilya
• Application Number:
  US14/163,849
• Time in Patent Office: 1,236 Days
• Field of Search: None
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition

Abstract

A matching or pairing system and method for matching first and second entities having a greater likelihood of forming a successful pairing includes a trained machine learning system to provide heuristic values useful in determining a compatibility score for the pairing. During training of the machine learning system, a training example selection device can provide attribute values logically associated with entities engaged in historically successful pairings and a number of hypothetically successful pairings. The hypothetically successful pairings may be based at least in part on historically successful pairings where at least one attribute value logically associated with at least one entity in the pairing is varied, adjusted, or subjected to a loosened constraint. During run-time operation a screening device can screen unsuccessful pairings and forward potentially successful pairings that meet a threshold value to the neural network. The system can then determine a compatibility score for the pairing.

Claims

1. A training system to train at least a first machine learning system of a candidate selection system, the training system comprising:

at least one non-transitory processor-readable medium that stores processor-executable instructions; and

at least one processor communicably coupled to the at least one non-transitory processor-readable medium and which executes the processor-executable instructions and in response;

identifies from a plurality of entities a first entity and a second entity that are a historically successful pairing based at least in part on an absence of both the first and the second entities from candidate selection system for a defined period of time, each of the plurality of entities associated with a plurality of attributes;

generates a plurality of hypothetical entities, each of the hypothetical entities based on the second entity in the historically successful pairing, and each of the hypothetical entities having a plurality of attributes which are based on a corresponding plurality of attributes for the second entity upon which the hypothetical entity is based, at least one attribute of the plurality of attributes of the hypothetical entity modified to be different from the corresponding attribute of the first entity in the successful pairing;

generates a plurality of hypothetical alternative pairings, each of the plurality of hypothetical alternative pairings comprising;

an alternative pairing between the first entity and one of the plurality of hypothetical entities based on the second entity;

provides the first machine learning system of the candidate selection system with the historically successful pairing as one training example; and

provides the first machine learning system of the candidate selection system with at least one of the hypothetical alternative pairings as an additional training example.

2. The training system of claim 1 wherein at least one attribute of the plurality of attributes has a respective range of values including at least three sequentially ordered possible values.

3. The training system of claim 2 wherein for at least one of the attributes, the at least one processor determines in which one of at least two discrete sub-ranges the value of the attribute resides, assigns a respective sub-range value, and employs the respective sub-range value to generate the plurality of hypothetical entities based at least in part on respective values of the plurality of attributes.

4. The training system of claim 1 wherein the at least one processor further:

for at least one of the entities,

receives a plurality of inputs; and

deduces the respective value for at least one of the attributes based at least in part on the received plurality of inputs.

5. The training system of claim 1 wherein the at least one processor further:

prunes the alternative pairing as an outlier instance before providing the first machine learning system of the candidate selection system with the alternative pairing as an additional training example.

6. A method of training at least a first machine learning system of a candidate selection system, the method comprising:

identifying by at least one processor a first entity and a second entity, out of a plurality of entities, that are a historically successful pairing;

generating by the at least one processor a plurality of hypothetical entities, each of the hypothetical entities based on the second entity in the historically successful pairing, and each of the hypothetical entities having a plurality of attributes which are based on a corresponding plurality of attributes for the second entity upon which the hypothetical entity is based, at least one attribute of the plurality of attributes of the hypothetical entity modified to be different from the corresponding attribute of the second entity in the historically successful pairing;

generating by the at least one processor a plurality of hypothetical alternative pairings, each of the plurality of hypothetical alternative pairings comprising;

an alternative pairing between the first entity and one of the plurality of hypothetical entities based on the second entity;

providing the first machine learning system of the candidate selection system with the historically successful match as one training example; and

providing the first machine learning system of the candidate selection system with at least one of the hypothetical alternative pairings as an additional training example.

7. The method of claim 6 wherein at least a first one of the attributes has a respective first range of values including at least three possible values.

8. The method of claim 6, wherein at least a first one of the attributes has a respective range of values including at least three sequentially ordered possible values.

9. The method of claim 8, further comprising:

for at least one of the attributes, identifying a respective sub-range value for the value of the attribute, and employing the sub-range value in generating the plurality of hypothetical entities based at least in part on respective values of the plurality of attributes.

10. The method of claim 6, further comprising:

for at least one of the first entity and the second entity, receiving an input that specifies the value for at least one of the attributes of the respective entity.

11. The method of claim 6, further comprising:

for at least one of the first entity and the second entity,

receiving a plurality of inputs; and

deducing the value for at least one of the attributes based at least in part on the received plurality of inputs.

12. The method of claim 6, further comprising:

pruning at the hypothetical alternative pairing as an outlier instance before providing the first machine learning system of the candidate selection system with the alternative pairing as an additional training example.

13. The method of claim 6, wherein identifying the historically successful pairing is based at least in part on an absence of both the first and the second entities from candidate selection system for a defined period of time.

14. A candidate selection system for finding matches between entities, comprising:

at least one machine learning system which includes;

at least one non-transitory processor-readable medium that stores processor-executable instructions; and

at least one processor communicably coupled to the at least one non-transitory processor-readable medium and which executes the processor-executable instructions to;

implement the at least one machine learning system including an input layer, an output layer and a hidden layer, the at least one machine learning system trained with at least a first entity and a second entity that are a historically successful pairing, and trained with a plurality of hypothetical alternative pairings between the first entity of the historically successful pairing and other entities of a first set of entities, the hypothetical alternative pairings based on at least one value of at least one attribute of a plurality of attributes associated with the second entity and at least one loosened constraint of a number of constraints on matching the at least one value of the at least one attribute associated with the second entity; and

at least one processor-based system, the at least one processor-based system communicatively coupled to the machine learning system, the at least one processor-based system includes;

at least one non-transitory processor-readable medium that stores processor-executable instructions; and

at least one processor communicably coupled to the at least one non-transitory processor-readable medium and which executes the processor-executable instructions to;

receive a number of heuristic values indicative of a strength of a pairing between the first entity and at least one of the entities of the first set of entities; and

execute a candidate selection algorithm which employs the received heuristic values and respective values for each of the plurality of attributes to identify prospective candidates within a second set of entities.

15. The candidate selection system of claim 14 wherein the at least one processor-based system determines a product of a plurality of respective scores for each of at least two of the attributes of the plurality of attributes.

16. The candidate selection system of claim 15 wherein at least a first one of the attributes has a respective first range of values including at least three sequentially ordered possible values, and for at least the first one of the attributes, the at least one processor determines in which one of at least two discrete sub-ranges the value of the attribute resides, each of the sub-ranges logically associated with a respective score.

17. The candidate selection system of claim 16 wherein at least one processor-based system weights the scores for at least some of the attributes relative to other of the attributes.

18. The candidate selection system of claim 16 wherein the received heuristic values are normalized values.

19. The candidate selection system of claim 16 wherein the attributes include at least one of:

an entity'"'"'s specified intent, an entity'"'"'s specified age, an entity'"'"'s specified height, an entity'"'"'s specified income, an entity'"'"'s specified educational attainment level, or an entity'"'"'s specified body type, represented by a respective data object logically associated with a respective one of the entities.

20. The candidate selection system of claim 14 wherein the hypothetical alternative pairings result from an exact match between respective values for less than all of the attributes for the respective entities.

21. The candidate selection system of claim 14 wherein at least some of the prospective candidates are presented to a user in a ranked order.

22. The candidate selection system of claim 14 wherein a plurality of messages sent to the first entity by other ones of the entities are presented to the first entity in a ranked order, the ranked order reflecting a strength of a pairing between the first entity and the other ones of the entities.

23. The candidate selection system of claim 14, further comprising:

at least a first processor-based server system, communicatively coupled to receive information about the first entity and the second entity, to receive requests for prospective candidates, to provide prospective candidate information to at least some of the entities, and to present a plurality of messages sent to the first entity by other ones of the entities in a ranked order, the ranked order reflecting a strength of a pairing between at least the first entity and the other ones of the entities.

24. The candidate selection system of claim 14, further comprising:

at least a first processor-based server system, communicatively coupled to receive information about the first entity and the second entity, to receive requests for potential candidates, and to provide prospective candidate identifying information to at least one of the first entity and the second entity.

25. The candidate selection system of claim 14 wherein the second set of entities is different from the first set of entities.

26. A method of using a candidate selection system that includes at least a first trained machine learning system, the method comprising:

receiving, by at least one processor-based system, a number of heuristic values each indicative of a strength of a respective pairing between two entities, the heuristic values generated by at least one machine learning system trained with at least a first entity and a second entity that are a historically successful pairing, and trained with a plurality of hypothetical alternative pairings between the first entity of the historically successful pairing and other entities of a first set of entities based at least in part on at least one value of each of at least one of a plurality of attributes associated with the second entity and based on at least one loosened constraint of a number of constraints applied to matches between the values of at least one of the attributes associated with the second entity; and

executing, by the at least one processor-based system, a candidate selection algorithm which employs the received heuristic values and respective values for each of the plurality of attributes for each of a second set of entities to identify prospective candidates.

27. The method of claim 26 wherein executing a candidate selection algorithm includes determining a product of a plurality of respective scores for each of at least two of the attributes and the heuristic values.

28. The method of claim 27 wherein at least a first one of the attributes has a respective first range of values including at least three sequentially ordered possible values, and further comprising:

for at least the first one of the attributes, determining in which one of at least two discrete sub-ranges the value of the attribute resides, by the at least one processor-based system, each of the sub-ranges logically associated with a respective score.

29. The method of claim 28 wherein executing a candidate selection algorithm includes weighting the scores for at least some of the attributes relative to other of the attributes, by the at least one processor-based system.

30. The method of claim 29, further comprising:

retrieving respective values of at least one of the plurality of attributes from a data object, including at least one of;

an entity'"'"'s specified intent, an entity'"'"'s specified age, an entity'"'"'s specified height, an entity'"'"'s specified income, an entity'"'"'s specified educational attainment level, or an entity'"'"'s specified body type.

31. The method of claim 26 wherein receiving the heuristic values includes receiving normalized heuristic values.

32. The method of claim 26 wherein the hypothetical alternative pairings are indicative of an exact match between respective values of the first entity of the historically successful pairing and the entities of the first set of entities for less than all of the attributes of the plurality of attributes.

33. The method of claim 26, further comprising:

presenting at least some of the prospective candidates to a user in a ranked order.

34. The method of claim 26, further comprising:

presenting to the first entity at least some of a plurality of messages sent to the first entity by other ones of the entities in a ranked order, the ranked order reflecting a strength of a pairing between the first entity and the other ones of the entities.

35. The method of claim 26, further comprising:

receiving by at least a first processor-based server system information about the plurality of attributes for each of the entities of the second set of entities; and

presenting to the first entity a plurality of messages sent to the first entity by other ones of the entities in a ranked order, the ranked order reflecting a strength of a pairing between at least the first entity and the other ones of the entities.

36. The method of claim 35 wherein presenting to the first entity the plurality of messages sent to the first entity by other ones of the entities in the ranked order includes setting an importance flag of a message.

37. The method of claim 26 wherein the second set of entities is different from the first set of entities.