USING LOG DATA TO TRAIN FOR AUTOMATED SOURCING

• US 20180239830A1
• Filed: 12/22/2017
• Published: 08/23/2018
• Est. Priority Date: 02/17/2017
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Dialani, Vijay, Geyik, Sahin Cem, Yan, Yan, Gupta, Abhishek
• Granted Patent: US 10,628,506 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/2178 based on feedback of a supervisor
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06Q 10/1053 Employment or hiring
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: G06V 2201/09 Recognition of logos

Abstract

Techniques for using recruiter review data to create training, validation and test sets for automated sourcing are described. An example system obtains sample suggested candidate member profiles and sample search result member profiles in an online system. The system identifies unique pairs of member profiles, each pair consisting of one of the suggested candidate profiles and one of the search result profiles. Additionally, the system generates a label for each of the unique pairs of profiles. The label is generated using a score generated from log information of the online system, the log information including historical records of searcher feedback regarding members of the online system, the score being higher if the searcher accepted the sample search result member profile in a search session. Furthermore, the system inputs the labels into a machine learning algorithm to train a combined ranking model that outputs ranking scores for search result member profiles.

Claims

1. A computer system, comprising:

one or more processors; and

a non-transitory computer readable storage medium storing instructions that when executed by the one or more processors cause the computer system to perform operations comprising;

obtaining one or more sample suggested candidate member profiles in an online system having a plurality of members;

obtaining one or more sample search result member profiles in the online system;

identifying unique pairs of member profiles, each unique pair consisting of one of the one or more sample suggested candidate member profiles and one of the one or more sample search result member profiles;

generating, for each of the unique pairs of member profiles, a label using a score generated from log information of the online system, the log information including historical records of searcher feedback regarding members of the online system, the score being higher if the searcher expressed acceptance of the sample search result member profile in a search session; and

inputting the generated labels into a machine learning algorithm to train a combined ranking model, wherein the combined ranking model is configured to output ranking scores for search result member profiles.

2. The system of claim 1, the system further comprising a time machine function configured to:

receive;

a member profile for a member of the online system; and

standardized log information in the log information of the online system; and

generate a position timeline for the member based on position-derived data in the standardized log information;

generate an education timeline for the member based on education-derived data in the standardized log information;

produce a member timeline for the member as specified by a given start date and given end date, the member timeline being based on the position timeline and the education timeline; and

provide a snapshot of the member profile as of a certain day in the past, the snapshot being based on the member timeline.

3. The system of claim 2, wherein the generating the label for each of the unique pairs of member profiles, is based at least in part on the member timeline.

4. The system of claim 1, wherein the online system hosts a social networking service including the member profiles,

5. The system of claim 4, wherein the log information further includes records of past user feedback by the searcher, the user feedback including interaction with member profiles in the social networking service.

6. The system of claim 5, wherein:

a high score is used if the searcher expressed approval of the member corresponding to the member corresponding to the sample search result member profile in a search session,

a low score if no interaction occurred in the search session or the searcher expressed rejection of the member corresponding to the member corresponding to the sample search result member profile in a search session; and

a medium score is used if the searcher expressed deferral of the member corresponding to the member corresponding to the sample search result member profile in a search session, the medium score corresponding to a value between the low score and the high score.

7. The system of claim 4, wherein a search session includes a period of time between a searcher initiating a search and the searcher submitting an unrelated search or logging off the social networking service.

8. The system of claim 1, wherein a search session is a candidate browsing session.

9. The system of claim 1, wherein the instruction set executable on the processor further cause the computer system to perform operations comprising:

aggregating label scores from multiple search sessions.

10. The system of claim 1, wherein the instruction set executable on the processor further cause the computer system to perform operations comprising:

ranking one or more search result member profiles using the combined ranking model.

11. A computer implemented method, comprising:

obtaining one or more sample suggested candidate member profiles in an online system having a plurality of members;

obtaining one or more sample search result member profiles in the online system;

identifying unique pairs of member profiles, each unique pair consisting of one of the one or more sample suggested candidate member profiles and one of the one or more sample search result member profiles;

generating, for each of the unique pairs member profiles a label using a score generated from log information of the online system, the log information including historical records of searcher feedback regarding members of the online system, the score being higher if the searcher expressed acceptance of the sample search result member profile in a search session; and

inputting the generated labels into a machine learning algorithm to train a combined ranking model used to output ranking scores for search result member profiles.

12. The method of claim 11, wherein the log information further includes records of past user feedback by the searcher, the user feedback including interaction with member profiles in the online system.

13. The method of claim 11, wherein a search session is a candidate browsing session.

14. The method of claim 11, wherein the online system hosts a social networking service including the member profiles.

15. The method of claim 14, wherein a search session includes a period of time between a searcher initiating a search and the searcher submitting an unrelated search or logging off the social networking service.

16. The method of claim 11, further comprising aggregating label scores from multiple search sessions.

17. The method of claim 11, further comprising ranking one or more search result member profiles using the combined ranking model.

18. A non-transitory machine-readable storage medium comprising instructions, which when implemented by one or more machines, cause the one or more machines to perform operations comprising:

obtaining one or more sample suggested candidate member profiles in a social networking service;

obtaining one or more sample search result member profiles in the social networking service;

identifying unique pairs of member profiles, each unique pair consisting of one of the one or more sample suggested candidate member profiles and one of the one or more sample search result member profiles;

generating, for each of the unique pairs member profiles a label using a score generated from log information of the social networking service, the log information including historical records of searcher feedback regarding members of the social networking service, the score being higher if the searcher expressed acceptance of the sample search result member profile in a search session; and

inputting the generated labels into a machine learning algorithm to train a combined ranking model used to output ranking scores for search result member profiles.

19. The non-transitory machine-readable storage medium of claim 18, wherein the log information further includes records of past user feedback by the searcher, the user feedback including interaction with member profiles in the social networking service.

20. The non-transitory machine-readable storage medium of claim 18, wherein a search session includes a period of time between a searcher initiating a search and the searcher submitting an unrelated search or logging off the social networking service.