Learning based on feedback for contextual personalized information retrieval

• US 8,001,064 B1
• Filed: 10/07/2010
• Issued: 08/16/2011
• Est. Priority Date: 06/01/2006
• Status: Active Grant

• Current Assignee: Monster Worldwide Incorporated (Randstad NV)
• Original Assignee: Monster Worldwide Incorporated (Randstad NV)
• Inventors: Rennison, Earl
• Primary Examiner(s): Holmes; Michael B
• Assistant Examiner(s): Kim; David H
• Application Number:
  US12/900,413
• Time in Patent Office: 313 Days
• Field of Search: 707/732, 707/734, 707/722, 707/733, 707/794, 707/600, 707/601, 707/602, 707/603, 706/14
• US Class Current: 706/14
• CPC Class Codes: G06F 16/319 Inverted lists
• CPC Class Codes: G06F 16/3332 Query translation
• CPC Class Codes: G06F 16/3334 Selection or weighting of terms from queries, including natural language queries
• CPC Class Codes: G06F 16/3335 Syntactic pre-processing, e.g. stopword elimination, stemming
• CPC Class Codes: G06F 16/335 Filtering based on additional data, e.g. user or group profiles filtering in web context G06F16/9535, G06F16/9536
• CPC Class Codes: G06F 16/906 Clustering; Classification
• CPC Class Codes: G06N 20/00 Machine learning

Abstract

Information retrieval systems face challenging problems with delivering highly relevant and highly inclusive search results in response to a user'"'"'s query. Contextual personalized information retrieval uses a set of integrated methodologies that can combine automatic concept extraction/matching from text, a powerful fuzzy search engine, and a collaborative user preference learning engine to provide accurate and personalized search results. The system can include constructing a search query to execute a search of a database, parsing an input query from a user into sub-strings, and matching the sub-strings to concepts in a semantic concept network of a knowledge base. The system can further map the matched concepts to criteria and criteria values that specify a set of constraints on and scoring parameters for the matched concepts. Furthermore, the system can learn user preferences to construct one or more profiles, including combined internal and profile weights, for producing personalized search results.

Claims

1. A method for learning user preferences in a search of knowledge base to construct one or more profiles for producing personalized search results, the method comprising:

using a computer system to execute method steps comprising;

receiving feedback from the user regarding quality of search results presented to the user in a search of a knowledge base that is a semantic network of relationships among concepts and that provides an index of a plurality of documents, the feedback representing how well the search results match an input query provided by the user, the search results including one or more of the documents indexed by the knowledge base;

constructing the one or more profiles for the user based on the feedback received, where for each of the search results that receive feedback, a plurality of feedback values are determined and are used to construct a model that includes profile weights computed from the feedback values;

modifying internal weights used for scoring search criteria applied in producing the search results presented to the user, the modifications made by combining the internal weights with the profile weights in the constructed model, wherein the internal weights are modified according to a function of the internal weights used for scoring search criteria and of the profile weights;

generating implicit search criteria for the user based on the one or more profiles; and

applying the implicit search criteria and modified weights during a subsequent search of the knowledge base conducted by the user producing a subsequent set of search results that are personalized to the user.

2. The method of claim 1, wherein the input query is selected from a group consisting of:

a query for job descriptions and a query for resumes for job candidates.

3. The method of claim 1, wherein the user provides ratings for the search results presented using a system selected from the group consisting of:

a point system;

a five star rating system, a slider icon, and any combination of these.

4. The method of claim 1, wherein the feedback is implicit feedback.

5. The method of claim 1, wherein the input query is selected from a group consisting of:

an input string of text and a document.

6. The method of claim 1, further comprising:

learning user profiles and search profiles;

applying these to the search to modify ranking of the search results.

7. The method of claim 1, wherein at least one of the one or more profiles is a user profile that pertains to the user'"'"'s general preferences that are not specifically associated with a search, and wherein at least one of the one or more profiles is a search profile that pertains to a particular query or query type not specifically associated with the user.

8. The method of claim 7, further comprising combining the user profile and the search profile to create a biased profile, wherein weights in the search profile are biased by weights in the user profile using a biasing function.

9. The method of claim 8, wherein the biasing function is selected from a group consisting of:

a mean, a geometric mean, a generalized mean, a trimmed mean, a winsorized mean, and a median.

10. The method of claim 1, wherein the one or more profiles are arranged hierarchically to include parent profiles and child profiles, and the child profiles inherit profile weights from the parent profiles where profile values are not defined in the child profile.

11. The method of claim 1, wherein constructing the one or more profiles further comprises determining a feedback value tuple for each search result, the feedback value tuple including an implicit feedback value, an explicit feedback value, and a negative feedback value, each having a value between −

1.0 and 1.0.

12. The method of claim 11, wherein, for concepts associated with the search criteria applied to retrieve each search result, feedback is received to create a source concept that is compared against a reference concept to form a set of source concept values and a set of reference concept values.

13. The method of claim 1, wherein the profile weights are computed by a weight function selected from a group consisting of:

geometric mean;

weighted geometric mean, weighted average, and average.

14. The method of claim 1, wherein the function is selected from a group consisting of:

log frequency, log inverse frequency, linear frequency, fuzzy frequency, fuzzy inverse frequency, and any combination of these.

15. The method of claim 1, further comprising removing oldest feedback information when a forgetting rate threshold is reached.

16. The method of claim 1, further comprising defining simple profiled score criteria that are based on a single attribute or attribute path, and defining compound profiled score criteria that are based on a set of profiled score criteria, wherein the simple profiled score criteria are instantiated as simple profiled score criteria values and the compound profiled score criteria are instantiated as compound profiled score criteria values, the simple profiled score criteria values and the compound profiled score criteria values each being composed of sub-profiled score criteria.

17. The method of claim 16, wherein the simple profiled score criteria values and the compound profiled score criteria values can be defined by a tuple of values selected from a group consisting of:

a weight range, a weight, a requirement that can be required or desired, and any combination of these.

18. The method of claim 17, wherein the simple profiled score criteria values can use an attribute path from a target concept to select a set of values to evaluate, which is an evaluation set, and wherein the weight of the simple profiled score criteria value is a function of a total number of values in the evaluation set and a number of values in the evaluation set for which profile weights are described.

19. The method of claim 17, wherein the weight for the simple profiled score criteria value is computed using weight functions selected from a group consisting of:

log frequency and log inverse frequency.

20. The method of claim 17, wherein the weight of the compound profiled score criteria values is computed using weight functions selected from a group consisting of:

a geometric mean and a mean of the weights of the sub-profiled score criteria.

21. The method of claim 16, wherein a partial score of a compound profiled score criteria value is computed by computing a score for each of the sub-profiled score criteria, and aggregating results using a score integration function.

22. The method of claim 1, further comprising:

analyzing concepts in each of the search results;

selecting concepts for which explicit user feedback is to be provided;

receiving the explicit user feedback regarding the selected concepts; and

applying the explicit user feedback to an explicit feedback value in the one or more profiles.

23. A system for learning user preferences in a search of knowledge base to construct one or more profiles for producing personalized search results, the system comprising:

at least one computer system;

a computer-readable storage medium storing software components for execution by the at least one computer system, the components comprising;

an adaptive and collaborative user profiling engine for;

receiving feedback from the user regarding quality of search results presented to the user in a search of a knowledge base that is a semantic network of relationships among concepts and that provides an index of a plurality of documents, the feedback representing how well the search results match an input query provided by the user, the search results including one or more of the documents indexed by the knowledge base;

constructing the one or more profiles for the user based on the feedback received, where for each of the search results that receive feedback, a plurality of feedback values are determined and are used to construct a model that includes profile weights computed from the feedback values; and

a personalized search and match engine for;

modifying internal weights used for scoring search criteria applied in producing the search results presented to the user, the modifications made by combining the internal weights with the profile weights in the constructed model, wherein the internal weights are modified according to a function of the internal weights used for scoring search criteria and of the profile weights;

generating implicit search criteria for the user based on the one or more profiles; and

applying the implicit search criteria and modified weights during a subsequent search of the knowledge base conducted by the user producing a subsequent set of search results that are personalized to the user.

24. The system of claim 23, wherein the input query is selected from a group consisting of:

a query for job descriptions and a query for resumes for job candidates.

25. The system of claim 23, wherein the feedback receiving module is adapted for receiving ratings from the user using a system selected from the group consisting of:

a point system;

a five star rating system, a slider icon, and any combination of these.

26. The system of claim 23, wherein the feedback is implicit feedback.

27. The system of claim 23, wherein at least one of the one or more profiles is a user profile that pertains to the user'"'"'s general preferences that are not specifically associated with a search, and wherein at least one of the one or more profiles is a search profile that pertains to a particular query or query type not specifically associated with the user.

28. The system of claim 27, further comprising combining the user profile and the search profile to create a biased profile, wherein weights in the search profile are biased by weights in the user profile using a biasing function selected from a group consisting of:

a mean, a geometric mean, a generalized mean, a trimmed mean, a winsorized mean, and a median.

29. The system of claim 23, wherein the one or more profiles are arranged hierarchically to include parent profiles and child profiles.

30. A computer program product having a non-transitory computer-readable medium having computer program instructions recorded thereon for learning user preferences in a search of knowledge base to construct one or more profiles for producing personalized search results, the computer program instruction comprising instructions for:

receiving feedback from the user regarding quality of search results presented to the user in a search of a knowledge base that is a semantic network of relationships among concepts and that provides an index of a plurality of documents, the feedback representing how well the search results match an input query provided by the user, the search results including one or more of the documents indexed by the knowledge base;

constructing the one or more profiles for the user based on the feedback received, where for each of the search results that receive feedback, a plurality of feedback values are determined and are used to construct a model that includes profile weights computed from the feedback values;

modifying internal weights used for scoring search criteria applied in producing the search results presented to the user, the modifications made by combining the internal weights with the profile weights in the constructed model, wherein the internal weights are modified according to a function of the internal weights used for scoring search criteria and of the profile weights;

generating implicit search criteria for the user based on the one or more profiles; and

applying the implicit search criteria and modified weights during a subsequent search of the knowledge base conducted by the user producing a subsequent set of search results that are personalized to the user.

31. The computer program product of claim 30, wherein the input query is selected from a group consisting of:

a query for job descriptions and a query for resumes for job candidates.

32. The computer program product of claim 30, wherein at least one of the one or more profiles is a user profile that pertains to the user'"'"'s general preferences that are not specifically associated with a search, and wherein at least one of the one or more profiles is a search profile that pertains to a particular query or query type not specifically associated with the user.

33. The computer program product of claim 30, wherein the one or more profiles are arranged hierarchically.

34. The computer program product of claim 30, wherein the function is selected from a group consisting of:

log frequency, log inverse frequency, linear frequency, fuzzy frequency, fuzzy inverse frequency, and any combination of these.