Method and system for improvement of relevance of search results

• US 8,700,615 B2
• Filed: 05/08/2012
• Issued: 04/15/2014
• Est. Priority Date: 10/15/2007
• Status: Expired due to Fees

• Current Assignee: ChaCha Search, Inc.
• Original Assignee: ChaCha Search, Inc.
• Inventors: Cooper, Thomas E, Jones, Scott A
• Primary Examiner(s): LEWIS, CHERYL RENEA
• Application Number:
  US13/466,185
• Publication Number:
  US 20120221564A1
• Time in Patent Office: 707 Days
• Field of Search: 707/723, 707/711, 707/805, 707/733, 705/319, 705/7.29, 705/14.4, 705/12, 715/758, 709/203
• US Class Current: 707/723
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/3326 using relevance feedback from the user, e.g. relevance feedback on documents, documents sets, document terms or passages
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06Q 30/0224 based on user history
• CPC Class Codes: G06Q 30/0241 Advertisements
• CPC Class Codes: H04L 67/30 Profiles

Abstract

A system and method for improving the relevance of search results is disclosed. Voters who may be human searchers or guides may review search results or other review items associated with a search request or other reference item. A review may be activated based on a usage indicator(s) which may improve utilization of guides. A vote by a voter may be weighted based on a voting history associated with the voter and one or more reference voters who may be designated by the system. A voter may be presented with a group of items for review including simultaneously. A number of comparison voting sessions or elections may be used to determine a rating or ranking of a review item associated with a reference item.

Claims

1. A method of determining relevance comprising:

obtaining a reference;

selecting, by a processor, an item based on the reference;

presenting an indicator of the reference and an identifier of the item to a reviewer;

receiving an opinion of the reviewer regarding the reference and the item;

comparing the opinion to a judgment of a designated representative regarding the reference and the item;

assigning a weight to opinions of the reviewer based on the comparing;

ranking items based on the weight and the opinions.

2. The method of claim 1 comprising:

designating the representative based on alignment between the judgment and opinions received from reviewers presented with the indicator and the reference; and

assigning the weight based on a number of the opinions in agreement with the opinion.

3. The method of claim 1 comprising:

presenting a question as the indicator and a phrase and a Uniform Resource Locator as the identifier.

4. The method of claim 1 comprising:

computing the weight based on opinions of designated representatives associated with a category determined by the reference.

5. The method of claim 1 comprising:

receiving a selection from a group of categories; and

ranking the relevance of the items based on the weight and the opinions when the judgment of the designated representative is received for the item.

6. The method of claim 1 comprising:

receiving a selection from a group of categories; and

ranking the relevance of the items based on the weight and the opinions when a guide has provided an evaluation of the item and the evaluation of the guide has been affirmed by a comparison of evaluations by the guide to judgments of the designated representative.

7. The method of claim 1 comprising:

linking the reviewer to a keyword;

selecting the item based on the keyword;

choosing the designated representative based on a ranking of designated representatives for the keyword.

8. The method of claim 7 comprising:

administering a test to a guide;

comparing judgments of the designated representatives to choices of the guide; and

ranking the designated representatives based on the comparing.

9. The method of claim 1 comprising:

associating a location with the designated representative; and

associating the weight with the location.

10. The method of claim 1 comprising:

obtaining judgments of the designated representative regarding groups;

including in the groups a query and a response;

receiving assessments of the groups including the reference and the item by users; and

performing the comparing based on a correlation between the judgments, the assessments and the opinion.

11. The method of claim 1 comprising:

presenting the indicator and the identifier to the reviewer based on a ranking of the reviewer for the reference; and

assigning the weight to the opinions of the reviewer associated with the reference.

12. The method of claim 1 comprising:

selecting the item based on a ranking of the item for the reference; and

presenting the identifier of the item in an order based on the ranking.

13. The method of claim 1 comprising:

determining a number of references associated with the reviewer;

ranking the references based on a number of judgments provided for the reference; and

presenting the references to the reviewer in an order based on the ranking.

14. A system comprising:

a reviewer system presenting an indicator of a reference and an identifier of an item and receiving an opinion;

a server system, comprising memory and a processor, selecting the item based on the reference, obtaining the reference, assigning a weight to opinions received from the reviewer system based on comparing the opinion to a judgment of a designated representative, and ranking items for a selection based on the weight and the opinions; and

a representative system providing the judgment of the designated representative regarding the reference and the item.

15. A non-transitory computer readable medium storing therein a program for causing a computer to execute an operation including rating a result, comprising:

obtaining a reference;

presenting an indicator of the reference and an identifier of an item selected based on the reference to a user;

receiving an opinion of the user;

presenting the indicator and the identifier to a human assistant;

designating the human assistant as a privileged guide;

presenting the indicator to the user based on a number of choices received regarding the indicator from privileged guides;

receiving a choice of the human assistant;

determining a weight associated with the opinion of the user based on the choice of the human assistant.

16. The computer readable medium of claim 15, the operation further comprising:

determining the weight based on a ranking of the human assistant for the reference.

17. The computer readable medium of claim 15, the operation further comprising:

determining the weight based on a number of times choices of human assistants agree with the opinion when the human assistants are presented with the indicator and the identifier.

18. The computer readable medium of claim 15, the operation further comprising:

obtaining the reference based on a number of human assistants who have provided choices associated with a category associated with the reference.

19. The computer readable medium of claim 15, the operation further comprising:

obtaining the reference based on a number of items associated with the reference.

20. The computer readable medium of claim 15, the operation further comprising:

determining the weight based on a comparison of choices of the human assistant to evaluations provided by a human guide ranked highest for the reference.