Method and system for improvement of relevance of search results

• US 8,200,663 B2
• Filed: 04/25/2008
• Issued: 06/12/2012
• Est. Priority Date: 10/15/2007
• Status: Active Grant

• Current Assignee: ChaCha Search, Inc.
• Original Assignee: ChaCha Search, Inc.
• Inventors: Jones, Scott A., Cooper, Thomas E.
• Primary Examiner(s): LEWIS, CHERYL RENEA
• Application Number:
  US12/109,776
• Publication Number:
  US 20080270389A1
• Time in Patent Office: 1,509 Days
• Field of Search: 707/723, 707/711, 707/733, 707/805, 705/12, 705/319, 705/7.29, 705/14.4, 709/203, 715/758
• US Class Current: 707/723
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/3326 using relevance feedback from the user, e.g. relevance feedback on documents, documents sets, document terms or passages
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06Q 30/0224 based on user history
• CPC Class Codes: G06Q 30/0241 Advertisements
• CPC Class Codes: H04L 67/30 Profiles

Abstract

A system and method for improving the relevance of search results is disclosed. Voters who may be human searchers or guides may review search results or other review items associated with a search request or other reference item. A review may be activated based on a usage indicator(s) which may improve utilization of guides. A vote by a voter may be weighted based on a voting history associated with the voter and one or more reference voters who may be designated by the system. A voter may be presented with a group of items for review including simultaneously. A number of comparison voting sessions or elections may be used to determine a rating or ranking of a review item associated with a reference item.

Claims

1. A method of rating review items associated with a reference item, comprising:

initiating a voting session;

selecting one or more voters;

presenting one or more reference items to a voter;

presenting one or more review items to the voter when determining that an unranked review item is associated with the one or more reference items;

obtaining voter preferences regarding relevance of the review items to the reference items;

tabulating voter preferences;

closing the voting session; and

rating the review items for the reference item based on the voter preferences.

2. The method of claim 1, comprising:

weighting the preferences obtained based on a trust level of a voter from whom the preference is obtained.

3. The method of claim 2, wherein the weighting is based on historical preferences of the voter relative to preferences of one or more reference voters.

4. The method of claim 2, wherein the voting session is closed when a predetermined plurality of votes is obtained.

5. The method of claim 3, wherein the one or more reference voters are designated by a system administrator.

6. The method of claim 3, wherein the historical preferences are associated with one or more items associated with a reference item.

7. The method of claim 5, wherein one or more temporary reference voters are determined based on historical preferences of a voter relative to preferences of the one or more reference voters.

8. The method of claim 1, wherein said one or more voters are searchers associated with the reference item.

9. The method of claim 1, wherein said one or more voters are selected based on ratings associated with the reference item.

10. The method of claim 1, wherein a review item includes one or more of an answer, a web page, an advertisement, a search resource, a category, a keyword, a search request, a query, and a profile.

11. The method of claim 1, wherein a reference item includes one or more of an answer, a web page, an advertisement, a search resource, a category, a keyword, a search request, a query, and a profile.

12. The method of claim 1, wherein a searcher associated with an index item linked to the reference item recommends a review item by performing an information search responsive to the reference item.

13. The method of claim 1, wherein said one or more voters are compensated for registering a preference.

14. The method of claim 13, wherein compensation is weighted.

15. A method of rating review items associated with a reference item based upon voting, comprising:

opening a voting session;

presenting a simultaneous comparison of two or more review items to a voter when determining that an unranked review item is associated with a reference item;

presenting the reference item to the voter;

obtaining a relevancy preference indication from the voter;

closing the voting session; and

rating relevance of the review items to the reference item based on an outcome of one or more voting sessions.

16. The method of claim 15, wherein the voter is selected based on a ranking of the voter based on one or more index items associated with the reference item and said review items.

17. The method of claim 15, wherein an unrated review item is compared to a review item which is rated with respect to the reference item.

18. The method of claim 15, wherein the review item is one or more of an answer, a web page, an advertisement, a search resource, a category, and a profile.

19. The method of claim 15, wherein the reference item is one or more of a search query, a web page, an advertisement, a search resource, a category, a location, and a profile.

20. The method of claim 15, wherein the voters are searchers associated with an index item linked to the reference item.

21. The method of claim 15, wherein the voting session compares an unranked review item to a predetermined number of rated review items selected based on a rating assigned to the rated review items.

22. The method of claim 15, wherein an unranked review item is compared to a next higher ranked review item until the unranked review item is determined to be inferior to the next higher ranked review item or to be a highest ranked review item.

23. The method of claim 15, comprising;

receiving a search request matching the reference item; and

presenting a review item based at least in part on the rating of the review item associated with the reference item.

24. A system, comprising;

a voter system providing an opinion regarding a review item associated with a reference item;

a server selecting a reference item, the review item, and the voter system based on a searcher system and tracking voting activities;

the searcher system performing a search for information responsive to the reference item to associate the review item with the reference item; and

a database recording opinion and rating information regarding the review item.

25. An interface component of a computing device comprising:

a plurality of indicators of review items including a review item ranked based on an index item and a reference item and a review item determined to be unranked for the index item and the reference item;

an indicator of the reference item; and

a plurality of opinion controls associated with the plurality of indicators of the review items causing the ranking of the plurality of review items to be modified when activated.

26. An interface component of a computing device comprising:

a plurality of reference item selection controls indicating reference items determined to be associated with an unranked review item;

an election status indicator providing status regarding the reference items;

a plurality of review item opinion controls indicating the unranked review item and a ranked review item of the reference item, receiving input from a voter regarding the review items and modifying a ranking of review items associated with a reference item when activated; and

a voter information selector indicating a status of a voter.

27. A method of rating review items associated with a reference item comprising:

associating a review item with a reference item;

presenting the review item responsive to a request associated with the reference item;

determining whether the review item has been presented more than a pre-determined number of times;

presenting the review item and the reference item to a reviewer when said determining indicates that the pre-determined number has been exceeded;

determining whether the review item is accepted by the reviewer based on the reference item;

associating an additional review item with the reference item when said determining indicates that the review item is not accepted;

presenting the review items and the reference item to a second reviewer, the second reviewer selecting a preferred review item; and

providing the preferred review item responsive to a subsequent request associated with the reference item.

28. The method of claim 27, wherein the reference item is a query and the review item is an answer produced responsive to the query.

29. The method of claim 28, wherein the reviewer is a searcher associated with one or more items associated with the reference item.

30. The method of claim 28, wherein items associated with the query are one or more of a category, a keyword, a profile, and a search query.

31. A method of rating review items associated with a reference item, comprising:

associating a number of review items with a topic reference item;

determining whether the number of review items associated with the topic reference item is greater than a pre-determined value;

presenting pairs of the review items and the topic reference item to one or more reviewers;

obtaining information from the one or more reviewers of a preferred review item in the pairs; and

determining a ranking of the review items for the topic reference item based on the preferred review item.

32. The method of claim 31, wherein the topic reference item is a query and the review items are answers produced by a search by a human searcher responsive to the query.

33. The method of claim 32, wherein the reviewer is a second searcher associated with one or more items associated with the query.

34. The method of claim 33, wherein the one or more items associated with the query are one or more of a category, a keyword, a profile, and a search query.

35. A user interface component of a computing device, comprising:

an indicator of a reference item;

an indicator of one or more review items including a review item unranked for the reference item or a review item ranked for the reference item when determining that the unranked review item is associated with the reference item; and

voting controls for indicating an opinion regarding pertinence of the review items to the reference item, the voting controls causing a ranking of the review items associated with the reference item to be adjusted when activated.

36. The user interface component of claim 35, wherein the indicators comprise one or more of a URL, text, an image, audio, and video.

37. The user interface component of claim 35, comprising:

an indicator of at least one other opinion regarding the one or more review items.

38. The user interface component of claim 35, comprising:

an interface for placing a wager on the pertinence of the one or more review items;

a chat interface;

an indicator of one or more voters; and

an indicator of a score associated with the wager.

39. A method, comprising:

ranking, by a processor, relevance of a review item when determining that the review item is an unranked review item associated with a first search query,

receiving, by the processor, a vote of at least one human searcher;

presenting, to the at least one human searcher, the first search query, a review item ranked for the first search query and the review item; and

providing the review items responsive to a second search query in an order based on the ranking when determining that the second search query matches the first search query.

40. The method of claim 39, wherein said review items comprise one or more of a search result, a search resource, and an advertisement.

41. The method of claim 39, wherein of said review items, review items ranked high are determined to be most relevant for said second search query.

42. A method of improving relevance of search results, comprising:

determining whether an unranked review item has been associated with a reference item;

receiving, by a processor, a vote of a human searcher, regarding relevance of a ranked review item and the unranked review item to a reference item presented to the human searcher;

selecting, by the processor, the human searcher based on a ranking of the human searcher associated with the reference item;

modifying a ranking of the ranked review item for the reference item based on the vote; and

presenting the review items responsive to a search request matching the reference item in an order based on the ranking.

43. A computer-readable storage medium having a program recorded therein for causing a computer to execute operations improving relevance of search results, comprising:

determining whether an unranked review item has been associated with a reference item;

receiving a vote of a human searcher regarding relevance of a ranked review item and the unranked review item to a reference item presented to the human searcher;

selecting the human searcher based on a ranking of the human searcher associated with the reference item;

modifying a ranking of the ranked review item for the reference item based on the vote; and

presenting the review items responsive to a search request matching the reference item in an order based on the ranking.

44. The computer-readable storage medium of claim 43, wherein the operations executed comprise:

selecting one or more of a search result, a search resource, and an advertisement as the reference item, the ranked review item, or the unranked review item.

45. The computer-readable storage medium of claim 43, wherein the operations executed comprise:

weighting the vote based on the ranking of the searcher.

46. The computer-readable storage medium of claim 43, wherein the operations executed comprise:

receiving a plurality of votes cast by human searchers, and notifying the human searchers to vote in an order based on availability of the human searchers.