Method for human ranking of search results

• US 8,924,376 B1
• Filed: 06/04/2012
• Issued: 12/30/2014
• Est. Priority Date: 01/31/2010
• Status: Active Grant

• Current Assignee: Advanced Research LLC
• Original Assignee: Bryant Christopher Lee
• Inventors: Lee, Bryant Christopher
• Primary Examiner(s): Lewis, Alicia
• Application Number:
  US13/487,868
• Time in Patent Office: 939 Days
• Field of Search: 707/706, 707/722, 707/723, 707/758, 707/769, 707/999.003, 707/748, 707/749, 707/751
• US Class Current: 707/723
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

A search engine may perform a search for a user search query. A rankings of result items may be returned as a result of the search. The user may enter new suggested rankings for some of the items. The new suggested rankings may change the rankings of result items when other users search for the same search query or a similar search query at a later time.

Claims

1. A method for ordering search engine results, the method performed by a computer system having access to a database, the method comprising:

receiving at the computer system a search query from an end user;

performing a search over a set of data in response to the search query, the set of data being at the computer system or accessible to the computer system;

generating a set of result items returned from the search over the set of data and a first order of the set of result items;

producing an adjusted order of the set of result items by a method of using human suggested locations comprising;

providing in the database a plurality of previously suggested locations of result items;

wherein the previously suggested locations were received from other end users from one or more prior results documents that were returned in response to one or more requests from the other end users to perform a search using the same search query earlier in time;

determining from the plurality of previously suggested locations that one of the previously suggested locations is a last user-suggested location for a first result item, wherein the last user-suggested location for the first result item is the previously suggested location for the first result item that was most recently received from the other end users;

moving the first result item to the last user-suggested location of the first result item;

wherein, as a result of determining from the plurality of previously suggested locations that one of the previously suggested locations is the last user-suggested location for the first result item and moving the first result item to the last user-suggested location of the first result item, the last user-suggested location for the first result item dictates the location of the first result item in the adjusted order;

providing the adjusted order of the set of result items to the end user in a results document;

receiving one or more suggested locations from the end user for result items in the results document and storing the suggested locations from the end user in the database;

using the suggested locations from the end user at a later time to adjust an order of a later set of result items returned in response to a later search query of a later end user.

2. The method of claim 1, wherein the method of producing the adjusted order of the set of result items by a method of using human suggested locations further comprises:

for each of one or more additional result items besides the first result item, performing a method comprising;

determining from the plurality of previously suggested locations that one of the previously suggested locations is the last user-suggested location for the additional result item;

moving the additional result item to the last user-suggested location for the additional result item;

wherein, as a result of determining from the plurality of previously suggested locations that one of the previously suggested locations is the last user-suggested location for the additional result item and moving the additional result item to the last user-suggested location of the additional result item, the last user-suggested location for the additional result item dictates the location of the additional result item in the adjusted order.

3. The method of claim 1, wherein the results document includes a discussion component for user discussion or a link to a discussion component for user discussion.

4. The method of claim 1, further comprising providing a version control component that allows the end user to view a previous version of the results document created by adjusting the locations of one or more result items according to user-suggested locations provided up to a specific time.

5. The method of claim 1, wherein the suggested locations from the end user are received via one or more interface components allowing selecting and moving one or more result items to a higher or lower location.

6. The method of claim 1, wherein receiving suggested locations from the end user comprises:

providing one or more interface components configured to allow the end user to drag one or more result items to the suggested locations.

7. The method of claim 1, wherein the end user and the other end users are divided into one or more classes;

wherein the end user is in a class;

wherein only previously suggested locations of results items that were provided by other end users in the same class as the end user are used to produce the adjusted order of the set of result items.

8. The method of claim 1, wherein the suggested locations from the end user move at least one result item to a location at least two positions above or two positions below the at least one result item'"'"'s location in the adjusted order.

9. The method of claim 1, further comprising:

in response to receiving a single suggested location from the end user, moving at least one result item to a location at least two positions above or two positions below the at least one result item'"'"'s location in the adjusted order.

10. The method of claim 1, wherein in response to two or more of the result items having the same last user-suggested location, the result item with the last user-suggested location provided at the latest time is moved to the last user-suggested location of the two or more result items and the other result items having the same last user-suggested location are moved to a lower location than the last user-suggested location of the two or more result items.

11. A system including at least one computer system having access to a database, the system configured to at least:

receive at the computer system a search query from an end user;

perform a search over a set of data in response to the search query, the set of data being at the computer system or accessible to the computer system;

generate a set of result items returned from the search over the set of data and a first order of the set of result items;

produce an adjusted order of the set of result items by;

providing in the database a plurality of previously suggested locations of result items;

wherein the previously suggested locations were received from other end users from one or more prior results documents that were returned in response to one or more requests from the other end users to perform a search using the same search query earlier in time;

determining from the plurality of previously suggested locations that one of the previously suggested locations is a last user-suggested location for a first result item, wherein the last user-suggested location for the first result item is the previously suggested location for the first result item that was most recently received from the other end users;

moving the first result item to the last user-suggested location of the first result item;

wherein, as a result of determining from the plurality of previously suggested locations that one of the previously suggested locations is the last user-suggested location for the first result item and moving the first result item to the last user-suggested location of the first result item, the last user-suggested location for the first result item dictates the location of the first result item in the adjusted order;

the system further configured to;

provide the adjusted order of the set of result items to the end user in a results document;

receive one or more suggested locations from the end user for result items in the results document and store the suggested locations from the end user in the database;

use the suggested locations from the end user at a later time to adjust an order of a later set of result items returned in response to a later search query of a later end user.

12. The system of claim 11, further configured to produce the adjusted order of the set of result items by being configured to:

for each of one or more additional result items besides the first result item;

determine from the plurality of previously suggested locations that one of the previously suggested locations is the last user-suggested location for the additional result item;

move the additional result item to the last user-suggested location for the additional result item;

wherein, as a result of determining from the plurality of previously suggested locations that one of the previously suggested locations is the last user-suggested location for the additional result item and moving the additional result item to the last user-suggested location of the additional result item, the last user-suggested location for the additional result item dictates the location of the additional result item in the adjusted order.

13. The system of claim 11, wherein the results document includes a discussion component for user discussion or a link to a discussion component for user discussion.

14. The system of claim 11, further configured to provide a version control component that allows the end user to view a previous version of the results document created by adjusting the locations of one or more result items according to user-suggested locations provided up to a specific time.

15. The system of claim 11, wherein the suggested locations from the end user are received via one or more interface components allowing selecting and moving one or more result items to a higher or lower location.

16. The system of claim 11, wherein the system is configured to receive suggested locations from the end user at least by:

providing one or more interface components configured to allow the end user to drag one or more result items to the suggested locations.

17. The system of claim 11, wherein the end user and the other end users are divided into one or more classes;

wherein the end user is in a class;

wherein only previously suggested locations of results items that were provided by other end users in the same class as the end user are used by the system to produce the adjusted order of the set of result items.

18. The system of claim 11, configured to receive suggested locations from the end user and move at least one result item to a location at least two positions above or two positions below the at least one result item'"'"'s location in the adjusted order.

19. The system of claim 11, configured to at least:

in response to receiving a single suggested location from the end user, move at least one result item to a location at least two positions above or two positions below the at least one result item'"'"'s location in the adjusted order.

20. The system of claim 11, further configured that in response to two or more of the result items having the same last user-suggested location, the result item with the last user-suggested location provided at the latest time is moved to the last user-suggested location of the two or more result items and the other result items having the same last user-suggested location are moved to a lower location than the last user-suggested location of the two or more result items.