Deriving and Using Interaction Profiles

• US 20160098488A1
• Filed: 12/11/2015
• Published: 04/07/2016
• Est. Priority Date: 06/28/2004
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Battle, Alexis J., Cohn, David Ariel, Grimes, Carrie Elizabeth, Lamping, John Ogden
• Granted Patent: US 10,387,512 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/30 Profiles
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

Systems and methods for deriving and using an interaction profile are described. In one described method, a plurality of metrics indicating a level of satisfaction for search results is determined. The metrics comprise at least one of click-duration data, multiple-click data, and query-refinement data. The values of the metrics for a plurality of instances of an object, such as search results from a search engine, are determined. An interaction profile for the object, based at least in part on the values of the metrics for a plurality of instances of the first object, is then determined. This interaction profile may be used in a variety of ways, such as determining the quality of ranking algorithms and detecting undesirable search results.

Claims

1. (canceled)

2. A method performed by a system of one or more computers, the method comprising:

identifying one or more classes of search queries;

comparing, by the system, for each class of search queries, a first scoring algorithm for scoring electronic documents responsive to search queries in the class with a second scoring algorithm for scoring the electronic documents responsive to search queries in the class, wherein the comparing includes;

obtaining a first profile that characterizes historical interaction, by a first plurality of users, with electronic documents in first result sets of electronic documents generated in response to search queries in the class, the electronic documents in each first result set scored by a search engine in accordance with the first scoring algorithm and ranked by the search engine in each first result set in accordance with the scoring,

obtaining a second profile that characterizes historical interaction, by a second plurality of users, with electronic documents in second result sets of electronic documents generated in response to search queries in the class, the electronic documents in each second result set scored by the search engine in accordance with the second scoring algorithm and ranked by the search engine in each second result set in accordance with the scoring,

determining that one or more metrics of the user interaction characterized in the first profile collectively indicate a higher level of user satisfaction with the first result sets of electronic documents than a level of user satisfaction with the second result sets indicated by the one or more metrics of the user interaction characterized in the second profile, and

in response to the determination, outputting an indication that the first scoring algorithm is better than the second scoring algorithm for scoring documents in result sets generated in response to search queries in the class.

3. The method of claim 3, wherein identifying one or more classes of search queries comprises:

receiving data identifying a group of queries as one class.

4. The method of claim 3, wherein identifying one or more classes of search queries comprises:

identifying queries that result in high click duration as one class; and

identifying queries that result in medium or low click duration as another class.

5. The method of claim 3, wherein:

each class of the one or more classes includes a plurality of queries;

obtaining the first profile includes;

obtaining, for each query in the plurality of queries, a first query profile that characterizes historical interaction, by the first plurality of users, with electronic documents in first result sets of electronic documents generated in response to the query; and

combining the first query profiles to obtain the first profile; and

obtaining the second profile includes;

obtaining, for each query in the plurality of queries, a second query profile that characterizes historical interaction, by the second plurality of users, with electronic documents in second result sets of electronic documents generated in response to the query; and

combining the first query profiles to obtain the second profile.

6. The method of claim 3, wherein identifying one or more classes of search queries comprises:

identifying queries that originated from one country as one class; and

identifying queries that originated from another country as another class.

7. The method of claim 3, wherein identifying one or more classes of search queries comprises:

identifying queries that are navigational queries as one class.

8. The method of claim 3, wherein identifying one or more classes of search queries comprises:

identifying queries that are commercial queries as one class.

9. The method of claim 3, wherein identifying one or more classes of search queries comprises:

identifying queries that are category queries as one class.

10. The method of claim 3, wherein identifying one or more classes of search queries comprises:

identifying queries that are general queries as one class.

11. A system comprising:

one or more computers programmed to perform operations, the operations comprising;

identifying one or more classes of search queries;

comparing, by the system, for each class of search queries, a first scoring algorithm for scoring electronic documents responsive to search queries in the class with a second scoring algorithm for scoring the electronic documents responsive to search queries in the class, wherein the comparing includes;

obtaining a first profile that characterizes historical interaction, by a first plurality of users, with electronic documents in first result sets of electronic documents generated in response to search queries in the class, the electronic documents in each first result set scored by a search engine in accordance with the first scoring algorithm and ranked by the search engine in each first result set in accordance with the scoring,

obtaining a second profile that characterizes historical interaction, by a second plurality of users, with electronic documents in second result sets of electronic documents generated in response to search queries in the class, the electronic documents in each second result set scored by the search engine in accordance with the second scoring algorithm and ranked by the search engine in each second result set in accordance with the scoring,

determining that one or more metrics of the user interaction characterized in the first profile collectively indicate a higher level of user satisfaction with the first result sets of electronic documents than a level of user satisfaction with the second result sets indicated by the one or more metrics of the user interaction characterized in the second profile, and

in response to the determination, outputting an indication that the first scoring algorithm is better than the second scoring algorithm for scoring documents in result sets generated in response to search queries in the class.

12. The system of claim 11, wherein identifying one or more classes of search queries comprises:

receiving data identifying a group of queries as one class.

13. The system of claim 11, wherein identifying one or more classes of search queries comprises:

identifying queries that result in high click duration as one class; and

identifying queries that result in medium or low click duration as another class.

14. The system of claim 11, wherein:

each class of the one or more classes includes a plurality of queries;

obtaining the first profile includes;

obtaining, for each query in the plurality of queries, a first query profile that characterizes historical interaction, by the first plurality of users, with electronic documents in first result sets of electronic documents generated in response to the query; and

combining the first query profiles to obtain the first profile; and

obtaining the second profile includes;

obtaining, for each query in the plurality of queries, a second query profile that characterizes historical interaction, by the second plurality of users, with electronic documents in second result sets of electronic documents generated in response to the query; and

combining the first query profiles to obtain the second profile.

15. The system of claim 11, wherein identifying one or more classes of search queries comprises:

identifying queries that originated from one country as one class; and

identifying queries that originated from another country as another class.

16. The system of claim 11, wherein identifying one or more classes of search queries comprises:

identifying queries that are navigational queries as one class.

17. The system of claim 11, wherein identifying one or more classes of search queries comprises:

identifying queries that are commercial queries as one class.

18. The system of claim 11, wherein identifying one or more classes of search queries comprises:

identifying queries that are category queries as one class.

19. The system of claim 11, wherein identifying one or more classes of search queries comprises:

identifying queries that are general queries as one class.