Deriving and using interaction profiles

• US 9,223,868 B2
• Filed: 12/01/2011
• Issued: 12/29/2015
• Est. Priority Date: 06/28/2004
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Cohn, David Ariel, Grimes, Carrie Elizabeth, Lamping, John Ogden, Battle, Alexis Jane
• Primary Examiner(s): Jalil, Neveen Abel
• Assistant Examiner(s): ELLIS, MATTHEW J
• Application Number:
  US13/308,770
• Publication Number:
  US 20150161255A1
• Time in Patent Office: 1,489 Days
• Field of Search: 707722-734
• US Class Current: 1/1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/30 Profiles
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

Systems and methods for deriving and using an interaction profile are described. In one described method, a plurality of metrics indicating a level of satisfaction for search results is determined. The metrics comprise at least one of click-duration data, multiple-click data, and query-refinement data. The values of the metrics for a plurality of instances of an object, such as search results from a search engine, are determined. An interaction profile for the object, based at least in part on the values of the metrics for a plurality of instances of the first object, is then determined. This interaction profile may be used in a variety of ways, such as determining the quality of ranking algorithms and detecting undesirable search results.

Claims

1. A method performed by a system of one or more computers, the method comprising:

comparing, by the system, a first scoring algorithm for scoring electronic documents responsive to search queries with a second scoring algorithm for scoring the electronic documents responsive to search queries, wherein the comparing includes;

obtaining a first profile that characterizes historical interaction, by a first plurality of users, with electronic documents in first result sets of electronic documents, the electronic documents in each first result set scored by a search engine in accordance with the first scoring algorithm and ranked by the search engine in each first result set in accordance with said scoring,

obtaining a second profile that characterizes historical interaction, by a second plurality of users, with electronic documents in second result sets of electronic documents, the electronic documents in each second result set scored by the search engine in accordance with the second scoring algorithm and ranked by the search engine in each second result set in accordance with said scoring,

determining that one or more metrics of the user interaction characterized in the first profile collectively indicate a higher level of user satisfaction with the first result sets of electronic documents than a level of user satisfaction with the second result sets indicated by the one or more metrics of the user interaction characterized in the second profile, and

in response to the determination, outputting an indication that the first scoring algorithm is better than the second scoring algorithm,

wherein the one or more metrics of the user interaction characterized in the first profile comprise a total number of respective selections of one or more electronic documents from each of the first result sets of electronic documents,

wherein the one or more metrics of the user interaction characterized in the second profile comprise a total number of respective selections of one or more electronic documents from each of the second result sets of electronic documents, and

wherein the higher level of user satisfaction is indicated by a lower total number of respective selections of the one or more electronic documents from each of the first result sets of electronic documents than the total number of respective selections of the one or more electronic documents from each of the second result sets of electronic documents.

2. The method of claim 1, wherein:

the one or more metrics of the user interaction further comprise a number of refinements of the search queries; and

the higher level of user satisfaction is further indicated by a lower number of refinements by the first plurality of users or the second plurality of users.

3. The method of claim 1, wherein:

the one or more metrics of the user interaction characterized in the first profile further comprise a first ratio of long duration selections of electronic documents from the first result sets of electronic documents to short duration selections of electronic documents from the first result sets of electronic documents;

the one or more metrics of the user interaction characterized in the second profile further comprise a second ratio of long duration selections of electronic documents from the second result sets of electronic documents to short duration selections of electronic documents from the second result sets of electronic documents; and

the higher level of user satisfaction is further indicated by the first ratio being greater than the second ratio.

4. The method of claim 1, wherein:

the first profile characterizes one or more respective interactions with electronic documents in each of the first result sets of electronic documents responsive to search queries of a first category,

the search queries of the first category are selected from among a plurality of queries including the search queries of the first category and other search queries,

the second profile characterizes one or more respective interactions with electronic documents in each of the second result sets of electronic documents responsive to search queries of the first category, and

the indication indicates that the first scoring algorithm is better than the second scoring algorithm for scoring electronic documents responsive to search queries of the first category.

5. The method of claim 4, wherein the first category is navigational queries, commercial queries, long queries, short queries, or general queries.

6. The method of claim 4, wherein the first category is queries having syntax.

7. The method of claim 4, wherein obtaining the first profile comprises accessing, for each of a plurality of search queries of the first category:

data characterizing one or more properties of a session in which the respective search query was submitted to the search engine and results returned in response to the submission of the respective search query,

data characterizing one or more properties of the respective search query, and

data characterizing one or more properties of the returned results.

8. A system comprising:

one or more computers programmed to perform operations, the operations comprising;

comparing, by the system, a first scoring algorithm for scoring electronic documents responsive to search queries with a second scoring algorithm for scoring the electronic documents responsive to search queries, wherein the comparing includes;

obtaining a first profile that characterizes historical interaction, by a first plurality of users, with electronic documents in first result sets of electronic documents, the electronic documents in each first result set scored by a search engine in accordance with the first scoring algorithm and ranked by the search engine in each first result set in accordance with said scoring,

obtaining a second profile that characterizes historical interaction, by a second plurality of users, with electronic documents in second result sets of electronic documents, the electronic documents in each second result set scored by the search engine in accordance with the second scoring algorithm and ranked by the search engine in each second result set in accordance with said scoring,

determining that one or more metrics of the user interaction characterized in the first profile collectively indicate a higher level of user satisfaction with the first result sets of electronic documents than a level of user satisfaction with the second result sets indicated by the one or more metrics of the user interaction characterized in the second profile, and

in response to the determination, outputting an indication that the first scoring algorithm is better than the second scoring algorithm,

wherein the one or more metrics of the user interaction characterized in the first profile comprise a total number of respective selections of one or more electronic documents from each of the first result sets of electronic documents,

wherein the one or more metrics of the user interaction characterized in the second profile comprise a total number of respective selections of one or more electronic documents from each of the second result sets of electronic documents, and

wherein the higher level of user satisfaction is indicated by a lower total number of respective selections of the one or more electronic documents from each of the first result sets of electronic documents than the total number of respective selections of the one or more electronic documents from each of the second result sets of electronic documents.

9. The system of claim 8, wherein:

the first profile characterizes one or more respective interactions with electronic documents in each of the first result sets of electronic documents responsive to search queries of a first category,

the search queries of the first category are selected from among a plurality of queries including the search queries of the first category and other search queries,

the second profile characterizes one or more respective interactions with electronic documents in each of the second result sets of electronic documents responsive to search queries of the first category, and

the indication indicates that the first scoring algorithm is better than the second scoring algorithm for scoring electronic documents responsive to search queries of the first category.

10. The system of claim 9, wherein the first category is navigational queries, commercial queries, long queries, short queries, or general queries.

11. The system of claim 9, wherein the first category is queries having syntax.

12. The system of claim 9, wherein obtaining the first profile comprises accessing, for each of a plurality of search queries of the first category:

data characterizing one or more properties of a session in which the respective search query was submitted to the search engine and results returned in response to the submission of the respective search query,

data characterizing one or more properties of the respective search query, and

data characterizing one or more properties of the returned results.

13. The system of claim 8, wherein:

the one or more metrics of the user interaction characterized in the first profile further comprise a first ratio of long duration selections of electronic documents from the first result sets of electronic documents to short duration selections of electronic documents from the first result sets of electronic documents;

the one or more metrics of the user interaction characterized in the second profile further comprise a second ratio of long duration selections of electronic documents from the second result sets of electronic documents to short duration selections of electronic documents from the second result sets of electronic documents; and

the higher level of user satisfaction is further indicated by the first ratio being greater than the second ratio.

14. The system of claim 8, wherein:

the one or more metrics of the user interaction further comprise a number of refinements of the search queries; and

the higher level of user satisfaction is further indicated by a lower number of refinements by the first plurality of users or the second plurality of users.