Query feature based data structure retrieval of predicted values

• US 10,600,090 B2
• Filed: 04/26/2016
• Issued: 03/24/2020
• Est. Priority Date: 12/30/2005
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Wright, Daniel, Pregibon, Daryl, Tang, Diane
• Primary Examiner(s): Mobin, Hasanul
• Application Number:
  US15/138,852
• Publication Number:
  US 20170308579A1
• Time in Patent Office: 1,428 Days
• CPC Class Codes: G06Q 30/0277 Online advertisement

Abstract

A system of content/query feature based data structure retrieval of predicted values is provided. The system can create a data structure having a plurality of rows corresponding to individual content/query features and a plurality of columns corresponding to individual predicted values. The processors can obtain a set of session features associated with a selection by a computing device in response to a query, and a set of content/query features associated with the selection of the content item. The processors can retrieve, from the data structure, a set of predicted values for each of the set of content/query features. The processors can generate, for each of the set of content/query features, a set of aggregate predicted values for each of the set of content/query features, and can include the set of aggregate predicted values in the data structure.

Claims

1. A system of query feature based data structure retrieval of predicted values, comprising:

a data processing system including a memory and one or more processors to;

create a data structure having a plurality of rows corresponding to content/query features and a plurality of columns corresponding to predicted values, wherein the predicted values comprise a total number of content item selections, a total good predictive value, a total bad predictive value, good content item odds, and bad content item odds, respectively;

determine a first session feature associated with a selection of a first content item during a first session;

populate the data structure based on content selected during the first session;

determine a second session feature associated with a selection of a second content item during a second session;

determine that the second session feature corresponds to the first session feature;

determine during the second session, using a statistical model, a set of predicted quality values associated with a content item, the statistical-model derived based on previously rated content items from the first session, the statistical model including a model parameter;

obtain during the second session a set of content/query features associated with the selection of the content item, the set of content/query features including;

an identifier associated with a provider of the content item, and

a keyword associated with the content item, and

wherein the set of content/query features further includes;

an identifier associated with the content item,

a word in the query that the content item did not target,

a length of the query, or

a quantity of words in the query that are not in keywords associated with the content item;

retrieve, during the second session, from the data structure populated during the first session, a set of predicted values for each of the set of content/query features and an odds value for each of the set of content/query features;

create, during the second session, for each of the set of content/query features, a set of aggregate predicted values for each of the set of content/query features by combining the set of predicted quality values with the set of predicted values;

estimate, during the second session, for each content/query feature, a predicted odds value based on the model parameter and the respective odds value; and

include the set of aggregate predicted values and the predicted odds value in the data structure.

2. The system of claim 1, comprising the data processing system configured to:

determine, for each content item of a set of content items obtained in response to a second query received from a second user computing device, a second set of content/query features;

retrieve, from the data structure for each content/query feature of each second set of content/query features, a second set of predicted values;

combine, for each content item of the set of content items, the second set of predicted values for each of the second set of content/query features to determine a set of total predicted values;

rank each content item of the set of content items based on the total predicted values; and

provide, to the second user computing device over the computing network, an content item of the set of content items based on the ranking.

3. The system of claim 2, comprising the data processing system configured to:

combine the second set of predicted values for each of the second set of content/query features to determine the set of total predicted values by multiplying the corresponding respective individual predicted values for each of the second set of content/query features.

4. The system of claim 1, wherein the query is a search query submitted to a search engine, and the content item is provided to the user computing device with search results responsive to the search query.

5. The system of claim 1, wherein the set of session features further includes:

a duration of a click on the content item,

a quantity of selections of other content items before or after the selection of the content item,

a quantity of selections of search results before or after the selection of the content item,

a quantity of selections of other results before or after the selection of the content item,

a quantity of document views before or after the selection of the content item,

a quantity of search queries before or after the selection of the content item, or

a quantity of search queries associated with a user session that show content items.

6. The system of claim 1, wherein the set of session features further includes:

an indication that the selection of the content item was a last selection in a session, a last content item selection in a session, a last selection for a given search query, or the last content item selection for a given search query.

7. A method of content/query feature based data structure retrieval of predicted values, comprising:

creating, using one or more processors of a data processing system, a data structure having a plurality of rows corresponding to individual content/query features and a plurality of columns corresponding to individual predicted values, wherein the individual predicted values comprise a total number of content item selections, a total good predictive value, a total bad predictive value, good content item odds, and bad content item odds, respectively;

determining a first session feature associated with a selection of a first content item during a first session;

populating the data structure based on content selected during the first session;

determining a second session feature associated with a selection of a second content item during a second session;

determining that the second session feature corresponds to the first session feature;

determining during the second session, by the one or more processors using a statistical model, a set of predicted quality values associated with a content item, the statistical model derived based on previously rated content items from the first session, the statistical model including a model parameter;

obtaining during the second session, by the data processing system, a set of content/query features associated with the selection of the content item, the set of content/query features including;

an identifier associated with a provider of the content item, and

a keyword associated with the content item, and

wherein the set of content/query features further includes;

an identifier associated with the content item,

a word in the query that the content item did not target,

a length of the query, or

a quantity of words in the query that are not in keywords associated with the content item;

retrieving during the second session, by the one or more processors from the data structure populated during the first session, a set of predicted values for each of the set of content/query features and an odds value for each of the set of content/query features;

creating during the second session, by the one or more processors, for each of the set of content/query features, a set of aggregate predicted values for each of the set of content/query features by combining the set of predicted quality values with the set of predicted values;

estimating during the second session, by the one or more processors, for each of the set of content/query features, a predicted odds value based on the model parameter and the respective odds value; and

including during the second session, by the one or more processors, the set of aggregate predicted values in the data structure.

8. The method of claim 7, comprising:

determining, by the one or more processors for each content item of a set of content items obtained in response to a second query received from a second user computing device, a second set of content/query features;

retrieving, by the one or more processors from the data structure for each content/query feature of each second set of content/query features, a second set of predicted values;

combining, by the one or more processors for each content item of the set of content items, the second set of predicted values for each of the second set of content/query features to determine a set of total predicted values;

ranking, by the one or more processors, each content item of the set of content items based on the total predicted values; and

providing, by the one or more processors to the second user computing device, a content items of the set of content items based on the ranking.

9. The method of claim 8, comprising:

combining the second set of predicted values for each of the second set of content/query features to determine the set of total predicted values by multiplying the corresponding respective individual predicted values for each of the second set of content/query features.

10. The system of claim 7, wherein the query is a search query submitted to a search engine, and the content item is provided to the user computing device with search results responsive to the search query.

11. The method of claim 7, wherein the set of session features further includes:

a duration of a click on the content item,

a quantity of selections of other content items before or after the selection of the content item,

a quantity of selections of search results before or after the selection of the content item,

a quantity of selections of other results before or after the selection of the content item,

a quantity of document views before or after the selection of the content item,

a quantity of search queries before or after the selection of the content item,

a quantity of search queries associated with a user session that show content items, or

an indication that the selection of the content item was a last selection in a session, a last content item selection in a session, a last selection for a given search query, or the last content item selection for a given search query.