Determining resource quality based on resource competition

• US 10,133,788 B1
• Filed: 03/17/2015
• Issued: 11/20/2018
• Est. Priority Date: 06/01/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Hammad, Moustafa A., Kim, Hyung-Jin, Patel, Rajan, Bagby, Thomas E.
• Primary Examiner(s): Jami, Hares
• Assistant Examiner(s): Sultana, Nargis
• Application Number:
  US14/660,388
• Time in Patent Office: 1,344 Days
• Field of Search: 707709, 707723
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

Methods, systems, and apparatus for determining resource quality based on resource competition. In an aspect, a method comprises: for each of a plurality of resource locators: generating first value for the resource locator that indicates, for a plurality of first sets of search results that each include the resource locator, a number of occurrences of other resource locators that were impressed and not selected when the resource locator was selected; generating a second value for the resource locator that indicates, for a plurality of second sets of search results that each include the resource locator, a number of occurrences of other resource locators that were selected when the resource locator was impressed and not selected; and generating, based on a difference between the first value and the second value, an adjustment factor for the resource locator for adjusting a score associated with the resource locator during a search operation.

Claims

1. A computer implemented method performed by data processing apparatus, the method comprising:

for each of a plurality of domains;

generating a first value for the domain that indicates, for a plurality of first sets of search results, wherein each first set was provided in response to a query and each first set includes a resource locator that belongs to the domain, a number of occurrences of other resource locators that were displayed with the resource locator and not selected when the resource locator was displayed and selected;

generating a second value for the resource locator that indicates, for a plurality of second sets of search results, wherein each second set was provided in response to the query and each second set includes the resource locator that belongs to the domain, a number of occurrences of other resource locators that were selected when the resource locator was displayed with the other resource locators and not selected; and

generating, for the search query and based on a difference between the first value and the second value, an adjustment factor for the domain for adjusting a score associated with a resource locator belonging to the domain during a search operation for the search query, the generating of the adjustment factor comprising determining a ratio of the difference of the first value and the second value to a maximum of the first value and the second value;

receiving a search query;

obtaining search results for the search query, wherein each search result references a resource locator;

for each resource locator, determining whether a sum of the first value and second value associated with the domain of the resource locator meets a threshold value;

for only each resource locator for which the sum is determined to meet the threshold value, determining the adjustment value for the domain to which the resource locator belongs as an adjustment factor for the resource locator; and

ranking the search results at least in part based on the adjustment factors associated with resource locators.

2. The method of claim 1, wherein each resource locator that belongs to the domain and that was displayed is a resource locator that was displayed in a list of search results presented in response to the query when another resource locator that was in a lower position in the list of search results, relative to the resource locator, was selected.

3. The method of claim 1, wherein the first value for the resource locator only indicates the number of occurrences of other resource locators being displayed and not selected when the other displayed and not selected resource locators are displayed in a higher position in a list of search results, relative to the resource locator.

4. The method of claim 1, wherein the second value for the resource locator only indicates the number of occurrences of other resource locators being selected when a selected resource locator is displayed in a lower position in a list of search results, relative to the resource locator.

5. The method of claim 1, wherein the first value is further based on, for each occurrence of other resource locators that were displayed and not selected when the resource locator was selected, an ordinal position of the resource locator in the corresponding set of search results relative ordinal positions of the other resource locators, and wherein the second value is further based on, for each occurrence of other resource locators that were selected when the resource locator was displayed and not selected, an ordinal position of the resource locator in the corresponding set of search results relative ordinal positions of the other resource locators.

6. A computer implemented method performed by data processing apparatus, the method comprising:

for each of a plurality of domains;

generating a first value for the domain that indicates, for a plurality of first sets of search results, wherein each first set was provided in response to a query and each first set includes a resource locator that belongs to the domain, a number of occurrences of other resource locators that were displayed with the resource locator and not selected when the resource locator was displayed and selected;

generating a second value for the resource locator that indicates, for a plurality of second sets of search results, wherein each second set was provided in response to the query and each second set includes the resource locator that belongs to the domain, a number of occurrences of other resource locators that were selected when the resource locator was displayed with the other resource locators and not selected; and

generating, for the search query and based on a difference between the first value and the second value, an adjustment factor for the domain for adjusting a score associated with a resource locator belonging to the domain during a search operation for the search query;

wherein generating an adjustment factor for the resource locator comprises;

determining a maximum value that is equal to the greater of the first value and second value;

determining a difference value that is equal to the first value minus the second value;

determining a quotient value that is equal to the difference value divided by the maximum value; and

determining the adjustment factor based on a product of a constant to the power of the quotient;

receiving a search query;

obtaining search results for the search query, wherein each search result references a resource locator;

for each resource locator, determining whether a sum of the first value and second value associated with the domain of the resource locator meets a threshold value;

for only each resource locator for which the sum is determined to meet the threshold value, determining the adjustment value for the domain to which the resource locator belongs as an adjustment factor for the resource locator; and

ranking the search results at least in part based on the adjustment factors associated with resource locators.

7. A system, comprising:

a data processing apparatus having one or more processors; and

a data store storing instructions that, when executed by the data processing apparatus, cause the data processing apparatus to perform operations comprising;

for each of a plurality of domains;

generating a first value for the domain that indicates, for a plurality of first sets of search results, wherein each first set was provided in response to a query and each first set includes a resource locator that be longs to the domain, a number of occurrences of other resource locators that were displayed with the resource locator and not selected when the resource locator was displayed and selected;

generating a second value for the resource locator that indicates, for a plurality of second sets of search results, wherein each second set was provided in response to the query and each second set includes the resource locator that belongs to the domain, a number of occurrences of other resource locators that were selected when the resource locator was displayed with the other resource locators and not selected; and

generating, for the search query and based on a difference between the first value and the second value, an adjustment factor for the domain for adjusting a score associated with a resource locator belonging to the domain during a search operation for the search query, the generating of the adjustment factor comprising determining a ratio of the difference of the first value and the second value to a maximum of the first value and the second value;

receiving a search query;

obtaining search results for the search query, wherein each search result references a resource locator;

for each resource locator, determining whether a sum of the first value and second value associated with the domain of the resource locator meets a threshold value;

for only each resource locator for which the sum is determined to meet the threshold value, determining the adjustment value for the domain to which the resource locator belongs an adjustment factor for the resource locator; and

ranking the search results at least in part based on the adjustment factors associated with resource locators.

8. The system of claim 7, wherein each resource locator that belongs to the domain and that was displayed is a resource locator that was displayed in a list of search results presented in response to the query when another resource locator that was in a lower position in the list of search results, relative to the resource locator, was selected.

9. The system of claim 7, wherein the first value for the resource locator only indicates the number of occurrences of other resource locators being displayed and not selected when the other displayed and not selected resource locators are displayed in a higher position in a list of search results, relative to the resource locator.

10. The system of claim 7, wherein the second value for the resource locator only indicates the number of occurrences of other resource locators being selected when a selected resource locator is displayed in a lower position in a list of search results, relative to the resource locator.

11. The system of claim 7, wherein the first value is further based on, for each occurrence of other resource locators that were displayed and not selected when the resource locator was selected, an ordinal position of the resource locator in the corresponding set of search results relative ordinal positions of the other resource locators, and wherein the second value is further based on, for each occurrence of other resource locators that were selected when the resource locator was displayed and not selected, an ordinal position of the resource locator in the corresponding set of search results relative ordinal positions of the other resource locators.

12. A system, comprising:

a data processing apparatus having one or more processors; and

a data store storing instructions that, when executed by the data processing apparatus, cause the data processing apparatus to perform operations comprising;

for each of a plurality of domains;

generating a first value for the domain that indicates, for a plurality of first sets of search results, wherein each first set was provided in response to a query and each first set includes a resource locator that be longs to the domain, a number of occurrences of other resource locators that were displayed with the resource locator and not selected when the resource locator was displayed and selected;

generating a second value for the resource locator that indicates, for a plurality of second sets of search results, wherein each second set was provided in response to the query and each second set includes the resource locator that belongs to the domain, a number of occurrences of other resource locators that were selected when the resource locator was displayed with the other resource locators and not selected; and

generating, for the search query and based on a difference between the first value and the second value, an adjustment factor for the domain for adjusting a score associated with a resource locator belonging to the domain during a search operation for the search query;

wherein generating an adjustment factor for the resource locator comprises;

determining a maximum value that is equal to the greater of the first value and second value;

determining a difference value that is equal to the first value minus the second value;

determining a quotient value that is equal to the difference value divided by the maximum value; and

determining the adjustment factor based on a product of a constant to the power of the quotient;

receiving a search query;

obtaining search results for the search query, wherein each search result references a resource locator;

for each resource locator, determining whether a sum of the first value and second value associated with the domain of the resource locator meets a threshold value;

for only each resource locator for which the sum is determined to meet the threshold value, determining the adjustment value for the domain to which the resource locator belongs as an adjustment factor for the resource locator; and

ranking the search results at least in part based on the adjustment factors associated with resource locators.

13. A data storage device storing instructions that, when executed by a data processing apparatus, cause the data processing apparatus to perform operations comprising:

for each of a plurality of domains;

generating a first value for the domain that indicates, for a plurality of first sets of search results, wherein each first set was provided in response to a query and each first set includes a resource locator that be longs to the domain, a number of occurrences of other resource locators that were displayed with the resource locator and not selected when the resource locator was displayed and selected;

generating a second value for the resource locator that indicates, for a plurality of second sets of search results, wherein each second set was provided in response to the query and each second set includes the resource locator that belongs to the domain, a number of occurrences of other resource locators that were selected when the resource locator was displayed with the other resource locators and not selected; and

generating, for the search query and based on a difference between the first value and the second value, an adjustment factor for the domain for adjusting a score associated with a resource locator belonging to the domain during a search operation for the search query;

wherein generating an adjustment factor for the resource locator comprises;

determining a maximum value that is equal to the greater of the first value and second value;

determining a difference value that is equal to the first value minus the second value;

determining a quotient value that is equal to the difference value divided by the maximum value; and

determining the adjustment factor based on a product of a constant to the power of the quotient;

receiving a search query;

obtaining search results for the search query, wherein each search result references a resource locator;

for each resource locator, determining whether a sum of the first value and second value associated with the domain of the resource locator meets a threshold value;

for only each resource locator for which the sum is determined to meet the threshold value, determining the adjustment value for the domain to which the resource locator belongs as an adjustment factor for the resource locator; and

ranking the search results at least in part based on the adjustment factors associated with resource locators.