Propagating information among web pages

• US 8,521,717 B2
• Filed: 04/21/2011
• Issued: 08/27/2013
• Est. Priority Date: 03/31/2006
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Egnor, Daniel, Haahr, Paul, Lamping, John, Singhal, Amitabh K., Yang, Ke, Lackner, Kevin
• Primary Examiner(s): Beausoliel, Jr., Robert
• Assistant Examiner(s): ALLEN, NICHOLAS E
• Application Number:
  US13/092,100
• Publication Number:
  US 20110196861A1
• Time in Patent Office: 859 Days
• Field of Search: 707/709, 707/999.005, 707/741
• US Class Current: 707/709
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking

Abstract

Web pages of a Website may be processed to improve search results. For example, information likely to pertain to more than just the Web page it is directly associated with may be identified. One or more other, related, Web pages that such information is likely to pertain to is also identified. The identified information is associated with the identified other Web page(s) and this association is saved in a way to affect a search result score of the Web page(s).

Claims

1. A computer-implemented method comprising:

selecting a candidate term from among one or more terms in a first web page of a web site, wherein the first web page is not a home or root web page of the web site;

determining that the candidate term is uncommon, wherein an uncommon term is a term that occurs with less than a threshold frequency in a collection of web pages; and

in an index of web pages that identifies terms and, for each term, identifies one or more web pages that are associated with the term, updating an association of the candidate term with a home or root web page of the web site to generate an updated index of web pages, including updating a boost value associated with the association of the candidate term with the home or root web page in the index of web pages.

2. The method of claim 1, wherein the home or root web page is above the first web page in a URL hierarchy of the web site.

3. The method of claim 1, wherein the index of web pages has no association between the candidate term and the home or root web page of the web site, and wherein updating an association of the candidate term with the home or root web page of the web site comprises adding an association of the candidate term with the home or root web page of the web site to generate the updated index of web pages.

4. The method of claim 1, wherein updating the association of the candidate term with the home or root web page of the web site comprises increasing a weight of an association of the candidate term with the home or root web page of the web site in the index of web pages to generate the updated index of web pages.

5. The method of claim 1, wherein determining that the candidate term is uncommon comprises determining that the candidate term has an inverse document frequency with reference to the collection of web pages, the inverse document frequency being above a threshold.

6. A system comprising:

one or more processors;

at least one input device; and

one or more storage devices storing processor-executable instructions which, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

selecting a candidate term from among one or more terms in a first web page of a web site, wherein the first web page is not a home or root web page of the web site;

determining that the candidate term is uncommon, wherein an uncommon term is a term that occurs with less than a threshold frequency in a collection of web pages; and

in an index of web pages that identifies terms and, for each term, identifies one or more web pages that are associated with the term, updating an association of the candidate term with a home or root web page of the web site to generate an updated index of web pages, including updating a boost value associated with the association of the candidate term with the home or root web page in the index of web pages.

7. The system of claim 6, wherein the home or root web page is above the first web page in a URL hierarchy of the web site.

8. The system of claim 6, wherein the index of web pages has no association between the candidate term and the home or root web page of the web site, and wherein updating an association of the candidate term with the home or root web page of the web site comprises adding an association of the candidate term with the home or root web page of the web site to generate the updated index of web pages.

9. The system of claim 6, wherein updating the association of the candidate term with the home or root web page of the web site comprises increasing a weight of an association of the candidate term with the home or root web page of the web site in the index of web pages to generate the updated index of web pages.

10. The system of claim 6, wherein determining that the candidate term is uncommon comprises determining that the candidate term has an inverse document frequency with reference to the collection of web pages, the inverse document frequency being above a threshold.

11. A computer-implemented method comprising:

selecting a candidate term from among one or more terms in a first web page of a web site, wherein the first web page is not a home or root web page of the web site;

determining that the candidate term is uncommon, wherein an uncommon term is a term that occurs with less than a threshold frequency in a collection of web pages;

identifying in the web site a second web page that is above the first web page in a hierarchy of the web site and that is within a predetermined distance from the first web page in the hierarchy; and

in an index of web pages that identifies terms and, for each term, identifies one or more web pages that are associated with the term, updating an association of the candidate term with the second web page of the web site to generate an updated index of web pages, including updating a boost value associated with the association of the candidate term with the second web page in the index of web pages to generate the updated index of web pages.

12. The method of claim 11, wherein the hierarchy of the web site is a web site topology.

13. The method of claim 11, wherein the hierarchy of the web site is a URL hierarchy.

14. The method of claim 11, wherein updating the index of web pages to update an association of the candidate term with the second web page of the web site comprises adding an association of the candidate term with the second web page of the web site to generate the updated index of web pages.

15. The method of claim 11, wherein updating the index of web pages to update an association of the candidate term with the second web page of the web site comprises increasing a weight of an association of the candidate term with the second web page of the web site in the index of web pages to generate the updated index of web pages.

16. The method of claim 11, wherein the predetermined distance comprises a number of links between the first web page and the second web page.

17. The method of claim 11, wherein the predetermined distance comprises a number of levels of a URL hierarchy.

18. A system comprising:

one or more processors;

at least one input device; and

one or more storage devices storing processor-executable instructions which, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

selecting a candidate term from among one or more terms in a first web page of a web site, wherein the first web page is not a home or root web page of the web site;

determining that the candidate term is uncommon, wherein an uncommon term is a term that occurs with less than a threshold frequency in a collection of web pages;

identifying in the web site a second web page that is above the first web page in a hierarchy of the web site and that is within a predetermined distance from the first web page in the hierarchy; and

in an index of web pages that identifies terms and, for each term, identifies one or more web pages that are associated with the term, updating an association of the candidate term with the second web page of the web site to generate an updated index of web pages, including updating a boost value associated with the association of the candidate term with the second web page in the index of web pages to generate the updated index of web pages.

19. The system of claim 18, wherein the hierarchy of the web site is a web site topology.

20. The system of claim 18, wherein the hierarchy of the web site is a URL hierarchy.

21. The system of claim 18, wherein updating the index of web pages to update an association of the candidate term with the second web page of the web site comprises adding an association of the candidate term with the second web page of the web site to generate the updated index of web pages.

22. The system of claim 18, wherein updating the index of web pages to update an association of the candidate term with the second web page of the web site comprises increasing a weight of an association of the candidate term with the second web page of the web site in the index of web pages to generate the updated index of web pages.

23. The system of claim 18, wherein the predetermined distance comprises a number of links between the first web page and the second web page.

24. The system of claim 18, wherein the predetermined distance comprises a number of levels of a URL hierarchy.

25. A computer-implemented method comprising:

selecting a candidate term from among one or more terms in a first web page of a web site, wherein the first web page is not a home or root web page of the web site;

determining that the candidate term is uncommon, wherein an uncommon term is a term that occurs with less than a threshold frequency in a collection of web pages; and

in an index of web pages that identifies terms and, for each term, identifies one or more web pages that are associated with the term, updating an association of the candidate term with a home or root web page of the web site to generate an updated index of web pages,

wherein the index of web pages has no association between the candidate term and the home or root web page of the web site, and wherein updating the association of the candidate term with the home or root web page of the web site comprises adding an association of the candidate term with the home or root web page of the web site to generate the updated index of web pages;

receiving a search query including the candidate term; and

determining that the home or root web page satisfies the query if the updated index is used to perform the search requested by the search query.

26. The method of claim 25, wherein the home or root web page is above the first web page in a URL hierarchy of the web site.

27. The method of claim 25, wherein updating the association of the candidate term with the home or root web page of the web site comprises updating a boost value associated with the association of the candidate term with the home or root web page in the index of web pages to generate the updated index of web pages.

28. The method of claim 25, wherein determining that the candidate term is uncommon comprises determining that the candidate term has an inverse document frequency with reference to the collection of web pages, the inverse document frequency being above a threshold.

29. A system comprising:

one or more processors;

at least one input device; and

one or more storage devices storing processor-executable instructions which, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

selecting a candidate term from among one or more terms in a first web page of a web site, wherein the first web page is not a home or root web page of the web site;

determining that the candidate term is uncommon, wherein an uncommon term is a term that occurs with less than a threshold frequency in a collection of web pages; and

in an index of web pages that identifies terms and, for each term, identifies one or more web pages that are associated with the term, updating an association of the candidate term with a home or root web page of the web site to generate an updated index of web pages;

receiving a search query including the candidate term; and

determining that the home or root web page satisfies the query if the updated index is used to perform the search requested by the search query.

30. The system of claim 29, wherein the home or root web page is above the first web page in a URL hierarchy of the web site.

31. The system of claim 29, wherein updating the association of the candidate term with the home or root web page of the web site comprises updating a boost value associated with the association of the candidate term with the home or root web page in the index of web pages to generate the updated index of web pages.

32. The system of claim 29, wherein determining that the candidate term is uncommon comprises determining that the candidate term has an inverse document frequency with reference to the collection of web pages, the inverse document frequency being above a threshold.

33. A computer-implemented method comprising:

selecting a candidate term from among one or more terms in a first web page of a web site, wherein the first web page is not a home or root web page of the web site;

determining that the candidate term is uncommon, wherein an uncommon term is a term that occurs with less than a threshold frequency in a collection of web pages;

identifying in the web site a second web page that is above the first web page in a hierarchy of the web site and that is within a predetermined distance from the first web page in the hierarchy; and

in an index of web pages that identifies terms and, for each term, identifies one or more web pages that are associated with the term, updating an association of the candidate term with the second web page of the web site to generate an updated index of web pages,

wherein the index of web pages has no association between the candidate term and the second web page of the web site, and wherein updating the association of the candidate term with the second web page of the web site comprises adding an association of the candidate term with the second web page of the web site to generate the updated index of web pages;

receiving a search query including the candidate term; and

determining that the second web page satisfies the query if the updated index is used to perform the search requested by the search query.

34. The method of claim 33, wherein the hierarchy of the web site is a web site topology.

35. The method of claim 33, wherein the hierarchy of the web site is a URL hierarchy.

36. The method of claim 33, wherein updating the index of web pages to update an association of the candidate term with the second web page of the web site comprises updating a boost value associated with the association of the candidate term with the second web page in the index of web pages to generate the updated index of web pages.

37. The method of claim 33, wherein the predetermined distance comprises a number of links between the first web page and the second web page.

38. The method of claim 33, wherein the predetermined distance comprises a number of levels of a URL hierarchy.

39. A system comprising:

one or more processors;

at least one input device; and

one or more storage devices storing processor-executable instructions which, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

selecting a candidate term from among one or more terms in a first web page of a web site, wherein the first web page is not a home or root web page of the web site;

determining that the candidate term is uncommon, wherein an uncommon term is a term that occurs with less than a threshold frequency in a collection of web pages;

identifying in the web site a second web page that is above the first web page in a hierarchy of the web site and that is within a predetermined distance from the first web page in the hierarchy; and

in an index of web pages that identifies terms and, for each term, identifies one or more web pages that are associated with the term, updating an association of the candidate term with the second web page of the web site to generate an updated index of web pages,

wherein the index of web pages has no association between the candidate term and the second web page of the web site, and wherein updating the association of the candidate term with the second web page of the web site comprises adding an association of the candidate term with the second web page of the web site to generate the updated index of web pages;

receiving a search query including the candidate term; and

determining that the second web page satisfies the query if the updated index is used to perform the search requested by the search query.

40. The system of claim 39, wherein the hierarchy of the web site is a web site topology.

41. The system of claim 39, wherein the hierarchy of the web site is a URL hierarchy.

42. The system of claim 39, wherein updating the index of web pages to update an association of the candidate term with the second web page of the web site comprises updating a boost value associated with the association of the candidate term with the second web page in the index of web pages to generate the updated index of web pages.

43. The system of claim 39, wherein the predetermined distance comprises a number of links between the first web page and the second web page.

44. The system of claim 39, wherein the predetermined distance comprises a number of levels of a URL hierarchy.