System and method for generating functions to predict the clickability of advertisements

• US 7,631,008 B2
• Filed: 06/29/2006
• Issued: 12/08/2009
• Est. Priority Date: 11/16/2005
• Status: Expired due to Fees

• Current Assignee: R2 Solutions LLC (Acacia Research Corporation)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: LuVogt, Christopher C., Leggetter, Christopher, Kannan, Ashvin, Iyer, Rukmini, Subasic, Pero, Cantu-Paz, Erick, Carson, Chad, Pedersen, Jan, Ku, David Cho-Lun
• Primary Examiner(s): LEWIS, CHERYL RENEA
• Application Number:
  US11/479,186
• Publication Number:
  US 20070112840A1
• Time in Patent Office: 1,258 Days
• Field of Search: 707 2- 5, 707/100, 707/104.1, 707/102, 705 10- 14, 715/812, 345/160, 345/163
• US Class Current: 705/14.45
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/02 Neural networks
• CPC Class Codes: G06N 5/01 Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
• CPC Class Codes: G06Q 10/04 Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem" market predictions or forecasting for commercial activities G06Q30/0202
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0202 Market predictions or forecasting for commercial activities
• CPC Class Codes: G06Q 30/0242 Determining effectiveness of advertisements
• CPC Class Codes: G06Q 30/0246 Traffic

Abstract

The present invention is directed towards systems and methods for predicting a frequency with which an advertisement displayed in response to a query will be selected. The method of the present invention comprises receiving analytics data associated with a display of one or more advertisements in response to one or more queries. One or more features associated with the one or more advertisements displayed in response to the one or more queries are identified. One or more functions are generated for predicting a frequency with which a given advertisement displayed in response to a query will be selected using the analytics data and features associated with the one or more advertisements displayed in response to the one or more queries.

Claims

1. A method for generating one or more functions for predicting a frequency with which an advertisement displayed in response to a query will be selected, the method comprising:

receiving analytics data associated with a display of one or more advertisements in response to one or more queries;

identifying one or more features associated with the one or more advertisements displayed in response to the one or more queries; and

generating one or more functions for predicting a frequency with which a given advertisement displayed in response to a query will be selected using the analytics data and features associated with the one or more advertisements displayed in response to the one or more queries.

2. The method of claim 1 wherein the receiving analytics data comprises receiving analytics data indicating the frequency with which a given advertisement was selected when displayed in response to a given query.

3. The method of claim 1 wherein receiving analytics data comprises receiving analytics data indicating the rank at which a given advertisement was displayed in response to a given query.

4. The method of claim 1 wherein identifying one or more features comprises identifying one or more query-independent features for a given advertisement.

5. The method of claim 4 wherein a query-independent feature comprises a characteristic associated with a given advertisement.

6. The method of claim 5 wherein a query-dependent feature comprises a geographic location associated with a given advertisement displayed in response to a given query.

7. The method of claim 5 wherein a query-independent feature comprises information associated with content corresponding to a given advertisement.

8. The method of claim 7 wherein content comprises a landing page.

9. The method of claim 1 wherein identifying one or more features comprises identifying one or more query-dependent features for a given advertisement displayed in response to a given query.

10. The method of claim 9 wherein a query-dependent feature comprises a characteristic associated with a given advertisement and query.

11. The method of claim 10 wherein a query-dependent feature comprises a time a given advertisement was displayed in response to a given query.

12. The method of claim 10 wherein a query-dependent feature comprises a type of match with which a given advertisement was retrieved in response to a given query.

13. The method of claim 10 wherein a query-dependent feature comprises information associated with a user that generated a given query or selected an advertisement displayed in response to the query.

14. The method of claim 13 wherein information associated with a user comprises an Internet Protocol (“

IP”

) address of a user.

15. The method of claim 13 wherein information associated with a user comprises a frequency with which a user selected one or more advertisements during a given time period.

16. The method of claim 13 wherein information associated with a user comprises information identifying one or more preferences associated with a user.

17. The method of claim 13 wherein information associated with a user comprises a geographic location of a given user.

18. The method of claim 1 wherein generating one or more functions comprises:

calculating an actual clickability score for the one or more advertisements displayed in response to the one or more queries; and

generating one or more functions through the use of machine learning using the actual clickability scores and the features associated with the one or more advertisements displayed in response to the one or more queries.

19. The method of claim 18 wherein calculating an actual clickability score comprises calculating an actual clickability score using the analytics data associated with a given advertisement.

20. The method of claim 18 wherein an actual clickability score for a given advertisement comprises a quotient of a normalized click through rate of a given advertisement displayed in response to a given query and an average normalized click through rate of one or more advertisements displayed in response to the query.

21. The method of claim 18 wherein calculating an actual clickability score for the one or more advertisements displayed in response to the one or more queries comprises calculating an actual clickability score using information from a human editor.

22. The method of claim 18 wherein generating one or more functions through the use of machine learning comprises generating one or more functions using one or more boosted decision trees constructed from the actual clickability scores and features associated with the one or more advertisements displayed in response to the one or more queries.

23. The method of claim 18 wherein generating one or more functions through the use of machine learning comprises generating one or more functions using one or more neural networks constructed from the actual clickability scores and features associated with the one or more advertisements displayed in response to the one or more queries.

24. The method of claim 18 wherein generating one or more functions through the use of machine learning comprises generating one or more functions using a linear regression analysis of the actual clickability scores and features associated with the one or more advertisements displayed in response to the one or more queries.

25. The method of claim 18 wherein generating one or more functions through the use of machine learning using the actual clickability scores and the features associated with the one or more advertisements displayed in response to the one or more queries comprises:

identifying one or more advertisements associated with a given classification;

generating one or more functions through the use of machine learning using one or more features common to the one or more advertisements associated with the classification.

26. The method of claim 18 wherein generating one or more functions through the use of machine learning using the actual clickability scores and the features associated with the one or more advertisements displayed in response to the one or more queries comprises:

performing a comparison of two or more advertisements displayed in response to a given query;

identifying one or more features unique to a given advertisement based upon the comparison; and

generating one or more functions through the use of machine learning using the one or more features identified as unique.

27. The method of claim 18 wherein generating one or more functions through the use of machine learning comprises generating one or more functions using information for one or more advertisers associated with the one or more advertisements.

28. The method of claim 18 wherein generating one or more functions through the use of machine learning comprises generating one or more functions using information associated with one or more advertisements with which a given advertisement is displayed.

29. The method of claim 1 comprising generating an aggregate clickability score for the one or more advertisements.

30. A system for generating one or more functions for predicting a frequency with which an advertisement displayed in response to a query will be selected, the system comprising:

an analytics data store operative to store analytics data associated with one or more advertisements displayed in response to one or more queries;

a feature component operative to identify features of the one or more advertisements displayed in response to the one or more queries; and

a clickability engine operative to generate one or more functions for predicting a frequency with which a given advertisement displayed in response to a query will be selected using the analytics data and features associated with the one or more advertisements displayed in response to the one or more queries.

31. The system of claim 30 wherein the analytics data store is operative to store analytics data indicating a frequency with which an advertisement was selected in response to a given query.

32. The system of claim 30 wherein the analytics data store is operative to store analytics data indicating a rank at which an advertisement was displayed in a ranked list of advertisements in response to a given query.

33. The system of claim 30 wherein the feature component is operative to identify one or more query-independent features of a given advertisement displayed in response to a given query.

34. The system of claim 33 wherein a query-independent feature comprises a characteristic of a given advertisement.

35. The system of claim 34 wherein a query-dependent feature comprises a geographic location associated with a given advertisement displayed in response to a given query.

36. The system of claim 34 wherein a query-independent feature comprises information associated with content corresponding to a given advertisement.

37. The system of claim 36 wherein content comprises a landing page.

38. The system of claim 30 wherein the feature component is operative to identify one or more query-dependent features of a given advertisement displayed in response to a given query.

39. The system of claim 38 wherein a query-dependent feature comprises a characteristic of an advertisement and a given query.

40. The system of claim 39 wherein a query-dependent feature comprises a time a given advertisement was displayed in response to a given query.

41. The system of claim 39 wherein a query-dependent feature comprises a type of match with which a given advertisement was retrieved in response to a given query.

42. The system of claim 39 wherein a query-dependent feature comprises a geographic location associated with a given advertisement displayed in response to a given query.

43. The system of claim 39 wherein a query-dependent feature comprises information associated with a user that generated a given query or selected an advertisement displayed in response to the query.

44. The system of claim 43 wherein information associated with a user comprises an Internet Protocol (“

IP”

) address of a user.

45. The system of claim 43 wherein information associated with a user comprises a frequency with which a user selected one or more advertisements during a given time period.

46. The system of claim 43 wherein information associated with a user comprises information identifying one or more preferences associated with a user.

47. The system of claim 43 wherein information associated with a user comprises a geographic location of a given user.

48. The system of claim 30 wherein the clickability engine is operative to:

calculate an actual clickability score for the one or more advertisements displayed in response to the one or more queries; and

generate one or more functions through the use of machine learning for predicting a frequency with which a given advertisement displayed in response to a query will be selected using the actual clickability scores and the features associated with the one or more advertisements displayed in response to the one or more queries.

49. The system of claim 48 wherein the clickability engine is operative to calculate an actual clickability score for the one or more advertisements using the analytics data associated with the one or more advertisements.

50. The system of claim 48 wherein the clickability engine is operative to calculate an actual clickability score for the one or more advertisements using information from a human editor.

51. The system of claim 48 wherein the clickability engine is operative to:

construct one or more boosted decision trees from the actual clickability scores and features associated with the one or more advertisements displayed in response to the one or more queries; and

generate one or more functions through the use of machine learning using the one or more boosted decision trees.

52. The system of claim 48 wherein the clickability engine is operative to:

construct one or more neural networks from the actual clickability scores and features associated with the one or more advertisements displayed in response to the one or more queries; and

generate one or more functions through the use of machine learning using the one or more neural networks.

53. The system of claim 48 wherein the clickability engine is operative to generate one or more functions through the use of machine learning using a linear regression analysis of the actual clickability scores and features associated with the one or more advertisements displayed in response to the one or more queries.

54. The system of claim 48 wherein the clickability engine is operative to generate one or more functions through the use of machine learning using information for one or more advertisers associated with one or more advertisements displayed in response to one or more queries.

55. The system of claim 48 wherein the clickability engine is operative to generate one or more functions through the use of machine learning using information associated with one or more advertisements with which a given advertisement is displayed in response to given query.

56. The system of claim 30 wherein the clickability engine is operative to generate an aggregate clickability score for the one or more advertisements.

57. A method for generating one or more functions for predicting a frequency with which an advertisement displayed in response to a query will be selected, the method comprising:

receiving analytics data associated with a display of one or more advertisements in response to one or more queries;

identifying one or more features associated with the one or more advertisements displayed in response to the one or more queries;

generating one or more functions for predicting a frequency with which a given advertisement displayed in response to a query will be selected using the analytics data and features associated with the one or more advertisements displayed in response to the one or more queries;

generating an aggregate clickability score for the one or more advertisements comprising;

generating a historical clickability score for the one or more advertisements using the analytics data associated with the one or more advertisements,

generating a predicted clickability score for the one or more advertisements using the one or more functions for predicting a frequency with which a given advertisement displayed in response to a query will be selected; and

combining the historical clickability score and the predicted clickability score.

58. The method of claim 57 wherein generating a historical clickability score comprises:

calculating one or more click through rates for one or more levels of an advertisement organizational hierarchy to which an advertisement belongs;

assigning a confidence score to the one or more calculated click through rates; and

combining the one or more click through rates and corresponding confidence scores to generate a historical clickability score.

59. The method of claim 57 wherein generating one or more functions comprises generating one or more functions through the use of machine learning using the historical clickability scores associated with the one or more advertisements.

60. A system for generating one or more functions for predicting a frequency with which an advertisement displayed in response to a query will be selected, the system comprising:

an analytics data store operative to store analytics data associated with one or more advertisements displayed in response to one or more queries;

a feature component operative to identify features of the one or more advertisements displayed in response to the one or more queries; and

a clickability engine operative to;

generate one or more functions for predicting a frequency with which a given advertisement displayed in response to a query will be selected using the analytics data and features associated with the one or more advertisements displayed in response to the one or more queries;

generate an aggregate clickability score for the one or more advertisements;

generate a historical clickability score for the one or more advertisements using the analytics data associated with the one or more advertisements;

generate a predicted clickability score for the one or more advertisements using the one or more functions for predicting a frequency with which a given advertisement displayed in response to a query will be selected; and

combine the historical clickability score and the predicted clickability score to generate an aggregate clickability score.

61. The system of claim 60 wherein the clickability engine is operative to:

calculate one or more click through rates for one or more levels of an advertisement organizational hierarchy to which an advertisement belongs;

assign a confidence score to the one or more calculated click though rates; and

generate a historical clickability score using the confidence score and click through rates associated with the one or more levels of the advertisement organizational hierarchy to which the advertisement belongs.

62. The system of claim 60 wherein the clickability engine is operative to generate one or more functions through the use of machine learning using the historical clickability scores associated with the one or more advertisements.