Using match confidence to adjust a performance threshold

• US 7,346,615 B2
• Filed: 11/14/2003
• Issued: 03/18/2008
• Est. Priority Date: 10/09/2003
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Bem, Jeremy
• Primary Examiner(s): Vo; Tim
• Assistant Examiner(s): GORTAYO, DANGELINO N
• Application Number:
  US10/713,964
• Publication Number:
  US 20050080772A1
• Time in Patent Office: 1,586 Days
• Field of Search: 707/3, 707/5, 707/104, 707/1, 707/4, 707/7, 707/104.1, 709/217
• US Class Current: 1/1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99937 Sorting
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

If some aspect of serving or scoring an ad is subject to a performance (e.g., click-through rate, etc.) threshold, such a threshold may be adjusted using a confidence factor of the ad targeting used. For example, ads served pursuant to a more relaxed notion of match might have to meet a higher performance threshold (e.g., than the threshold applied to ads served pursuant to a stricter notion of match). Alternatively, or in addition, ads served pursuant to a stricter notion of match might be subject to a lower performance threshold (e.g., than the threshold applied to ads served pursuant to a more relaxed notion of match). Thus, in general, a performance threshold could increase as match confidence decreases, and/or a performance threshold could decrease as match confidence increases.

Claims

1. A machine-implemented method comprising:

a) accepting search query information including a word;

b) determining one or more words related to the word included in the accepted search query;

c) generating an item request including

i) the word included in the accepted search query, and

ii) the one or more words determined to be related to the word included in the accepted search query;

d) retrieving items using the item request, each of the items having an associated performance value which is a function of at least one of (A) a selection rate associated with the item and (B) a conversion rate associated with the item;

e) applying the performance value of each of the retrieved items to a minimum performance threshold to generate a set of filtered items, wherein the performance value of any item retrieved on the basis of the one or more words determined to be related to the word included in the accepted search query is applied to a higher minimum performance threshold than the minimum performance threshold used for the performance value of any item retrieved on the basis of the word included in the accepted search query; and

f) sending at least some of the filtered items towards a client device for rendering.

2. The machine-implemented method of claim 1 wherein the act of applying the performance value of each of the retrieved items to a minimum performance threshold includes adjusting the minimum performance threshold for any items retrieved on the basis of the one or more words determined to be related to the word included in the accepted search query.

3. The machine-implemented method of claim 2 wherein the act of adjusting the minimum performance threshold includes multiplying a first performance threshold by a multiplier that is greater than one.

4. The machine-implemented method of claim 3 further comprising:

g) updating the multiplier using performance information.

5. The machine-implemented method of claim 4 wherein the items are ads.

6. The machine-implemented method of claim 5 wherein the performance information includes ad selection information.

7. The machine-implemented method of claim 5 wherein the performance information includes ad conversion information.

8. A machine-implemented method comprising:

a) accepting search query information;

b) determining relevant items using the search query information and item targeting information, wherein the act of determining each relevant item includes associating the items with a confidence measure, and wherein each of the items has an associated performance value which is a function of at least one of (A) a selection rate associated with the item and (B) a conversion rate associated with the item;

c) applying the performance value of each of the relevant items to a minimum performance threshold to generate a set of filtered items, wherein for a given item, the minimum performance threshold is a function of the confidence measure associated with the determination of relevance of the item; and

d) sending at least some of the filtered items towards a client device for rendering.

9. The machine-implemented method of claim 8 wherein the minimum performance threshold increases as the confidence measure decreases.

10. The machine-implemented method of claim 8 wherein the minimum performance threshold decreases as the confidence measure increases.

11. The machine-implemented method of claim 9 wherein the item is an ad and wherein the minimum performance threshold is a minimum ad selection rate threshold value.

12. The machine-implemented method of claim 10 wherein the item is an ad and wherein the minimum performance threshold is a minimum ad selection rate threshold value.

13. The machine-implemented method of claim 9 wherein the item is an ad and wherein the minimum performance threshold is a minimum ad conversion rate threshold value.

14. The machine-implemented method of claim 10 wherein the item is an ad and wherein the minimum performance threshold is a minimum ad conversion rate threshold value.

15. The machine-implemented method of claim 8 wherein the act of determining relevant items uses the search query information, item targeting information, and at least one match function, and wherein the confidence measure is determined based on the at least one match function used in the act of determining relevant items.

16. The machine-implemented method of claim 15 wherein the minimum performance threshold increases as the confidence measure decreases.

17. The machine-implemented method of claim 15 wherein the minimum performance threshold decreases as the confidence measure increases.

18. The machine-implemented method of claim 16 wherein the item is an ad and wherein the minimum performance threshold is a minimum ad selection rate threshold value.

19. The machine-implemented method of claim 17 wherein the item is an ad and wherein the minimum performance threshold is a minimum ad selection rate threshold value.

20. The machine-implemented method of claim 16 wherein the item is an ad and wherein the minimum performance threshold is a minimum ad conversion rate threshold value.

21. The machine-implemented method of claim 17 wherein the item is an ad and wherein the minimum performance threshold is a minimum ad conversion rate threshold value.

22. Apparatus comprising:

a) an input for accepting search query information including a word;

b) means for determining one or more words related to the word included in the accepted search query;

c) means for generating an item request including

i) the word included in the accepted search query, and

ii) the one or more words determined to be related to the word included in the accepted search query;

d) means for retrieving items using the item request, each of the items having an associated performance value which is a function of at least one of (A) a selection rate associated with the item and (B) a conversion rate associated with the item;

e) means for applying the performance value of each of the retrieved items to a minimum performance threshold to generate a set of filtered items, wherein the performance value of any item retrieved on the basis of the one or more words determined to be related to the word included in the accepted search query is applied to a higher minimum performance threshold than the minimum performance threshold used for the performance value of any item retrieved on the basis of the word included in the accepted search query; and

f) means for sending at least some of the filtered items towards a client device for rendering.

23. The apparatus of claim 22 wherein the means for applying the performance value of each of the retrieved items to a minimum performance threshold include means for adjusting the minimum performance threshold for any items retrieved on the basis of the one or more words determined to be related to the word included in the accepted search query.

24. The apparatus of claim 23 wherein the means for adjusting the minimum performance threshold include means for multiplying a first performance threshold by a multiplier that is greater than one.

25. The apparatus of claim 24 further comprising:

g) means for updating the multiplier using performance information.

26. The apparatus of claim 25 wherein the items are ads.

27. The apparatus of claim 26 wherein the performance information includes ad selection information.

28. The apparatus of claim 26 wherein the performance information includes ad conversion information.

29. Apparatus comprising:

a) an input for accepting search query information;

b) means for determining relevant items using the search query information and item targeting information and for associating the items with a confidence measure, wherein each of the items has an associated performance value which is a function of at least one of (A) a selection rate associated with the item and (B) a conversion rate associated with the item;

c) means for applying the performance value of each of the relevant items to a minimum performance threshold to generate a set of filtered items, wherein for a given item, the minimum performance threshold is a function of the confidence measure associated with the determination of relevance of the item; and

d) means for sending at least some of the filtered items towards a client device for rendering.

30. The apparatus of claim 29 wherein the minimum performance threshold increases as the confidence measure decreases.

31. The apparatus of claim 29 wherein the minimum performance threshold decreases as the confidence measure increases.

32. The apparatus of claim 30 wherein the item is an ad and wherein the minimum performance threshold is a minimum ad selection rate threshold value.

33. The apparatus of claim 31 wherein the item is an ad and wherein the minimum performance threshold is a minimum ad selection rate threshold value.

34. The apparatus of claim 30 wherein the item is an ad and wherein the minimum performance threshold is a minimum ad conversion rate threshold value.

35. The apparatus of claim 31 wherein the item is an ad and wherein the minimum performance threshold is a minimum ad conversion rate threshold value.

36. The apparatus of claim 29 wherein the means for determining relevant items use the search query information, item targeting information, and at least one match function, and wherein the confidence measure is determined based on the at least one match function used by the means for determining relevant items.

37. The apparatus of claim 36 wherein the minimum performance threshold increases as the confidence measure decreases.

38. The apparatus of claim 36 wherein the minimum performance threshold decreases as the confidence measure increases.

39. The apparatus of claim 37 wherein the item is an ad and wherein the minimum performance threshold is a minimum ad selection rate threshold value.

40. The apparatus of claim 38 wherein the item is an ad and wherein the minimum performance threshold is a minimum ad selection rate threshold value.

41. The apparatus of claim 37 wherein the item is an ad and wherein the minimum performance threshold is a minimum ad conversion rate threshold value.

42. The apparatus of claim 38 wherein the item is an ad and wherein the minimum performance threshold is a minimum ad conversion rate threshold value.