Mixing items, such as ad targeting keyword suggestions, from heterogeneous sources

• US 7,801,899 B1
• Filed: 10/01/2004
• Issued: 09/21/2010
• Est. Priority Date: 10/01/2004
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Spitkovsky, Valentin
• Primary Examiner(s): Cottingham; John R.
• Assistant Examiner(s): Pham; Michael
• Application Number:
  US10/957,337
• Time in Patent Office: 2,181 Days
• Field of Search: 707/3, 709/219, 709/203, 705/14
• US Class Current: 707/748
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0241 Advertisements
• CPC Class Codes: G06Q 30/0243 Comparative campaigns

Abstract

The way in which targeting keywords are suggested to advertisers, particularly in instances where different suggestion tools or techniques provide a number of different sets of targeting keyword suggestions, may be improved by (a) accepting at least two heterogeneous sets of suggestions (where each of the sets of suggestions includes suggestions that are ranked and/or scored), (b) determining, for each of the sets of suggestions, a new score for each of the suggestions of the set, where the new score combines an cardinal aspect of the suggestion and an ordinal aspect of the suggestion, and (c) combining at least some of the suggestions from each of the sets using the new scores to generate a final set of ordered and/or scored suggestions. Each of the suggestions may include information for targeting the serving of an online advertisement. For example, each of the suggestions may include a keyword for targeting the serving of an online advertisement. Each of the suggestions may include a keyword type.

Claims

1. A machine-implemented method comprising:

receiving a request for keyword suggestions, the request including a seed keyword with which the keyword suggestions are to be generated;

accepting, from two or more keyword suggestion tools, at least two heterogeneous sets of keyword suggestions for an online advertisement, wherein each set of keyword suggestions includes targeting keyword suggestions that are ranked and scored by a keyword suggestion tool that suggested the set of keyword suggestions, and wherein targeting keyword suggestions in each set of targeting keyword suggestions have been generated based on the seed keyword;

for each heterogeneous sets of keyword suggestions accepted from the at least two or more keyword suggestion tools, determining, by one or more processors, a new normalized score for each of the targeting keyword suggestions in the heterogeneous set of keyword suggestions, wherein the new normalized score is computed based on a cardinal aspect of the targeting keyword suggestion and an ordinal aspect of the targeting keyword suggestion in the heterogeneous set of keyword suggestions, the cardinal aspect representing an absolute score corresponding to the targeting keyword suggestion and the ordinal aspect representing a rank of the targeting keyword suggestion in the heterogeneous set of keyword suggestions, and wherein the new normalized score for each targeting keyword suggestion in a particular set of heterogeneous keyword suggestions is defined as a sum of a first weight multiplied by the cardinal aspect and a second weight multiplied by the ordinal aspect;

generating, by the one or more processors, an adjusted new score for each targeting keyword suggestion based on a result of a function of a new normalized score corresponding to the targeting keyword suggestion and trust factor of a keyword suggestion tool from which the targeting keyword was accepted, the trust factor representing a measure of reliability of the keyword suggestion tool;

combining, by the one or more processors, the targeting keyword suggestions scored by a first keyword suggestion tool selected from the at least two or more keyword suggestion tools and the targeting keyword suggestions scored by a second suggestion tool selected from the at least two or more keyword suggestion tools using the new scores to generate a combined set of ordered and scored suggestions according to the adjusted new score for each targeting keyword suggestion; and

providing the combined set of keyword suggestions to a user device.

2. The machine-implemented method of claim 1 wherein the ordinal aspect of a ranked targeting keyword suggestion is defined as:

3. The machine-implemented method of claim 1 wherein the cardinal aspect of a scored targeting keyword suggestion is defined as a score of the targeting keyword suggestion divided by a sum of scores of all targeting keyword suggestions of the heterogeneous set of keyword suggestions.

4. The machine-implemented method of claim 1 wherein a sum of the first and second weights is one.

5. The machine-implemented method of claim 4 wherein the first weight is equal to the second weight.

6. The machine-implemented method of claim 1 wherein each of the targeting keyword suggestions includes information for targeting the servicing of an online advertisement.

7. The machine-implemented method of claim 1 wherein each of the targeting keyword suggestions includes a keyword type.

8. The machine-implemented method of claim 7 wherein the keyword type is selected from a set consisting of (A) refinement, (B) broadening, (C) expansion, (D) negative.

9. The machine-implemented method of claim 7 further comprising removing targeting keyword suggestions belonging to an unwanted keyword type.

10. The machine-implemented method of claim 1 further comprising:

removing targeting keyword suggestions already targeted.

11. The machine-implemented method of claim 1 further comprising:

removing targeting keyword suggestions that are trademarks.

12. The machine-implemented method of claim 1 further comprising:

removing targeting keyword suggestions determined to be bad words.

13. The machine-implemented method of claim 1 further comprising:

for each of the at least two heterogeneous sets of keyword suggestions, and for each new normalized score for each of the keyword suggestions of the heterogeneous set of keyword suggestions, adjusting the new normalized score to generate an adjusted new score so that a sum of the adjusted new scores for each of the sets equals the same value.

14. The machine-implemented method of claim 1 further comprising:

for each of the at least two sets of keyword suggestions, and for each new normalized score for each of the keyword suggestions of the heterogeneous set of keyword suggestions, adjusting the new normalized score to generate an adjusted new score so that a sum of the adjusted new score for each of the sets equals one.

15. The machine-implemented method of claim 14 wherein the act of adjusting the new normalized score includes multiplying the new normalized score by:

16. The machine-implemented method of claim 1 wherein each of the targeting keyword suggestions includes a keyword used to target the serving of an online advertisement and a language.

17. The machine-implemented method of claim 16 further comprising:

removing targeting keyword suggestions having an unwanted language.

18. The method of claim 1, further comprising:

receiving, for a keyword suggestion tool, performance feedback that represents a measure of keyword suggestion tool performance; and

adjusting the trust factor of the keyword suggestion tool based on the performance feedback.