Computer-based evaluation tool for selecting personalized content for users

• US 9,703,877 B2
• Filed: 10/23/2015
• Issued: 07/11/2017
• Est. Priority Date: 01/19/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: LinkedIn Corporation (Microsoft Corporation)
• Inventors: Thompson, Dean Sherwood, Mawhinney, David S., DiBiase, Evan S., Fleckenstein, Matthew J., Ammirati, Sean J., Avrahami, Thi T.
• Primary Examiner(s): Bhargava, Anil
• Application Number:
  US14/921,654
• Publication Number:
  US 20160042083A1
• Time in Patent Office: 627 Days
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9577 Optimising the visualization of content, e.g. distillation of HTML documents

Abstract

The invention relates to a method and system for selecting personalized content for a user, the method being performed by an evaluation tool instantiated on a computing device and comprising the evaluation tool. The evaluation tool creates a content selection rule for the user for finding and filtering content items, such as advertising content. The tool generates a content selection algorithm from the content selection rule for determining which content items to present to the user and presents the content item to the user based on the content selection algorithm and allows the user to interact with the presented content item.

Claims

1. A method, comprising:

creating a plurality of content selection rules for a user to find content items;

generating a content selection algorithm based on the plurality of content selection rules to determine which content items to present to the user, the generating of the content selection algorithm including assigning an initial weight to each content selection rule of the plurality of content selection rules, each respective initial weight determining a contribution of the corresponding content selection rule in determining which content items to present to the user;

identifying multiple content items using the content selection algorithm;

filtering the identified multiple content items using one or more of the plurality of content selection rules associated with the user;

selecting one or more content items using the content selection algorithm based on the filtering of the identified content items;

presenting the one or more content items to the user;

monitoring user interactions with the one or more content items presented to the user; and

modifying the content selection algorithm according to a fitness function generated from one or more user interactions with the one or more content items presented to the user, the modifying of the content selection algorithm further including at least one of;

adding one or more new content selection rules to the content selection algorithm based on fitness data from the fitness function; and

removing one or more of the plurality of content selection rules from the content selection algorithm based on fitness data from the fitness function.

2. The method of claim 1, wherein the modifying of the content selection algorithm includes assigning a new weight to at least one content selection rule of the plurality of content selection rules.

3. The method of claim 1, further comprising:

learning the one or more new content selection rules that should be added to the content selection algorithm based on the one or more user interactions with the one or more content items presented to the user;

determining the one or more selection rules to be removed from the content selection algorithm based on the one or more user interactions with the one or more content items presented to the user; and

adjusting parameters that influence how the selection rules are weighted in the content selection algorithm based on the one or more user interactions with the one or more content items presented to the user.

4. The method of claim 2, further comprising selecting an additional content item using the modified content selection algorithm.

5. The method of claim 1, wherein filtering the identified content items includes:

scoring each of the identified multiple content items based on one or more of the content selection rules; and

sorting the identified multiple content items into a list based on a score assigned to each of the identified multiple content items;

wherein accessing a content item using the content selection algorithm includes accessing the identified multiple content items at a top of the list.

6. The method of claim 1, wherein the user is part of a group of users, the plurality of selection rules are created for the group, the content selection algorithm is generated for the group, and interaction of the group with respect to the first content item is monitored to determine the modification of the content selection algorithm based on interaction by the group.

7. The method of claim 6, wherein the content selection algorithm of at least one user is influenced by the content selection algorithm generated for the group.

8. The method of claim 1, further comprising adjusting the at least one content selection rule based on selection rules of other users.

9. The method of claim 1, wherein a plurality of content selection algorithms are generated for the user based on the content selection rules.

10. The method of claim 1, wherein the first content item includes advertising content.

11. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

creating a plurality of content selection rules for a user to find content items;

generating a content selection algorithm based on the plurality of content selection rules to determine which content items to present to the user, the generating of the content selection algorithm including assigning an initial weight to each content selection rule of the plurality of content selection rules, each respective initial weight determining a contribution of the corresponding content selection rule in determining which content items to present to the user;

identifying multiple content items using the content selection algorithm;

filtering the identified content items using one or more of the plurality of content selection rules associated with the user;

selecting one or more content items using the content selection algorithm based on the filtering of the identified content items;

presenting the one or more content items to the user;

monitoring user interactions with the one or more content items presented to the user; and

modifying the content selection algorithm according to a fitness function generated from one or more user interactions with the one or more content items presented to the user, the modifying of the content selection algorithm further including at least one of;

adding one or more new content selection rules to the content selection algorithm based on fitness data from the fitness function; and

removing one or more of the plurality of content selection rules from the content selection algorithm based on the fitness data from the fitness function.

12. The non-transitory machine-readable storage medium of claim 11, wherein the modifying of the content selection algorithm includes assigning a new weight to at least one content selection rule of the plurality of content selection rules.

13. The non-transitory machine-readable storage medium of claim 11, further comprising:

learning the one or more new content selection rules that should be added to the content selection algorithm based on the one or more user interactions with the one or more content items presented to the user;

determining the one or more content selection rules to be removed from the content selection algorithm based on the one or more user interactions with the one or more content items presented to the user; and

adjusting parameters that influence how the content selection rules are weighted in the content selection algorithm based on the one or more user interactions with the one or more content items presented to the user.

14. The non-transitory machine-readable storage medium of claim 12, wherein the operations further comprise selecting an additional content item using the modified content selection algorithm.

16. The non-transitory machine-readable storage medium of claim 11, wherein the user is part of a group of users, the plurality of content selection rules are created for the group, the content selection algorithm is generated for the group, and interaction of the group with respect to the first content item is monitored to determine modification of the content selection algorithm based on interaction by the group.

17. The non-transitory machine-readable storage medium of claim 16, wherein the content selection algorithm of at least one user is influenced by the content selection algorithm generated for the group.

18. The non-transitory machine-readable storage medium of claim 11, wherein the operations further comprise adjusting the at least one content selection rule based on selection rules of other users.

19. The non-transitory machine-readable storage medium of claim 11, wherein the content selection algorithm identifies the multiple content items from a pre-selected stream of candidate items.

15. The non-transitory machine-readable storage medium of claim wherein filtering the identified content items includes:

scoring each of the identified multiple content items based on one or more of the content selection rules; and

sorting the identified multiple content items into a list based on a score assigned to each of the identified multiple content items;

wherein accessing a content item using the content selection algorithm includes accessing the identified multiple content items at a top of the list.

20. A system comprising:

at least one processor of a machine; and

a machine-readable storage medium storing instructions that configure the at least one processor of the machine to perform operations comprising;

creating a plurality of content selection rules for a user to find content items;

generating a content selection algorithm based on the plurality of content selection rules to determine which content items to present to the user, the generating of the content selection algorithm including assigning an initial weight to each content selection rule of the plurality of content selection rules, each respective initial weight determining a contribution of the corresponding content selection rule in determining which content items to present to the user;

identifying multiple content items using the content selection algorithm;

filtering the identified content items using one or more of the plurality of content selection rules associated with the user;

selecting one or more content items using the content selection algorithm based on the filtering of the identified content items;

presenting the one or more content items to the user;

monitoring user interactions with the one or more content items presented to the user; and

modifying the content selection algorithm according to a fitness function generated from one or more user interactions with the one or more content items presented to the user, the modifying of the content selection algorithm further including at least one of;

adding one or more new content selection rules to the content selection algorithm based on fitness data from the fitness function; and

removing one or more of the plurality of content selection rules from the content selection algorithm based on the fitness data from the fitness function.