Real-time content integration based on machine learned selections

• US 10,581,953 B1
• Filed: 05/31/2017
• Issued: 03/03/2020
• Est. Priority Date: 05/31/2017
• Status: Active Grant

• Current Assignee: Snap, Inc.
• Original Assignee: Snap, Inc.
• Inventors: Brewer, Jason, Farnham, Rodrigo B., Lue, David B., Stucky-Mack, Nicholas J.
• Primary Examiner(s): Tiv, Backhean
• Application Number:
  US15/610,301
• Time in Patent Office: 1,007 Days
• Field of Search: None
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/20 Ensemble learning
• CPC Class Codes: G06N 5/01 Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/14 Session management for real-time applications in data packet communications networks H04L65/1066
• CPC Class Codes: H04L 67/306 User profiles
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

A content integration system is configured to rapidly select online content for distribution in response to a user-generated request. The content integration system can analyze available online content items and data describing the user to generate one or more numerical likelihoods estimating how the user will interact with each of the given online content items. The highest scoring content can be selected and transmitted to the user without a noticeable delay.

Claims

1. A method comprising:

receiving, over a network from a client device of a user, a request for online content, the request generated from an active network session of an application executing on the client device of the user;

in response to the request, identifying, in a database, a plurality of online content items submitted with a plurality of bid values, the plurality of online content items submitted over the network by a plurality of additional client devices;

automatically generating, using a machine learning classifier, a plurality of negative relevancy values for the plurality of online content items, the plurality of negative relevancy values indicating the plurality of online content items are likely to be skipped by the user in the active network session, each given negative relevancy numerical value of a given online content item comprising a select value for the given online content item and a skip value for the given online content item, the select value being a numerical value that describes a likelihood that the user of the active network session will use a first user input action to select the given online content item, the skip value being an additional numerical value that describes an additional likelihood that the user of the active network session will use a second user input action to bypass the given online content item, the second user input action being different than the first user input action, the skip value being higher than the corresponding select value for each of the plurality of negative relevancy values;

generating, for each of the plurality of content items, a combined value by lowering a bid value from the plurality of bid values with a corresponding negative relevancy value from the plurality of negative relevancy values;

automatically selecting an online content item from the plurality of online content items based on the online content item having a highest combined value; and

causing, on the client device of the user, a presentation of the online content item during the active network session.

2. The method of claim 1, wherein the request is a request for aggregated content comprising a placeholder space and one or more pre-selected content items, and wherein the selected online content item is integrated into the placeholder space among the one or more pre-selected content items, the pre-selected content items being selected before the client device generates the request for the aggregated content.

3. The method of claim 1, wherein the request for online content is generated in response to the user navigating, within the application, to a page configured to receive one or more of the plurality of online content items.

4. The method of claim 1, wherein the first user input action and the second user input action are gestures received through the client device, and the client device is configured to distinguish between the first user input action and the second user input action.

5. The method of claim 1, further comprising:

identifying historical user data that describes past user actions of past users using the application; and

training the machine learning classifier on the historical user data.

6. The method of claim 5, wherein the past user actions include browse path data, subscription data, and user profile data.

7. The method of claim 6, wherein the browse path data describes a sequence of content items displayed to a past given user as the user navigates through the content items using the application, wherein the subscription data specifies whether a past given user has subscribed to a series of content items, and wherein the user profile data describes user preferences of types of content.

8. The method of claim 5, wherein the machine learning classifier implements a random forest scheme.

9. A system comprising:

one or more processors of a machine; and

a memory storing instructions that, when executed by the one or more processors, cause the machine to perform operations comprising;

receive, over a network from a client device of a user, a request for online content, the request generated from an active network session of an application executing on the client device of the user;

in response to the request, identify, in a database, a plurality of online content items submitted with a plurality of bid values, the plurality of online content items submitted over the network by a plurality of additional client devices;

automatically generate, using a machine learning classifier, a plurality of negative relevancy values for the plurality of online content items, the plurality of negative relevancy values indicating the plurality of online content items are likely to be skipped by the user in the active network session, each given negative relevancy numerical value of a given online content item comprising a select value for the given online content item and a skip value for the given online content item, the select value being a numerical value that describes a likelihood that the user of the active network session will use a first user input action to select the given online content item, the skip value being an additional numerical value that describes an additional likelihood that the user of the active network session will use a second user input action to bypass the given online content item, the second user input action being different than the first user input action, the skip value being higher than the corresponding select value for each of the plurality of negative relevancy values;

generate, for each of the plurality of content items, a combined value by lowering a bid value from the plurality of bid values with a corresponding negative relevancy value from the plurality of negative relevancy values;

automatically select an online content item from the plurality of online content items based on the online content item having a highest combined value; and

cause, on the client device of the user, a presentation of the online content item during the active network session.

10. The system of claim 9, wherein the request is a request for aggregated content comprising a placeholder space and one or more pre-selected content items, and wherein the selected online content item is integrated into the placeholder space among the one or more pre-selected content items, the pre-selected content items being selected before the client device generates the request for the aggregated content.

11. The system of claim 9, wherein the request for online content is generated in response to the user navigating, within the application, to a page configured to receive one or more of the plurality of online content items.

12. The system of claim 9, the operations further comprising:

identify historical user data that describes past user actions of past users using the application, wherein the past user actions include browse path data, subscription data, and user profile data; and

train the machine learning classifier on the historical user data.

13. The system of claim 12, wherein the browse path data describes a sequence of content items displayed to a past given user as the user navigates through the content items using the application, wherein the subscription data specifies whether a past given user has subscribed to a series of content items, and wherein the user profile data describes user preferences of types of content.

14. A non-transitory machine-readable storage device embodying instructions that, when executed by a machine, cause the machine to perform operations comprising:

receive, over a network from a client device of a user, a request for online content, the request generated from an active network session of an application executing on the client device of the user;

in response to the request, identify, in a database, a plurality of online content items submitted with a plurality of bid values, the plurality of online content items submitted over the network by a plurality of additional client devices;

automatically generate, using a machine learning classifier, a plurality of negative relevancy values for the plurality of online content items, the plurality of negative relevancy values indicating the plurality of online content items are likely to be skipped by the user in the active network session, each given negative relevancy numerical value of a given online content item comprising a select value for the given online content item and a skip value for the given online content item, the select value being a numerical value that describes a likelihood that the user of the active network session will use a first user input action to select the given online content item, the skip value being an additional numerical value that describes an additional likelihood that the user of the active network session will use a second user input action to bypass the given online content item, the second user input action being different than the first user input action, the skip value being higher than the corresponding select value for each of the plurality of negative relevancy values;

generate, for each of the plurality of content items, a combined value by lowering a bid value from the plurality of bid values with a corresponding negative relevancy value from the plurality of negative relevancy values;

automatically select an online content item from the plurality of online content items based on the online content item having a highest combined value; and

cause, on the client device of the user, a presentation of the online content item during the active network session.

15. The non-transitory machine-readable storage device of claim 14, wherein the request is a request for aggregated content comprising a placeholder space and one or more pre-selected content items, and wherein the selected online content item is integrated into the placeholder space among the one or more pre-selected content items, the pre-selected content items being selected before the client device generates the request for the aggregated content.