Temporal features in a messaging platform

• US 9,892,431 B1
• Filed: 09/08/2016
• Issued: 02/13/2018
• Est. Priority Date: 03/31/2011
• Status: Active Grant

• Current Assignee: X Corp. (f/k/a Twitter, Inc.) (X Holdings Corp.)
• Original Assignee: X Corp. (f/k/a Twitter, Inc.) (X Holdings Corp.)
• Inventors: Agrawal, Parag, Srivastava, Utkarsh, Chellapilla, Kumar, Ma, Qicheng
• Primary Examiner(s): Lu, Kuen S
• Application Number:
  US15/259,924
• Time in Patent Office: 523 Days
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/335 Filtering based on additional data, e.g. user or group profiles filtering in web context G06F16/9535, G06F16/9536
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06Q 30/0242 Determining effectiveness of advertisements
• CPC Class Codes: G06Q 30/0254 based on statistics
• CPC Class Codes: G06Q 30/0275 Auctions
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]

Abstract

A real-time messaging platform allows advertiser accounts to pay to insert candidate messages into the message streams requested by account holders. To accommodate multiple advertisers, the messaging platform controls an auction process that determines which candidate messages are selected for inclusion in a requested account holder'"'"'s message stream. Selection is based on a bid for the candidate message, the message stream that is requested, and a variety of other factors that vary depending upon the implementation. The process for selection of candidate messages generally includes the following steps, though any given step may be omitted or combined into another step in a different implementation: targeting, filtering, prediction, ranking, and selection.

Claims

1. A computer-implemented method comprising:

receiving, from a computing device accessing a social media platform via a first account, a request for a first message stream, wherein the first message stream is for a second account and includes one or more messages provided by the second account, and wherein the social media platform is accessible by a plurality of accounts, including the first and second accounts;

retrieving two or more candidate messages;

computing, based on a first machine learning model and one or more temporal features of past engagements by the first account with messages received by the first account on the social media platform, an account weight associated with the first account;

for each of the candidate messages;

computing, based on a second machine learning model and one or more features of the respective candidate message, a message weight for the respective candidate message; and

computing, based on the respective message weight and the account weight, a value that represents a likelihood of engagement with the respective candidate message by a user of the first account;

selecting, based on the computed values representing the likelihood of engagement, one of the candidate messages and inserting the selected candidate message in the first message stream; and

sending the first message stream to the computing system for display via the first account.

2. The method of claim 1, wherein each value representing the likelihood of engagement is further based on a temporal feature representing activity in the first account.

3. The method of claim 1, wherein the one or more temporal features of past engagements represents an inferred attribute of the first account, the inferred attribute based on engagement data that satisfies one or more temporal criteria.

4. The method of claim 3, wherein the engagement data satisfying the one or more temporal criteria comprises engagement data corresponding to messages received during certain periods of time.

5. The method of claim 3, wherein the inferred attribute is determined based on at least one of:

a location, an interest, a device type used to access the first account, and a software type used to access the first account.

6. The method of claim 1, wherein the first account is the second account.

7. The method of claim 1, wherein the first account is a follower of the second account.

8. The method of claim 1, wherein the first account and the second account are configured to receive messages authored by a publishing account, wherein the first message stream includes messages authored by the publishing account.

9. The method of claim 1, wherein the method further comprises:

creating a candidate message/first account pair for each candidate message of the set of two or more candidate messages;

associating a cross feature with each candidate message/first account pair; and

determining a temporal cross feature associated with the cross feature.

10. The method of claim 9, wherein determining a temporal cross feature associated with the cross feature includes determining temporal features of the cross feature based fatigue maps or a fatigue score.

11. The method of claim 10, wherein a fatigue score is determined for each candidate message/first account pair based on a weighted combination of exponential decay factors, each exponential decay factor corresponding to an impression between the first account and the candidate message in the candidate message/first account pair, each exponential decay factor based on time elapsed since the impression.

12. The method of claim 10, wherein a fatigue score is determined for each candidate message/first account pair based on a weighted combination of impressions between the first account and the candidate message in the candidate message/first account pair.

13. The method of claim 10, wherein the fatigue score is determined based on impressions between the first account and other messages authored by the second account.

14. The method of claim 9, wherein determining the values representing the likelihood of engagement value for each candidate message in the set of two or more candidate messages further comprises:

accessing a temporal cross weight for a temporal cross feature associated with each candidate message/first account pair, the temporal cross weight calculated based on temporal features of past activity within the first account; and

determining the respective value representing the likelihood of engagement for each candidate message in the set of two or more candidate messages based in part on the respective temporal cross weight.

15. The method of claim 1, wherein determining the account weight associated with the first account comprises:

responsive to receiving the one or more signals indicating the request for the first message stream and in real time, determining the account weight associated with the first account, and

wherein the method further comprises;

after sending the first message stream, receiving, from the computing system, one or more additional signals indicating whether the user engaged with the first candidate message in the first message stream; and

updating the machine learning model based on whether the user engaged with the first candidate message in the first message stream.

16. A non-transitory computer-readable storage medium comprising instructions executable by a processor, the instructions when executed causing the processor to:

receive, from a computing device accessing a social media platform via a first account, a request for a first message stream, wherein the first message stream is for a second account and includes one or more messages provided by the second account, and wherein the social media platform is accessible by a plurality of accounts, including the first and second accounts;

retrieve two or more candidate messages;

compute, based on a first machine learning model and one or more temporal features of past engagements by the first account with messages received by the first account on the social media platform, an account weight associated with the first account;

for each of the candidate messages;

compute, based on a second machine learning model and one or more features of the respective candidate message, a message weight for the respective candidate message; and

compute, based on the respective message weight and the account weight, a value that represents a likelihood of engagement with the respective candidate message by a user of the first account;

select, based on the computed values representing the likelihood of engagement, one of the candidate messages and inserting the selected candidate message in the first message stream; and

send the first message stream to the computing system for display via the first account.

17. The non-transitory computer-readable storage medium of claim 16, wherein each value representing the likelihood of engagement is further based on a temporal feature associated with activity in the first account.

18. The non-transitory computer-readable storage medium of claim 16, wherein the first account is one of:

the second account, an account that subscribed to receive messages authored by the second account, an account that subscribed to a publishing account other than the second account, or the publishing account.

19. The non-transitory computer-readable storage medium of claim 16, wherein the instructions that when executed cause the processor to determine the values representing the likelihood of engagement for each candidate message in the set of two or more candidate messages include instructions that, when executed, cause a processor to:

create a candidate message/first account pair for each candidate message of the set of two or more candidate messages;

associate a cross feature with each candidate message/first account pair;

determine a temporal cross feature associated with the cross feature, wherein the temporal cross weight is based on a fatigue score;

access a temporal cross weight for a temporal cross feature associated with each candidate message/first account pair, the temporal cross weight calculated based on temporal features of past activity within the first account; and

determine the respective value representing the likelihood of engagement for each candidate message in the set of two or more candidate messages based in part on the respective temporal cross weight.

20. A system comprising:

a processor; and

a non-transitory computer-readable storage medium comprising instructions executable by the processor, the instructions when executed causing the processor to;

receive, from a computing device accessing a social media platform via a first account, a request for a first message stream, wherein the first message stream is for a second account and includes one or more messages provided by the second account, and wherein the social media platform is accessible by a plurality of accounts, including the first and second accounts;

retrieve two or more candidate messages;

compute, based on a first machine learning model and one or more temporal features of past engagements by the first account with messages received by the first account on the social media platform, an account weight associated with the first account;

for each of the candidate messages;

compute, based on a second machine learning model and one or more features of the respective candidate message, a message weight for the respective candidate message; and

compute, based on the respective message weight and the account weight, a value that represents a likelihood of engagement with the respective candidate message by a user of the first account;

select, based on the computed values representing the likelihood of engagement, one of the candidate messages and inserting the selected candidate message in the first message stream; and

send the first message stream to the computing system for display via the first account.