Quality scoring system for advertisements and content in an online system

• US 9,460,451 B2
• Filed: 07/01/2013
• Issued: 10/04/2016
• Est. Priority Date: 07/01/2013
• Status: Active Grant

• Current Assignee: Slack Technologies LLC (Salesforce.com, Inc.)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Ruiz, Bruno Fernandez, Krishnamurthy, Prabhakar, Langlois, Jean-Marc, Gupta, Abhay, Lavi, Aharon
• Primary Examiner(s): Ahn, Sangwoo
• Application Number:
  US13/932,655
• Publication Number:
  US 20150006280A1
• Time in Patent Office: 1,191 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 30/00 Commerce
• CPC Class Codes: G06Q 30/0246 Traffic
• CPC Class Codes: G06Q 30/0273 Determination of fees for advertising

Abstract

A server system of an online information system displays advertising items and content items retrieved from storage devices as a stream viewable by a user on a user device. The advertisement items and the content items are ordered in the stream by a ranking score computed for each of the advertisement items and each of the content items. A quality scoring system determines an affinity score between a user and a present content item based on features of the present content item matching user profile parameters associated with the user and identifies post-interaction satisfaction with a prior content item. The quality scoring system determines a quality score based on the affinity score and the post-interaction satisfaction. The quality score is used for ordering items in the stream. The server system transmits a web page including the stream to a user device over a network. In this manner, advertising items and content items compete in a unified marketplace for inclusion in the stream for viewing by the end user.

Claims

1. A system for determining a quality score for a content item in a streaming media feed, comprising:

a processor;

memory, the memory communicatively coupled to the processor, the memory comprising instructions executable by the processor to;

determine an affinity score between a user and a present content item based on features of the present content item matching user profile parameters associated with the user, wherein the affinity score determination further comprises;

instructions to determine a first probability, which is a probability the user will select to interact with the present content item in a streaming media feed;

instructions to determine a second probability, which is a probability the user will select to interact with a content item, in general, in the streaming media feed, wherein the relationship between the first determined probability and the second determined probability is a ratio; and

instructions to determine the affinity score based on a relationship between the first determined probability and the second determined probability;

identify post-interaction satisfaction with a prior content item that matches categorically the present content item;

determine a quality score based on the affinity score and the post-interaction satisfaction; and

a display, communicatively coupled to the processor, the display configured to display the quality score.

2. The system of claim 1, wherein the instructions to determine the quality score based on the affinity score and the post-interaction satisfaction include instructions to:

derive a post-interaction satisfaction score based on the identified post-interaction satisfaction; and

determine the quality score by computing a product of the affinity score and the post-interaction satisfaction score.

3. The system of claim 1, wherein the affinity score is independent of a position of the present content item in a streaming media feed.

4. The system of claim 1, wherein the affinity score is independent of a configuration of the present content item in a streaming media feed.

5. The system of claim 1, wherein the memory comprises instructions executable by the processor to determine the quality score based on the affinity score, the post-interaction satisfaction, and a popularity score.

6. The system of claim 5, wherein the popularity score is based on a click through rate for the present content item within a window of time.

7. The system of claim 1, wherein the present content item is an advertisement, and wherein the present content item is a first content item, wherein a second content item is an article, wherein the first and the second content items include similar subject matter categorically, and wherein the instructions to determine the affinity score include instructions to determine an affinity score of the second content item based on the affinity score of the first content item.

8. The system of claim 7, wherein the similarity of the subject matter categorically is identified by matching metadata elements embedded in the first and the second content items, and wherein the matching metadata elements are categories of a content taxonomy.

9. The system of claim 7, wherein the similarity of the subject matter categorically is identified by matching metadata elements embedded in the first and the second content items, and wherein the matching metadata elements are associations with a webpage of a series of webpages provided by a web content provider.

10. The system of claim 9, wherein the webpage is an online encyclopedia or dictionary entry.

11. A system for determining a quality score for an advertisement or article in a streaming media feed, comprising:

a processor; and

memory, the memory communicatively coupled to the processor, the memory comprising instructions executable by the processor to;

determine a first probability, which is a probability a user will select to interact with a present advertisement or article in a streaming media feed, based on features of the present advertisement or article matching user profile parameters associated with the user;

determine a second probability, which is a probability the user will select to interact with an advertisement or article, in general, in a streaming media feed wherein the relationship between the first determined probability and the second determined probability is a ratio;

determine an affinity score of the present advertisement or article, wherein the affinity score is a relationship between the first determined probability and the second determined probability;

identify post-interaction satisfaction with a prior advertisement or article that matches categorically the present advertisement or article; and

determine a quality score based on the affinity score and the post-interaction satisfaction.

12. The system of claim 11, wherein the instructions to determine the quality score based on the affinity score and the post-interaction satisfaction include instructions to determine the quality score by computing the product of the affinity score and the post-interaction satisfaction.

13. The system of claim 11, wherein selection to interact with the present advertisement or article includes clicking the present advertisement or article or clicking a hyperlink to the present advertisement or article.

14. The system of claim 11, wherein selection to interact with the present advertisement or article includes a gesture made with respect to the present advertisement or article or a gesture made with respect to a hyperlink to the present advertisement or article.