Accounting for bias of user characteristics when determining consumption of content by online system users

• US 10,554,721 B2
• Filed: 09/25/2015
• Issued: 02/04/2020
• Est. Priority Date: 09/25/2015
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Predovic, Goran, Xu, Liang, Taine, Stephane
• Primary Examiner(s): Abedin, Normin
• Application Number:
  US14/866,059
• Publication Number:
  US 20170093650A1
• Time in Patent Office: 1,593 Days
• CPC Class Codes: H04L 12/1475 the splitting involving a third party
• CPC Class Codes: H04L 12/1496 involving discounts
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/306 User profiles
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04M 15/00 Arrangements for metering, time-control or time indication ; Metering, charging or billing arrangements for voice wireline or wireless communications, e.g. VoIP
• CPC Class Codes: H04M 15/07 Split billing, i.e. both A-party and B-party charged for the communication
• CPC Class Codes: H04M 15/09 Third party charged communications
• CPC Class Codes: H04M 15/64 On-line charging system [OCS]
• CPC Class Codes: H04M 15/8083 involving reduced rates or discounts, e.g. time-of-day reductions or volume discounts

Abstract

An online system determines one or more metrics describing consumption of content by various users by identifying users of the online system capable of being identified based on information received from multiple client devices. For example, the online system identifies users associated with user identifiers that are also associated with other types of identifying information (e.g., cookies, device identifiers). From the identified users, the online system generates a set of users based on a distribution of characteristics. The distribution of characteristics may be determined by the online system as characteristics of a group of users or received by the online system from a third party system and describes characteristics of users of the third party system. Based on interactions with content by users in the set, the online system determines one or more metrics describing consumption of content.

Claims

1. A method comprising:

identifying a set of trackable users of an online system wherein each user of the set of trackable users is a user of the online system that can be tracked across multiple client devices by the online system using identifying information received from the multiple client devices, each of the multiple client devices being used to access the online system;

accessing a distribution of characteristics of target users, the distribution of characteristics of target users indicating values of each of one or more characteristics of the target users;

generating a set of trackable target users of the online system selected from the set of trackable users of the online system such that a distribution of characteristics of the set of trackable target users of the online system match the distribution of characteristics of the target users;

receiving, from a plurality of client devices, identifying information regarding a plurality of viewing users and viewing information regarding each content item of a plurality of presented content items that are presented to each of the plurality of viewing users, the identifying information for each viewing user stored by one or more of the plurality of client devices;

associating the viewing information regarding each content item of at least some of the plurality of presented content items with a user of the set of trackable target users by matching the identifying information of the viewing user to which the content item was presented with user identifiers of the user of the set of trackable target users, the user identifiers stored at the online system;

and

determining one or more metrics describing consumption of content by the set of trackable target users based at least in part on the viewing information.

2. The method of claim 1, wherein determining one or more metrics describing consumption of content by the set of trackable target users based at least in part on the received information comprises:

identifying a content item presented to the trackable target users via the plurality of client devices from the viewing information received from the plurality of client devices; and

determining a number of unique users in the set of viewing users to whom the identified content item was presented based at least in part on the user identifiers associated with the trackable target users of the online system that are associated with the identifying information for each viewing user.

3. The method of claim 2, wherein determining the number of unique users in the set of users to whom the identified content item was presented comprises:

identifying user identifiers associated with users in the set of viewing users that the online system has associated with identifying information identifying users maintained by the plurality of client devices who were presented with the identified content item; and

determining a number of unique user identifiers from the identified user identifiers.

4. The method of claim 1, wherein determining one or more metrics describing consumption of content by the set of trackable target users based at least in part on the received information comprises:

identifying a content item presented to the viewing users via the plurality of client devices from the viewing information;

identifying unique users to whom the content item was presented based at least in part on user identifiers associated with the set of trackable target users that are associated with at least a set of the viewing users;

identifying a number of times the content item was presented to each of the identified unique users; and

determining a metric as an average of the identified number of times.

5. The method of claim 1, wherein accessing the distribution of characteristics of target users comprises:

selecting users of the online system who were presented with content from a third party system based on information associated with the users by the online system;

determining the distribution of characteristics based at least in part on characteristics of the selected users.

6. The method of claim 5, wherein the information associated with the users by the online system comprises information received by the online system from one or more tracking mechanisms included in content presented by the third party system that, when executed by a client device, communicate identifying information identifying users maintained by a client device and an identifier of content from the third party system presented by the client device.

7. The method of claim 1, wherein accessing the distribution of characteristics of target users comprises:

receiving the distribution of characteristics from a third party system external to the online system.

8. The method of claim 1, wherein accessing the distribution of characteristics of target users comprises:

sampling users of the online system; and

determining the distribution of characteristics based at least in part on characteristics of the sampled users.

9. The method of claim 1, further comprising:

applying one or more models to the one or more metrics for determining metrics describing consumption of content by a larger number of users than a number of users included in the set.

10. The method of claim 1, wherein the received identifying information is selected from a group consisting of:

device identifiers associated with the client devices, cookies stored on the client devices, and any combination thereof.

11. A non-transitory computer program product comprising a computer readable storage medium having instructions encoded thereon that, when executed by a processor, cause the processor to:

identify a set of trackable users of an online system wherein each user of the set of trackable users is a user of the online system that can be tracked across multiple client devices by the online system using identifying information received from the multiple client devices, each of the multiple client devices being used to access the online system;

access a distribution of characteristics of target users, the distribution of characteristics of target users indicating values of each of one or more characteristics of the target users;

generate a set of trackable target users of the online system selected from the set of trackable users of the online system such that a distribution of characteristics of the set of trackable target users of the online system match the distribution of characteristics of the target users;

receive from a plurality of client devices, identifying information regarding a plurality of viewing users and viewing information regarding each content item of a plurality of presented content items that are presented to each of the plurality of viewing users, the identifying information for each viewing user stored by one or more of the plurality of client devices;

associate the viewing information regarding each content item of at least some of the plurality of presented content items with a user of the set of trackable target users by matching the identifying information of the viewing user to which the content item was presented with user identifiers of the user of the set of trackable target users, the user identifiers stored at the online system;

and

determine one or more metrics describing consumption of content by the set of trackable target users based at least in part on the viewing information.

12. The non-transitory computer program product of claim 11, wherein the determination of one or more metrics describing consumption of content by the set of trackable target users based at least in part on the received information comprises:

identify a content item presented to the trackable target users via the plurality of client devices from the viewing information received from the plurality of client devices; and

determine a number of unique users in the set of viewing users to whom the identified content item was presented based at least in part on the user identifiers associated with the trackable target users of the online system that are associated with the identifying information for each viewing user.

13. The non-transitory computer program product of claim 12, wherein the determination of one or more metrics describing consumption of content by the set of trackable target users to whom the identified content item was presented comprises:

identify user identifiers associated with users in the set of viewing users that the online system has associated with identifying information identifying users maintained by the plurality of client devices who were presented with the identified content item; and

determine a number of unique user identifiers from the identified user identifiers.

14. The non-transitory computer program product of claim 11, wherein the determination of one or more metrics describing consumption of content by the set of trackable target users based at least in part on the received information comprises:

identify a content item presented to the viewing users via the plurality of client devices from the viewing information;

identify unique users to whom the content item was presented based at least in part on user identifiers associated with the set of trackable target users that are associated with at least a set of the viewing users;

identify a number of times the content item was presented to each of the identified unique users; and

determine a metric as an average of the identified number of times.

15. The non-transitory computer program product of claim 11, wherein the access of the distribution of characteristics of target users comprises:

select users of the online system who were presented with content from a third party system based on information associated with the users by the online system;

determine the distribution of characteristics based at least in part on characteristics of the selected users.

16. The non-transitory computer program product of claim 15, wherein the information associated with the users by the online system comprises information received by the online system from one or more tracking mechanisms included in content presented by the third party system that, when executed by a client device, communicate identifying information identifying users maintained by the client device and an identifier of content from the third party system presented by the client device.

17. The non-transitory computer program product of claim 11, wherein the access of the distribution of characteristics of target users comprises:

receive the distribution of characteristics from a third party system external to the online system.

18. The non-transitory computer program product of claim 11, wherein the access of the distribution of characteristics of target users comprises:

sample users of the online system; and

determine the distribution of characteristics based at least in part on characteristics of the sampled users.

19. The non-transitory computer program product of claim 11, wherein the computer readable storage medium further has instructions encoded thereon that, when executed by the processor, cause the processor to:

apply one or more models to the one or more metrics for determining metrics describing consumption of content by a larger number of users than a number of users included in the set.

20. The computer program product of claim 11, wherein the received identifying information is selected from a group consisting of:

device identifiers associated with the client devices, cookies stored on the client devices, and any combination thereof.