System and method for detecting unwanted content

• US 8,918,473 B1
• Filed: 10/09/2012
• Issued: 12/23/2014
• Est. Priority Date: 10/09/2012
• Status: Active Grant

• Current Assignee: WhatsApp LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Original Assignee: WhatsApp LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Inventors: O'Connor, Bryan D.
• Primary Examiner(s): NANO, SARGON N
• Application Number:
  US13/647,957
• Time in Patent Office: 805 Days
• Field of Search: 709/206, 709/207, 709/217, 709/220, 709/223, 709/224, 709/225, 709/246
• US Class Current: 709/206
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]
• CPC Class Codes: H04L 43/10 Active monitoring, e.g. heartbeat, ping or trace-route
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 51/212 using filtering or selective blocking
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

A system and method for detecting unwanted electronic content, such as spam. As a user operates an application to send messages, several metrics are tracked to allow the system to analyze her activity. Illustrative metrics may include, but are not limited to, block count (e.g., how many other users have blocked her), block rate (e.g., the rate at which other users block her), peer symmetry (e.g., percentage of her messages that are to other users that have her in their address book), message uniqueness (e.g., how unique her messages are from each other), peer uniqueness (e.g., how unique the addressees of her messages are), message rate (e.g., the rate at which she sends messages). Periodically, metrics may be compared to corresponding thresholds. Depending on whether a threshold is crossed, and which threshold is crossed, she may be banned from using the application or placed on a watch list.

Claims

1. A processor-implemented method of detecting distribution of unwanted content, the method comprising:

receiving from a first user an electronic message having one or more addressees;

for each addressee in the one or more addressees, updating a peer symmetry metric to reflect whether the first user is included in an address book of the addressee;

updating a message rate reflecting a rate at which the first user is sending electronic messages;

comparing the updated message rate to one or more threshold message rates;

if the updated message rate exceeds a first message rate threshold, preventing the user from sending additional electronic messages; and

if the updated message rate exceeds a second message rate threshold, but not the first message rate threshold, placing the first user in a watch list.

2. The method of claim 1, further comprising:

creating a representation of the electronic message that is different than the electronic message;

comparing the representation of the electronic message to representations of one or more previous electronic messages sent by the first user during a predetermined time period;

if the electronic message was sent previously by the first user during the predetermined time period, incrementing a message count associated with the electronic message;

if the electronic message was not previously sent by the first user during the predetermined time period, storing the representation of the electronic message; and

updating a message uniqueness metric in a manner depending on whether the electronic message had been sent previously during the predetermined time period.

3. The method of claim 1, further comprising:

comparing a first addressee of the electronic message to addressees of one or more previous electronic messages sent by the first user during a predetermined time period;

if the first addressee was sent a previous electronic message during the predetermined time period, incrementing a message count associated with the first addressee;

if the first addressee was not sent a previous electronic message during the predetermined time period, storing the first addressee; and

updating a peer uniqueness metric in a manner depending on whether the first addressee had been sent a previous message during the predetermined time period.

4. The method of claim 1, further comprising:

receiving from a second user a block request regarding the first user;

incrementing a total block count of the first user; and

updating a block rate reflecting a rate at which the first user is blocked by other users.

5. The method of claim 4, further comprising:

preventing the first user from sending additional electronic messages if the total block count exceeds a first block count threshold; and

placing the first user on the watch list if the total block count exceeds a second block count threshold but not the first block count threshold.

6. The method of claim 5, further comprising:

placing the first user on the watch list if the block rate exceeds a first block rate threshold.

7. The method of claim 1, further comprising:

adding the first user to an address book of a second user; and

updating the peer symmetry metric.

8. The method of claim 1, further comprising, prior to said receiving from the first user an electronic message:

logging the first user into a computer system hosting an application for sending electronic messages;

retrieving from storage a first set of metrics, including a total block count reflecting how many other users have blocked the first user;

if the user is on the watch list, retrieving from storage a second set of metrics, including;

a message uniqueness metric reflecting a percentage of previous electronic messages sent by the first user that were unique from other messages sent by the first user; and

a peer uniqueness metric reflecting a percentage of addressees of the previous electronic messages that were unique from addressees of the other messages sent by the first user; and

if the user is not on the watch list, initializing the second set of metrics.

9. The method of claim 8, further comprising:

retrieving a first list of other users to whom the first user is subscribed;

retrieving a second list of other users that have blocked the first user; and

striking from the first list any users identified on the second list.

10. The method of claim 1, further comprising:

logging the user out of a computer system hosting an application for sending electronic messages;

storing a first set of metrics, including a total block count reflecting how many other users have blocked the first user; and

if the user is on the watch list, storing a second set of metrics, including;

a message uniqueness metric reflecting a percentage of previous electronic messages sent by the first user that were unique from other messages sent by the first user; and

a peer uniqueness metric reflecting a percentage of addressees of the previous electronic messages that were unique from addressees of the other messages sent by the first user.

11. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform a method of detecting distribution of unwanted content, the method comprising:

receiving from a first user an electronic message having one or more addressees;

for each addressee in the one or more addressees, updating a peer symmetry metric to reflect whether the first user is included in an address book of the addressee;

updating a message rate reflecting a rate at which the first user is sending electronic messages;

comparing the updated message rate to one or more threshold message rates;

if the updated message rate exceeds a first message rate threshold, preventing the user from sending additional electronic messages; and

if the updated message rate exceeds a second message rate threshold, but not the first message rate threshold, placing the first user in a watch list.

12. The non-transitory computer-readable medium of claim 11, wherein the method further comprises:

creating a representation of the electronic message that is different than the electronic message;

comparing the representation of the electronic message to representations of one or more previous electronic messages sent by the first user during a predetermined time period;

if the electronic message was sent previously by the first user during the predetermined time period, incrementing a message count associated with the electronic message;

if the electronic message was not previously sent by the first user during the predetermined time period, storing the representation of the electronic message; and

updating a message uniqueness metric in a manner depending on whether the electronic message had been sent previously during the predetermined time period.

13. The non-transitory computer-readable medium of claim 12, wherein the method further comprises:

comparing a first addressee of the electronic message to addressees of one or more previous electronic messages sent by the first user during a predetermined time period;

if the first addressee was sent a previous electronic message during the predetermined time period, incrementing a message count associated with the first addressee;

if the first addressee was not sent a previous electronic message during the predetermined time period, storing the first addressee; and

updating a peer uniqueness metric in a manner depending on whether the first addressee had been sent a previous message during the predetermined time period.

14. The non-transitory computer-readable medium of claim 11, wherein the method further comprises:

receiving from a second user a block request regarding the first user;

incrementing a total block count of the first user; and

updating a block rate reflecting a rate at which the first user is blocked by other users.

15. The non-transitory computer-readable medium of claim 14, wherein the method further comprises:

preventing the first user from sending additional electronic messages if the total block count exceeds a first block count threshold; and

placing the first user on the watch list if the total block count exceeds a second block count threshold but not the first block count threshold.

16. The non-transitory computer-readable medium of claim 15, wherein the method further comprises:

placing the first user on the watch list if the block rate exceeds a first block rate threshold.

17. The non-transitory computer-readable medium of claim 11, wherein the method further comprises:

adding the first user to an address book of a second user; and

updating the peer symmetry metric.

18. The non-transitory computer-readable medium of claim 11, wherein the method further comprises, prior to said receiving from the first user an electronic message:

logging the first user into a computer system hosting an application for sending electronic messages;

retrieving from storage a first set of metrics, including a total block count reflecting how many other users have blocked the first user;

if the user is on the watch list, retrieving from storage a second set of metrics, including;

a message uniqueness metric reflecting a percentage of previous electronic messages sent by the first user that were unique from other messages sent by the first user; and

a peer uniqueness metric reflecting a percentage of addressees of the previous electronic messages that were unique from addressees of the other messages sent by the first user; and

if the user is not on the watch list, initializing the second set of metrics.

19. The non-transitory computer-readable medium of claim 18, wherein the method further comprises:

retrieving a first list of other users to whom the first user is subscribed;

retrieving a second list of other users that have blocked the first user; and

striking from the first list any users identified on the second list.

20. The non-transitory computer-readable medium of claim 11, wherein the method further comprises:

logging the user out of a computer system hosting an application for sending electronic messages;

storing a first set of metrics, including a total block count reflecting how many other users have blocked the first user; and

if the user is on the watch list, storing a second set of metrics, including;

a message uniqueness metric reflecting a percentage of previous electronic messages sent by the first user that were unique from other messages sent by the first user; and

a peer uniqueness metric reflecting a percentage of addressees of the previous electronic messages that were unique from addressees of the other messages sent by the first user.

21. An apparatus, comprising:

at least one processor; and

memory storing instructions that, when executed by the at least one processor, cause the apparatus to;

receive from a first user an electronic message having one or more addressees;

for each addressee in the one or more addressees, update a peer symmetry metric to reflect whether the first user is included in an address book of the addressee;

update a message rate reflecting a rate at which the first user is sending electronic messages;

compare the updated message rate to one or more threshold message rates;

if the updated message rate exceeds a first message rate threshold, prevent the user from sending additional electronic messages; and

if the updated message rate exceeds a second message rate threshold, but not the first message rate threshold, place the first user in a watch list.