DIFFERENTIAL PRIVACY FOR MESSAGE TEXT CONTENT MINING

• US 20180091466A1
• Filed: 09/20/2017
• Published: 03/29/2018
• Est. Priority Date: 09/23/2016
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Friedman, Eric D., Kumar, Ritwik K., Winstrom, Lucas
• Granted Patent: US 10,778,633 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 21/6254 by anonymising data, e.g. decorrelating personal data from the owner's identification
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]
• CPC Class Codes: H04L 51/212 using filtering or selective blocking
• CPC Class Codes: H04L 51/23 Reliability checks, e.g. acknowledgments or fault reporting

Abstract

Systems and methods are disclosed for determining whether a message received by a client may be spam, in a computing environment that preserves privacy. The message may be encrypted. A client invokes the methods when a message is received from a sender that is not known to the client. A client can decrypt the message, break the message into chunks, and apply a differentially private algorithm to the set of chunks. The client transmits the differentially private message sketches to an aggregation server. The aggregation server receives a large collection of such message sketches for a large plurality of clients. The aggregation server returns aggregated message chunk (e.g. frequency) information to the client to assist the client in determining whether the message may be spam. The client can process the message based on the determination without disclosing the message content to the server.

Claims

1. A computer-implemented method performed by a client device, the method comprising:

receiving an electronic message;

transforming the electronic message into a set of chunks;

applying a differential privacy algorithm to the set of chunks of the electronic message to generate a differentially private message sketch;

transmitting the differentially private message sketch to an aggregation server;

receiving aggregated message information from the aggregation server comprising a frequency estimator for the set of chunks of the electronic message; and

determining an action for processing the electronic message based at least in part on received aggregated message information.

2. The method of claim 1, further comprising:

determining whether a sender of the electronic message is known; and

wherein determining an action for processing the message is based at least in part on the sender not being known.

3. The method of claim 2, wherein a sender is known when an address of the sender is found in a contacts database on the client device or an address of the sender is found in a sender or receiver field of a message in a message database on the client device.

4. The method of claim 1, further comprising decrypting the electronic message prior to applying the differential privacy algorithm to the set of chunks of the electronic message.

5. The method of claim 1, wherein aggregated message information comprises an estimator for an aggregated value for each of the message chunks including an estimated frequency of each message chunk within the messages reported to the aggregation server.

6. The method of claim 1, wherein aggregated message information is used to construct an aggregate score for the electronic message.

7. The method of claim 6, wherein an action comprises at least one of:

quarantining the electronic message in response to the aggregate score indicating with a probability above a high threshold that the electronic message is likely to comprise a spam message;

allowing the electronic message to be received for presentation in response to the aggregate score indicating with a probability below a low threshold that the electronic message is likely to comprise a spam message; and

prompting for an action to take comprising at least one of reporting the electronic message as spam, quarantining the electronic message, or allowing the electronic message to be presented, in response to the aggregate score indicating with a probability above a low threshold and below a high threshold that the electronic message is likely to comprise a spam message.

8. The method of claim 7, wherein the action comprises quarantining the electronic message in response to the aggregate score indicating with a probability above a high threshold that the electronic message is likely to comprise a spam message, and a sender of the electronic message is not known.

9. A non-transitory computer readable medium programmed with instructions that, when executed by a client device comprising a processing system having at least one hardware processor, performs operations comprising:

receiving an electronic message;

transforming the electronic message into a set of chunks;

applying a differential privacy algorithm to the set of chunks of the electronic message to generate a differentially private message sketch;

transmitting the differentially private message sketch to an aggregation server;

receiving message aggregate information from the aggregation server comprising a frequency estimator for the set of chunks of the electronic message; and

determining an action for processing the electronic message based at least in part on received aggregated message information.

10. The non-transitory computer readable medium of claim 9, the operations further comprising determining whether a sender of the electronic message is known, wherein determining an action for processing the message is based at least in part on the sender not being known.

11. The non-transitory computer readable medium of claim 10, wherein a sender is known to a user when at least one of:

an address of the sender is found in a contacts database on the client device, or an address of the sender is found in a sender or receiver field of a message in a message database on the client device.

12. The non-transitory computer readable medium of claim 9, the operations further comprising decrypting the electronic message prior to applying the differential privacy algorithm to the set of chunks of the electronic message.

13. The non-transitory computer readable medium of claim 9, wherein aggregated message information comprises an estimator for an aggregated value for each of the message chunks, the estimator comprising an estimated frequency of each of the message chunks within the messages reported to the aggregation server.

14. The non-transitory computer readable medium of claim 9, wherein aggregated message information is used to construct an aggregate score for the set of message chunks.

15. The non-transitory computer readable medium of claim 14, wherein an action comprises at least one of:

quarantining the electronic message in response to the aggregate score indicating with a probability above a high threshold that the electronic message is likely to comprise a spam message;

allowing the message to be received for presentation to in response to the aggregate score indicating with a probability below a low threshold that the electronic message is likely to comprise a spam message; and

prompting for an action to take comprising at least one of reporting the electronic message as spam, quarantining the electronic message, or allowing the electronic message to be presented, in response to the aggregate score indicating with a probability above a low threshold and below a high threshold that the electronic message is likely to comprise a spam message.

16. The non-transitory computer readable medium of claim 15, wherein the action comprises quarantining the electronic message in response to the aggregate score indicating with a probability above a high threshold that the electronic message is likely to comprise a spam message, and a sender of the electronic message is not known.

17. A system comprising:

a processing system coupled to a memory programmed with executable instructions that, when executed by the processing system on a client device perform operations, comprising;

receiving an electronic message;

transforming the electronic message into a set of chunks;

applying a differential privacy algorithm to the set of chunks of the electronic message to generate a differentially private message sketch;

transmitting the differentially private message sketch to an aggregation server;

receiving aggregated message information from the aggregation server comprising a frequency estimator for the set of chunks of the electronic message; and

determining an action for processing the electronic message based at least in part on received aggregated message information.

18. The system of claim 17, the operations further comprising determining whether a sender of the electronic message is known, and wherein determining an action for processing the message is based at least in part on the sender not being known.

19. The system of claim 18, wherein a sender is known to a user when at least one of:

an address of the sender is found in a contacts database on the client device, or an address of the sender is found in a sender or receiver field of a message in a message database on the client device.

20. The system of claim 17, the operations further comprising decrypting the electronic message prior to applying the differential privacy algorithm to the set of chunks of the electronic message.

21. The system of claim 17, wherein aggregated message information comprises a frequency count for each chunk of the set of chunks of a message or an aggregate count of message chunks received by the aggregation server.

22. The system of claim 17, wherein aggregated message information comprises a composite frequency score for the set of message chunks.

23. The system of claim 22, wherein an action comprises at least one of:

quarantining the electronic message in response to the composite frequency score indicating with a probability above a high threshold that the electronic message is likely to comprise a spam message;

allowing the message to be received for presentation in response to the composite frequency score indicating with a probability below a low threshold that the electronic message is likely to comprise a spam message; and

prompting for an action to take comprising at least one of reporting the electronic message as spam, quarantining the electronic message, or allowing the electronic message to be presented, in response to the composite frequency score indicating with a probability above a low threshold and below a high threshold that the electronic message is likely to comprise a spam message.

24. The system of claim 23, wherein the action comprises quarantining the electronic message in response to the composite frequency score indicating with a probability above a high threshold that the electronic message is likely to comprise a spam message, and a sender of the electronic message is not known.

25. A computer-implemented method performed by a server device, the method comprising:

receiving a differentially private message sketch from a plurality of client devices;

generating aggregated message information from multiple received differentially private message sketches including a global frequency estimator for chunks of the messages; and

transmitting the aggregated message information to one or more client devices.

26. The method of claim 25, wherein the global frequency estimator includes a frequency vector.

27. The method of claim 26, further comprising:

encrypting the frequency vector with a public homomorphic encryption key;

transmitting the public homomorphic encryption key to one or more client devices; and

encrypting the message chunks with the public homomorphic encryption key on a client device.

28. The method of claim 27, further comprising:

transmitting the public homomorphic encryption key from a client to the server device.