Removing personal information from text using a neural network

• US 10,169,315 B1
• Filed: 04/27/2018
• Issued: 01/01/2019
• Est. Priority Date: 04/27/2018
• Status: Active Grant

• Current Assignee: Asapp, Inc.
• Original Assignee: Asapp, Inc.
• Inventors: Heckel, Frederick William Poe, Henry, Shawn
• Primary Examiner(s): Bloomquist, Keith D
• Application Number:
  US15/964,629
• Time in Patent Office: 249 Days
• Field of Search: None
• CPC Class Codes: G06F 21/60 Protecting data
• CPC Class Codes: G06F 21/6245 Protecting personal data, e.g. for financial or medical purposes
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/048 Activation functions
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

A neural network may be used to remove personal information from text (such as names, addresses, credit card numbers, or social security numbers), and replace the personal information with a label indicating the type or class of the removed information. The neural network may comprise multiple layers that compute a context vector for words of the text, compute label scores for words of the text using the context vectors, and select a label for each word using the label scores. Words corresponding to certain labels may be replaced with a label, such as replacing the digits of a credit card number with a label <cc_number>. The redacted text may then be presented to a person or stored for later processing.

Claims

1. A computer-implemented method for removing personal information from text using a neural network, the method comprising:

obtaining the neural network, wherein the neural network is configured to process the text and select a label from a plurality of possible labels for each word of the text, wherein each label corresponds to a class of words, and wherein at least one label corresponds to a class of words to be removed from the text;

receiving the text;

obtaining a word embedding for each word of the text, where a word embedding represents a word in a vector space;

computing a context vector for each word of the text by processing the word embeddings with a first layer of the neural network, where a context vector for a given word includes information about words before or after the given word;

computing label scores for each word of the text by processing each of the context vectors with a second layer of the neural network, wherein each label score indicates a match between a word and a class of words;

selecting a label for each word of the text by processing the label scores with a third layer of the neural network; and

generating redacted text by replacing a first word of the text with a first label corresponding to the first word.

2. The computer-implemented method of claim 1, comprising causing the redacted text to be presented to a person.

3. The computer-implemented method of claim 1, comprising classifying the redacted text by processing the redacted text with a text classifier.

4. The computer-implemented method of claim 1, wherein the first layer is a recurrent neural network layer or a bidirectional recurrent neural network layer.

5. The computer-implemented method of claim 1, wherein the second layer comprises a classifier.

6. The computer-implemented method of claim 1, wherein the third layer comprises a conditional random field.

7. The computer-implemented method of claim 1, wherein the text is from a message received from a customer of a company and relates to obtaining support from the company.

8. The computer-implemented method of claim 1, comprising:

storing the redacted text in a first data store;

generating second redacted text, wherein the second redacted text has a higher redaction level then the redacted text; and

storing the second redacted text in a second data store.

9. A system for removing personal information from text using a neural network, the system comprising at least one computer configured to:

obtain the neural network, wherein the neural network is configured to process the text and select a label from a plurality of possible labels for each word of the text, wherein each label corresponds to a class of words, and wherein at least one label corresponds to a class of words to be removed from the text;

receive the text;

obtain a word embedding for each word of the text;

compute a context vector for each word of the text by processing the word embeddings with a first layer of the neural network;

compute label scores for each word of the text by processing each of the context vectors with a second layer of the neural network, wherein each label score indicates a match between a word and a class of words;

select a label for each word of the text by processing the label scores with a third layer of the neural network; and

generate redacted text by replacing a first word of the text with a first label corresponding to the first word.

10. The system of claim 9, wherein the at least one computer is configured to obtain a word embedding by:

obtaining a first embedding corresponding to words of a vocabulary;

obtaining character embeddings for characters of the word, wherein each character embedding corresponds to a character of a set of characters;

computing a second embedding using the character embeddings; and

obtaining the word embedding by combining the first embedding and the second embedding.

11. The system of claim 9, wherein the at least one computer is configured to:

generate linguistic features for each word of the text; and

computing the label scores for each word of the text comprises processing the linguistic features with the second layer.

12. The system of claim 9, wherein the second layer comprises a multi-layer perceptron.

13. The system of claim 9, wherein the at least one computer is configured to generate the redacted text by replacing sequences of a label with a single label.

14. The system of claim 9, wherein the at least one computer is configured to, before obtaining the word embedding for each word of the text, replacing digits in the text with a token that represents digits.

15. The system of claim 9, wherein the system is implemented by a third party providing services to a plurality of companies.

16. One or more non-transitory computer-readable media comprising computer executable instructions that, when executed, cause at least one processor to perform actions comprising:

obtaining a neural network, wherein the neural network is configured to process text and select a label from a plurality of possible labels for each word of the text, wherein each label corresponds to a class of words, and wherein at least one label corresponds to a class of words to be removed from the text;

receiving the text;

obtaining a word embedding for each word of the text;

computing a context vector for each word of the text by processing the word embeddings with a first layer of the neural network;

computing label scores for each word of the text by processing each of the context vectors with a second layer of the neural network, wherein each label score indicates a match between a word and a class of words;

selecting a label for each word of the text by processing the label scores with a third layer of the neural network; and

generating redacted text by replacing a first word of the text with a first label corresponding to the first word.

17. The one or more non-transitory computer-readable media of claim 16, wherein a context vector of a word corresponds to a hidden state vector of the first layer of the neural network.

18. The one or more non-transitory computer-readable media of claim 16, wherein the first layer of the neural network is a convolutional layer.

19. The one or more non-transitory computer-readable media of claim 16, wherein the third layer comprises a sequence model.

20. The one or more non-transitory computer-readable media of claim 16, wherein generating the redacted text comprises removing personally identifiable information from the text.