SYSTEM AND METHOD FOR LEARNING LATENT REPRESENTATIONS FOR NATURAL LANGUAGE TASKS

• US 20120150531A1
• Filed: 12/08/2010
• Published: 06/14/2012
• Est. Priority Date: 12/08/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: BANGALORE, Srinivas, Chopra, Sumit
• Granted Patent: US 9,135,241 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30

Abstract

Disclosed herein are systems, methods, and non-transitory computer-readable storage media for learning latent representations for natural language tasks. A system configured to practice the method analyzes, for a first natural language processing task, a first natural language corpus to generate a latent representation for words in the first corpus. Then the system analyzes, for a second natural language processing task, a second natural language corpus having a target word, and predicts a label for the target word based on the latent representation. In one variation, the target word is one or more word such as a rare word and/or a word not encountered in the first natural language corpus. The system can optionally assigning the label to the target word. The system can operate according to a connectionist model that includes a learnable linear mapping that maps each word in the first corpus to a low dimensional latent space.

Claims

1. A method of learning latent representations for natural language tasks, the method comprising:

analyzing, for a first natural language processing task, a first natural language corpus to generate a latent representation for words in the first natural language corpus;

analyzing, for a second natural language processing task, a second natural language corpus having a target word; and

predicting, via a processor, a label for the target word based on the latent representation.

2. The method of claim 1, wherein the target word is at least one of a rare word and a word not encountered in the first natural language corpus.

3. The method of claim 1, wherein predicting the label for the target word is further based on a connectionist model.

4. The method of claim 3, wherein the connectionist model comprises a learnable linear mapping which maps each word in the first natural language corpus to a low dimensional latent space.

5. The method of claim 3, wherein the connectionist model comprises a classifier that classifies low dimensional representations of words.

6. The method of claim 1, further comprising assigning the label to the target word.

7. The method of claim 1, wherein the first natural language processing task is a part-of-speech tagging task.

8. The method of claim 1, wherein the second natural language processing task is a supertagging task.

9. The method of claim 1, wherein the target word is a collection of target words.

10. A system for learning latent representations for natural language tasks, the system comprising:

a processor;

a first module configured to control the processor to analyze, for a first natural language processing task, a first natural language corpus to generate a latent representation for words in the first natural language corpus;

a second module configured to control the processor to analyze, for a second natural language processing task, a second natural language corpus having a target word; and

a third module configured to control the processor to predict, via a processor, a label for the target word based on the latent representation.

11. The system of claim 10, wherein the target word is at least one of a rare word and a word not encountered in the first natural language corpus.

12. The system of claim 10, wherein the third module is further configured to control the processor to predict the label for the target word based on a connectionist model.

13. The system of claim 12, wherein the connectionist model comprises a learnable linear mapping which maps each word in the first natural language corpus to a low dimensional latent space.

14. The system of claim 12, wherein the connectionist model comprises a classifier that classifies low dimensional representations of words.

15. The system of claim 10, further comprising a fourth module configured to control the processor to assign the label to the target word.

16. A non-transitory computer-readable storage medium storing instructions which, when executed by a computing device, cause the computing device to learn latent representations for natural language tasks, the instructions comprising:

analyzing, for a first natural language processing task, a first natural language corpus to generate a latent representation for words in the first natural language corpus;

analyzing, for a second natural language processing task, a second natural language corpus having a target word; and

predicting, via a processor, a label for the target word based on the latent representation.

17. The non-transitory computer-readable storage medium of claim 16, wherein predicting the label for the target word is further based on a connectionist model, and wherein the connectionist model comprises a classifier that classifies low dimensional representations of words.

18. The non-transitory computer-readable storage medium of claim 16, the instructions further comprising assigning the label to the target word.

19. The non-transitory computer-readable storage medium of claim 16, wherein the first natural language processing task is a part-of-speech tagging task.

20. The non-transitory computer-readable storage medium of claim 16, wherein the second natural language processing task is a supertagging task.