Deep Neural Networks and Methods for Using Same

• US 20090210218A1
• Filed: 02/09/2009
• Published: 08/20/2009
• Est. Priority Date: 02/07/2008
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Laboratories America Inc (NEC Corporation)
• Inventors: Weston, Jason, Collobert, Ronan
• Granted Patent: US 8,504,361 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates

Abstract

A method and system for labeling a selected word of a sentence using a deep neural network includes, in one exemplary embodiment, determining an index term corresponding to each feature of the word, transforming the index term or terms of the word into a vector, and predicting a label for the word using the vector. The method and system, in another exemplary embodiment, includes determining, for each word in the sentence, an index term corresponding to each feature of the word, transforming the index term or terms of each word in the sentence into a vector, applying a convolution operation to the vector of the selected word and at least one of the vectors of the other words in the sentence, to transform the vectors into a matrix of vectors, each of the vectors in the matrix including a plurality of row values, constructing a single vector from the vectors in the matrix, and predicting a label for the selected word using the single vector.

Claims

1. A method for labeling a selected word of a sentence using a deep neural network, the method comprising the steps of:

determining in a first computer process, an index term corresponding to each feature of the word;

transforming the index term or terms of the word into a vector in a second computer process; and

predicting a label for the word in a final computer process using the vector.

2. The method of claim 1, wherein the labeling comprises at least two different natural language processing tasks and further comprising the step of performing the determining, transforming and predicting steps for each task simultaneously.

3. The method of claim 1, wherein prior to the label predicting step, further comprising the steps of:

determining in a third computer process, an index term corresponding to data relating to a distance between the word and a selected predicate;

transforming the index term corresponding to the data into another vector in a fourth computer process; and

concatenating the vectors in a fifth computer process to generate a single vector,

wherein the single vector comprises the vector used in the label predicting step.

4. The method of claim 3, wherein the step of transforming the index term or terms of the word into a vector is performed by applying a lookup table to the word and wherein the step of transforming the index term corresponding to the data into another vector is performed by applying a lookup table to additional data for each of the words of the sentence.

5. The method of claim 1, wherein the transforming step is performed by applying a lookup table to the word.

10. The method of claim 5, wherein the labeling comprises at least two different natural language processing tasks and further comprising the step of performing the determining, transforming, applying, constructing and predicting steps for each task simultaneously.

6. A method for labeling a selected word of a sentence using a deep neural network, the method comprising the steps of:

determining in a first computer process, for each word in the sentence, an index term corresponding to each feature of the word;

transforming the index term or terms of each word in the sentence into a vector in a second computer process;

applying a convolution operation in a third computer process to the vectors to transform the vectors into a matrix of vectors, each of the vectors in the matrix including a plurality of row values;

constructing a single vector from the vectors in the matrix in a fourth computer process; and

predicting a label for the selected word in a fifth computer process using the single vector.

7. The method of claim 6, wherein prior to the convolution applying step, further comprising the steps of:

determining in a first additional computer process, for each word in the sentence, an index term corresponding to data relating to a distance between the word and a selected predicate;

transforming the index term corresponding to the data of each word in the sentence into another vector in a second additional computer process; and

concatenating the vectors of each word in the sentence in a third additional computer process to generate the vector of the selected word and the at least one vector of the other words in the convolution applying step.

8. The method of claim 7, wherein the step of transforming the index term or terms of each word in the sentence into a vector is performed by applying a lookup table to each word in the sentence and wherein the step of transforming the index term corresponding to the data of each word in the sentence into another vector is performed by applying a lookup table to to additional data for each each of the words of the sentence.

9. The method of claim 6, wherein the transforming step is performed by applying a lookup table to each word in the sentence.

11. A system comprising:

a central processing unit; and

a memory communicating with the central processing unit, the memory comprising instructions executable by the processor for labeling a selected word of a sentence using a deep neural network by;

determining an index term corresponding to each feature of the word;

transforming the index term or terms of the word into a vector; and

predicting a label for the word using the vector.

13. The system of claim 11, wherein prior to predicting the label predicting, further comprising:

determining an index term corresponding to data relating to a distance between the word and a selected predicate;

transforming the index term corresponding to the data into another vector; and

concatenating the vectors to generate a single vector,

wherein the single vector comprises the vector used in the label predicting step.

14. The system of claim 13, wherein the transforming of the index term or terms of the word into a vector is performed by applying a lookup table to the word and wherein the transforming of the index term corresponding to the data into another vector is performed by applying a lookup table to additional data for each of the words of the sentence.

15. The system of claim 11, wherein the transforming is performed by applying a lookup table to the word.

16. The system of claim 11, wherein the labeling comprises at least two different natural language processing tasks and further comprising the step of performing the determining, transforming and predicting steps for each task simultaneously.

17. A system comprising:

a central processing unit; and

a memory communicating with the central processing unit, the memory comprising instructions executable by the processor for labeling a selected word of a sentence using a deep neural network by;

determining, for each word in the sentence, an index term corresponding to each feature of the word;

transforming the index term or terms of each word in the sentence into a vector;

applying a convolution operation to the vector of the selected word and at least one of the vectors of the other words in the sentence, to transform the vectors into a matrix of vectors, each of the vectors in the matrix including a plurality of row values;

constructing a single vector from the vectors in the matrix; and

predicting a label for the selected word using the single vector.

18. The system of claim 17, wherein prior to the convolution, further comprising:

determining, for each word in the sentence, an index term corresponding to data relating to a distance between the word and a selected predicate;

transforming the index term corresponding to the data of each word in the sentence into another vector; and

concatenating the vectors of each word in the sentence to generate the vector of the selected word and the at least one vector of the other words.

19. The system of claim 18, wherein the transforming of the index term or terms of each word in the sentence into a vector is performed by applying a lookup table to each word in the sentence and wherein the transforming of the index term corresponding to the data of each word in the sentence into another vector is performed by applying a lookup table to additional data for each of the words of the sentence.

20. The system of claim 17, wherein the transforming is performed by applying a lookup table to each word in the sentence.

21. The method of claim 17, wherein the labeling comprises at least two different natural language processing tasks and further comprising the step of performing the determining, transforming, applying, constructing and predicting steps for each task simultaneously.