TRANS-LINGUAL REPRESENTATION OF TEXT DOCUMENTS

• US 20100324883A1
• Filed: 06/19/2009
• Published: 12/23/2010
• Est. Priority Date: 06/19/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Sutskever, Ilya, Platt, John
• Granted Patent: US 8,738,354 B2
• Time in Patent Office: Days
• US Class Current: 704/2
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06N 3/045 Combinations of networks

Abstract

A method of creating translingual text representations takes in documents in a first language and in a second language and creates a matrix using the words in the documents to represent which words are present in which language. An algorithm is applied to each matrix such that like documents are placed close to each other and unlike documents are moved far from each other.

Claims

1. A method of creating a trans-lingual text representation comprising:

accepting first language data, wherein the first language data comprises a plurality of documents in a first language;

accepting second language data, wherein the second language data comprises a plurality of documents in a second language, wherein each document in a second language is comparable to a corresponding document in the first language;

creating a first document-term matrix from the first language data, comprising a plurality of rows, each of said row corresponding to one of a plurality of documents in a first language;

creating a second document-term matrix from the second language data, comprising a plurality of rows, each of said rows corresponding to one of a plurality of documents in a second language;

applying an algorithm to the first matrix and the second matrix to produce a translingual text representation, wherein the translingual text representation comprises a plurality of vectors, each vector corresponding to either one row in the first document-term matrix or one row in the second document-term matrix, wherein the algorithm;

minimizes the distance between pairs of translingual text representation vectors which correspond to a document in a first language and a document in a second language that is comparable to the document in the first language; and

,

maximizes the distance between pairs of translingual text representation vectors which do not correspond to a document in a first language and a document in a second language that is comparable to the document in the first language.

2. The method of claim 1, wherein the algorithm is an oriented principal component analysis algorithm.

3. The method of claim 2, further comprising:

splitting a first document-term matrix into N non-overlapping document-term submatrices for the first language

splitting a second document-term matrix into N non-overlapping document-term submatrices for the second language for each value i of N,

applying the algorithm to the ith document-term matrix for the first language and the ith document-term matrix for the second language to create a result that is an ith set of stored matrices;

storing the results which comprises an ith set of stored matrices that are within an ith instance of an algorithm.

applying the ith set of stored matrices to the ith submatrix of the first language create TTRs in the first language.

accumulating each of the TTR vectors as rows in an ith TTR submatrix for the first language.

applying the ith set of stored matrices to the ith submatrix of the second language, to produce an ith TTR submatrix for the second language;

combining all TTR submatrices for the first language into a TTR matrix for the first language, by appending columns together

combining all TTR submatrices for the second language into a TTR matrix for the second language;

applying the algorithm to the TTR matrices from the first and second language as if they were document-term matrices; and

producing and storing a final stored matrix.

4. The method of claim 1, wherein the algorithm comprises a Siamese network training algorithm.

5. The method of claim 1, wherein the algorithm comprises a neural network training algorithm.

6. The method of claim 1, wherein the entries in the first matrix further comprises how many times the first language term is in the first language data and wherein entries in the second matrix comprises how many times the second language term is in the second language data.

7. A computer storage medium comprising computer executable instructions for creating a trans-lingual text representation, the computer executable instructions comprising instructions for:

accepting first language data, wherein the first language data comprises a plurality of documents in a first language;

accepting second language data, wherein the second language data comprises a plurality of documents in a second language, wherein each document in a second language is comparable to a corresponding document in the first language;

creating a first document-term matrix from the first language data, comprising a plurality of rows, each of said row corresponding to one of a plurality of documents in a first language;

creating a second document-term matrix from the second language data, comprising a plurality of rows, each of said rows corresponding to one of a plurality of documents in a second language;

applying an algorithm to the first matrix and the second matrix to produce a translingual text representation, wherein the translingual text representation comprises a plurality of vectors, each vector corresponding to either one row in the first document-term matrix or one row in the second document-term matrix, wherein the algorithm;

minimizes the distance between pairs of translingual text representation vectors which correspond to a document in a first language and a document in a second language that is comparable to the document in the first language; and

,

maximizes the distance between pairs of translingual text representation vectors which do not correspond to a document in a first language and a document in a second language that is comparable to the document in the first language.

8. The computer storage medium of claim 7, wherein the algorithm is an oriented principal component analysis algorithm.

9. The computer storage medium of claim 8, further comprising computer executable instructions for:

splitting a first document-term matrix into N non-overlapping document-term submatrices for the first language

splitting a second document-term matrix into N non-overlapping document-term submatrices for the second language

for each value i of N;

applying the algorithm to the ith document-term matrix for the first language and the ith document-term matrix for the second language to create a result that is an ith set of stored matrices;

storing the results which comprises an ith set of stored matrices that are within an ith instance of an algorithm.

applying the ith set of stored matrices to the ith submatrix of the first language create TTRs in the first language.

accumulating each of the TTR vectors as rows in an ith TTR submatrix for the first language.

applying the ith set of stored matrices to the ith submatrix of the second language, to produce an ith TTR submatrix for the second language.

combining all TTR submatrices for the first language into a TTR matrix for the first language, by appending columns together

combining all TTR submatrices for the second language into a TTR matrix for the second language;

applying the algorithm to the TTR matrices from the first and second language as if they were document-term matrices; and

producing and storing a final stored matrix.

10. The computer storage medium of claim 7, wherein the algorithm comprises a Siamese network training algorithm.

11. The computer storage medium of claim 7, wherein the algorithm comprises a neural network training algorithm.

12. The computer storage medium of claim 7, wherein the entries in the first matrix further comprises how many times the first language term is in the first language data and wherein entries in the second matrix comprises how many times the second language term is in the second language data.

13. A computer system comprising a processor for executing computer executable instructions, a memory for assisting execution of the computer executable instructions and an input/output circuit, the computer executable instructions comprising instructions for

accepting first language data, wherein the first language data comprises a plurality of documents in a first language;

accepting second language data, wherein the second language data comprises a plurality of documents in a second language, wherein each document in a second language is comparable to a corresponding document in the first language;

creating a first document-term matrix from the first language data, comprising a plurality of rows, each of said row corresponding to one of a plurality of documents in a first language;

creating a second document-term matrix from the second language data, comprising a plurality of rows, each of said rows corresponding to one of a plurality of documents in a second language;

applying an algorithm to the first matrix and the second matrix to produce a translingual text representation, wherein the translingual text representation comprises a plurality of vectors, each vector corresponding to either one row in the first document-term matrix or one row in the second document-term matrix, wherein the algorithm;

minimizes the distance between pairs of translingual text representation vectors which correspond to a document in a first language and a document in a second language that is comparable to the document in the first language; and

,

maximizes the distance between pairs of translingual text representation vectors which do not correspond to a document in a first language and a document in a second language that is comparable to the document in the first language.

14. The computer system of claim 13, wherein the algorithm is an oriented principal component analysis algorithm.

15. The computer system of claim 14, further comprising computer executable instructions for:

splitting a first document-term matrix into N non-overlapping document-term submatrices for the first language

splitting a second document-term matrix into N non-overlapping document-term submatrices for the second language

for each value i of N;

applying the algorithm to the ith document-term matrix for the first language and the ith document-term matrix for the second language to create a result that is an ith set of stored matrices;

storing the results which comprises an ith set of stored matrices that are within an ith instance of an algorithm.

applying the ith set of stored matrices to the ith submatrix of the first language create TTRs in the first language.

accumulating each of the TTR vectors as rows in an ith TTR submatrix for the first language.

applying the ith set of stored matrices to the ith submatrix of the second language, to produce an ith TTR submatrix for the second language.

combining all TTR submatrices for the first language into a TTR matrix for the first language, by appending columns together

combining all TTR submatrices for the second language into a TTR matrix for the second language;

applying the algorithm to the TTR matrices from the first and second language as if they were document-term matrices; and

producing and storing a final stored matrix.

16. The computer system of claim 13, wherein the algorithm comprises a Siamese network training algorithm.

17. The computer system of claim 13, wherein the algorithm comprises a neural network training algorithm.

18. The computer system of claim 13, wherein the entries in the first matrix further comprises how many times the first language term is in the first language data and wherein entries in the second matrix comprises how many times the second language term is in the second language data.