Learning Discriminative Projections for Text Similarity Measures

• US 20120323968A1
• Filed: 06/14/2011
• Published: 12/20/2012
• Est. Priority Date: 06/14/2011
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Yih, Wen-tau, Toutanova, Kristina N., Meek, Christopher A., Platt, John C.
• Application Number:
  US13/160,485
• Publication Number:
  US 20120323968A1
• Time in Patent Office: Days
• US Class Current: 707/780
• CPC Class Codes: G06F 16/31 Indexing; Data structures therefor; Storage structures

Abstract

A model for mapping the raw text representation of a text object to a vector space is disclosed. A function is defined for computing a similarity score given two output vectors. A loss function is defined for computing an error based on the similarity scores and the labels of pairs of vectors. The parameters of the model are tuned to minimize the loss function. The label of two vectors indicates a degree of similarity of the objects. The label may be a binary number or a real-valued number. The function for computing similarity scores may be a cosine, Jaccard, or differentiable function. The loss function may compare pairs of vectors to their labels. Each element of the output vector is a linear or non-linear function of the terms of an input vector. The text objects may be different types of documents and two different models may be trained concurrently.

Claims

1. A method performed on at least one processor for optimizing model parameters, comprising:

mapping raw text representations of text objects to a compact vector space using the model parameters;

computing similarity scores based upon compact vectors for two text objects;

calculating error values using a loss function operating on the computed similarity scores and labels associated with pairs of text objects; and

adjusting the model parameters to minimize the error values.

2. The method of claim 1, wherein the raw text representation is a term-level feature vector or a collection of terms associated with a weighting value.

3. The method of claim 1, wherein the labels are either binary numbers or real-valued numbers, and the numbers indicate a degree of similarity of the pairs of text objects.

4. The method of claim 1, wherein the text objects are documents, and the method further comprising:

identifying pairs of similar documents in different languages based upon the similarity scores; and

use the pairs of similar documents in different languages to train a machine translation system.

5. The method of claim 1, wherein the text objects are documents, and the method further comprising:

detecting whether the documents are duplicates or near-duplicates based upon the similarity scores.

6. The method of claim 1, wherein the text objects are queries and advertisements, and the method further comprising:

judging relevance between the queries and the advertisements based upon the similarity scores.

7. The method of claim 1, wherein the text objects are queries and Web pages, and the method further comprising:

ranking the relevance of the Web pages to the queries based upon the similarity scores.

8. The method of claim 1, wherein the text objects are words, phrases, or queries, and the method further comprising:

measuring the similarity between the words, phrases, or queries based upon the similarity scores.

9. The method of claim 1, wherein a function for computing similarity scores is selected from a cosine function, a Jaccard function, or any differentiable function.

10. The method of claim 1, wherein the loss function comprises comparing the similarity score for a pair of vectors to a label associated with the pair of vectors.

11. The method of claim 1, wherein each element of the compact vector is a linear or non-linear function of all or a subset of elements of an input vector for the text object.

12. The method of claim 1, wherein each of the text objects in the pairs of text objects are of different types.

13. The method of claim 1, wherein two different sets of model parameters are trained concurrently.

14. A system, comprising:

a data storage device for storing model parameters for use in mapping raw text representations of text objects to a compact vector space;

a circuit for creating a compact vector using model parameters, the compact vector representing a text object;

a circuit for generating a similarity score by applying a similarity function to two compact vectors;

a circuit for applying a loss function to the similarity score and to a label, the label identifying a similarity of the text objects associated with the two compact vectors; and

a circuit for modifying the model parameters in a manner that minimizes an error value generated by the loss function.

15. The system of claim 14, wherein the label is either a binary number or a real-valued number.

16. The system of claim 14, wherein the similarity scores are generated using a function selected from a cosine function, a Jaccard function, or any differentiable function.

17. The system of claim 14, wherein the loss function comprises comparing the similarity score to the label.

18. The system of claim 14, wherein two different sets of model parameters are trained concurrently.

19. One or more computer-readable media having computer-executable instructions, which when executed perform steps, comprising:

mapping raw text representations of text objects to a compact vector space using the model parameters;

computing similarity scores based upon compact vectors for two text objects;

calculating error values using a loss function operating on the computed similarity scores and labels associated with pairs of text objects, wherein the labels indicate a degree of similarity of the pairs of text objects; and

adjusting the model parameters to minimize the error values.

20. The computer-readable media of claim 19, wherein a function for computing similarity scores is selected from a cosine function, a Jaccard function, or any differentiable function;

and

wherein the loss function comprises comparing the similarity score for a pair of vectors to a label associated with the pair of vectors.