System for identifying paraphrases using machine translation

• US 7,412,385 B2
• Filed: 11/12/2003
• Issued: 08/12/2008
• Est. Priority Date: 11/12/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Brockett, Christopher J., Dolan, William B., Quirk, Christopher B.
• Primary Examiner(s): Hudspeth; David R.
• Assistant Examiner(s): Rider; Justin W
• Application Number:
  US10/706,102
• Publication Number:
  US 20050102614A1
• Time in Patent Office: 1,735 Days
• Field of Search: 704/10, 704/245
• US Class Current: 704/245
• CPC Class Codes: G06F 40/253 Grammatical analysis; Style critique
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: Y10S 707/99931 Database or file accessing

Abstract

The present invention obtains a set of text segments from a cluster of different articles written about a common event. The set of text segments is then subjected to textual alignment techniques to identify paraphrases from the text segments in the text. The invention can also be used to generate paraphrases.

Claims

1. A method of training a paraphrase processing system, comprising:

i. accessing a plurality of documents;

ii. identifying, from the plurality of documents, a cluster of related texts that are written by different authors about a common subject, wherein the related texts are further identified as being from different news agencies and about a common event;

iii. receiving the cluster of related texts;

iv. selecting a set of text segments from the cluster, wherein selecting comprises grouping desired text segments of the related texts into a set of related text segments;

v. using textual alignment to identify paraphrase relationships between texts in the text segments included in the set of related text segments; and

vi. wherein textual alignment comprises;

using statistical textual alignment to align words in the text segments in the set; and

identifying the paraphrase relationships based on the aligned words.

2. The method of claim 1 and further comprising:

calculating an alignment model based on the paraphrase relationships identified.

3. The method of claim 2 and further comprising:

receiving an input text; and

generating a paraphrase of the input text based on the alignment model.

4. The method of claim 1 and wherein selecting a set of text segments comprises:

selecting text segments for the set based on a number of shared words in the text segments.

5. The method of claim 1 wherein identifying a cluster of related texts comprises identifying texts written within a predetermined time of one another.

6. The method of claim 1 wherein grouping desired text segments comprises grouping a first predetermined number of sentences of each news article in each cluster into the set of related text segments.

7. The method of claim 6 wherein selecting a set of text segments comprises:

pairing each sentence in a given set of related text segments with each other sentence in the given set.