NEURAL PARAPHRASE GENERATOR
First Claim
1. In a computer-based system comprising a processor in electrical communication with a memory, the memory adapted to store data and instructions for executing by the processor, a neural paraphrase generator comprising:
- an input adapted to receive a sequence of tuples (t=(t1, . . . , tn)) comprising a source sequence of words, each tuple (ti=(wi,pi)) comprising a word data element (wi) and a structured tag element (pi), the structured tag element representing a linguistic attribute about the word data element;
a recurrent neural network (RNN) comprising an encoder and a decoder, wherein the encoder is adapted to receive a sequence of vectors representing a source sequence of words, and the decoder is adapted to predict a probability of a target sequence of words representing a target output sentence based on a recurrent state in the decoder, a set of previous words and a context vector;
an input composition component connected to the input and comprising a word embedding matrix and a tag embedding matrix, the input composition component being adapted to receive and transform the input sequence of tuples into a sequence of vectors by
1) mapping the word data elements to the word embedding matrix to generate word vectors,
2) mapping the structured tag elements to the tag embedding matrix to generate tag vectors, and
3) respectively concatenating together the word vectors and the tag vectors; and
an output decomposition component connected to the decoder and adapted to output a target sequence of tuples representing predicted words and structured tag elements, wherein the probability of each single tuple from the output target sequence of tuples is predicted based on a recurrent state of the decoder.
3 Assignments
0 Petitions
Accused Products
Abstract
A neural paraphrase generator receives a sequence of tuples comprising a source sequence of words, each tuple comprising word data element and structured tag element representing a linguistic attribute about the word data element. An RNN encoder receives a sequence of vectors representing a source sequence of words, and RNN decoder predicts a probability of a target sequence of words representing a target output sentence based on a recurrent state in the decoder. An input composition component includes a word embedding matrix and a tag embedding matrix, and receives and transforms the input sequence of tuples into a sequence of vectors by 1) mapping word data elements to word embedding matrix to generate word vectors, 2) mapping structured tag elements to tag embedding matrix to generate tag vectors, and 3) concatenating word vectors and tag vectors. An output decomposition component outputs a target sequence of tuples representing predicted words and structured tag elements, the probability of each single tuple from the output is predicted based on a recurrent state of the decoder.
67 Citations
10 Claims
-
1. In a computer-based system comprising a processor in electrical communication with a memory, the memory adapted to store data and instructions for executing by the processor, a neural paraphrase generator comprising:
-
an input adapted to receive a sequence of tuples (t=(t1, . . . , tn)) comprising a source sequence of words, each tuple (ti=(wi,pi)) comprising a word data element (wi) and a structured tag element (pi), the structured tag element representing a linguistic attribute about the word data element; a recurrent neural network (RNN) comprising an encoder and a decoder, wherein the encoder is adapted to receive a sequence of vectors representing a source sequence of words, and the decoder is adapted to predict a probability of a target sequence of words representing a target output sentence based on a recurrent state in the decoder, a set of previous words and a context vector; an input composition component connected to the input and comprising a word embedding matrix and a tag embedding matrix, the input composition component being adapted to receive and transform the input sequence of tuples into a sequence of vectors by
1) mapping the word data elements to the word embedding matrix to generate word vectors,
2) mapping the structured tag elements to the tag embedding matrix to generate tag vectors, and
3) respectively concatenating together the word vectors and the tag vectors; andan output decomposition component connected to the decoder and adapted to output a target sequence of tuples representing predicted words and structured tag elements, wherein the probability of each single tuple from the output target sequence of tuples is predicted based on a recurrent state of the decoder. - View Dependent Claims (2, 5, 6, 7, 8, 9, 10)
-
- 3. The neural paraphrase generator of claim 3 wherein the attention module is further adapted to generate an attentional vector by concatenating the decoder state and the context vector.
Specification