Feature-augmented neural networks and applications of same
First Claim
Patent Images
1. A method performed using one or more processing devices, the method comprising:
- receiving a word input vector at an input layer of a neural network, the word input vector representing an individual word from an input sequence of words;
receiving a topic feature vector at the input layer of the neural network, the topic feature vector being separate from the word input vector and representing topics expressed in the input sequence of words;
using the neural network to generate an output vector at an output layer of the neural network based at least on the word input vector and the topic feature vector, wherein using the neural network includes, by a hidden layer of the neural network;
modifying the word input vector using a first learned matrix; and
modifying the topic feature vector using a second learned matrix that is separate from the first learned matrix,wherein the output vector represents a word probability given the word input vector and the topic feature vector; and
performing a natural language processing operation based at least on the word probability represented by the output vector.
2 Assignments
0 Petitions
Accused Products
Abstract
A system is described herein which uses a neural network having an input layer that accepts an input vector and a feature vector. The input vector represents at least part of input information, such as, but not limited to, a word or phrase in a sequence of input words. The feature vector provides supplemental information pertaining to the input information. The neural network produces an output vector based on the input vector and the feature vector. In one implementation, the neural network is a recurrent neural network. Also described herein are various applications of the system, including a machine translation application.
-
Citations
20 Claims
-
1. A method performed using one or more processing devices, the method comprising:
-
receiving a word input vector at an input layer of a neural network, the word input vector representing an individual word from an input sequence of words; receiving a topic feature vector at the input layer of the neural network, the topic feature vector being separate from the word input vector and representing topics expressed in the input sequence of words; using the neural network to generate an output vector at an output layer of the neural network based at least on the word input vector and the topic feature vector, wherein using the neural network includes, by a hidden layer of the neural network; modifying the word input vector using a first learned matrix; and modifying the topic feature vector using a second learned matrix that is separate from the first learned matrix, wherein the output vector represents a word probability given the word input vector and the topic feature vector; and performing a natural language processing operation based at least on the word probability represented by the output vector. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A system comprising:
-
at least one processing device; and at least one computer readable medium storing instructions which, when executed by the at least one processing device, cause the at least one processing device to; receive a word input vector at an input layer of a neural network, the word input vector representing an individual word from an input sequence of words; receive a topic feature vector at the input layer of the neural network, the topic feature vector being separate from the word input vector and representing topics expressed in the input sequence of words; use the neural network to generate an output vector at an output layer of the neural network based at least on the word input vector and the topic feature vector, wherein using the neural network includes, by a hidden layer of the neural network; modifying the word input vector using a first learned matrix; and modifying the topic feature vector using a second learned matrix that is separate from the first learned matrix, wherein the output vector represents a word probability given the word input vector and the topic feature vector; and perform a natural language processing operation based at least on the word probability represented by the output vector. - View Dependent Claims (9, 10, 11, 12, 13, 14)
-
-
15. At least one computer readable storage medium storing instructions which, when executed by at least one processing device, cause the at least one processing device to perform acts comprising:
-
receiving a word input vector at an input layer of a neural network, the word input vector representing an individual word from an input sequence of words; receiving a topic feature vector at the input layer of the neural network, the topic feature vector being separate from the word input vector and representing topics expressed in the input sequence of words; using the neural network to generate an output vector at an output layer of the neural network based at least on the word input vector and the topic feature vector, wherein using the neural network includes, by a hidden layer of the neural network; modifying the word input vector using a first learned matrix; and modifying the topic feature vector using a second learned matrix that is separate from the first learned matrix, wherein the output vector represents a word probability given the word input vector and the topic feature vector; and performing a natural language processing operation based at least on the word probability represented by the output vector. - View Dependent Claims (16, 17, 18, 19, 20)
-
Specification