INTEGRATION OF SEMANTIC CONTEXT INFORMATION

• US 20140278379A1
• Filed: 04/16/2013
• Published: 09/18/2014
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Coccaro, Noah B., Nguyen, Patrick An Phu
• Granted Patent: US 9,558,743 B2
• Time in Patent Office: Days
• US Class Current: 704/202
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 25/30 using neural networks

Abstract

In one implementation, a computer-implemented method includes receiving, at a computer system, a request to predict a next word in a dialog being uttered by a speaker; accessing, by the computer system, a neural network comprising i) an input layer, ii) one or more hidden layers, and iii) an output layer; identifying the local context for the dialog of the speaker; selecting, by the computer system and using a semantic model, at least one vector that represents the semantic context for the dialog; applying input to the input layer of the neural network, the input comprising i) the local context of the dialog and ii) the values for the at least one vector; generating probability values for at least a portion of the candidate words; and providing, by the computer system and based on the probability values, information that identifies one or more of the candidate words.

Claims

1. A computer-implemented method comprising:

receiving, at a computer system, a request to predict a next word in a dialog being uttered by a speaker;

accessing, by the computer system, a neural network comprising i) an input layer that includes a first portion representing a local context for the dialog and a second portion representing a semantic context for the dialog, ii) one or more hidden layers that are at least partially interconnected with the input layer by first connections, and iii) an output layer that represents a vocabulary of candidate words and that is at least partially interconnected with at least one of the one or more hidden layers by second connections;

identifying the local context for the dialog of the speaker;

selecting, by the computer system and using a semantic model, at least one vector that represent the semantic context for the dialog, the at least one vector including values for a plurality of dimensions;

applying input to the input layer of the neural network, the input comprising i) the local context of the dialog and ii) the values for the plurality of dimensions of the at least one vector that represents the semantic context of the dialog;

generating probability values for at least a portion of the candidate words in the vocabulary of the output layer based on propagation of the input through the neural network using, at least, the first connections and the second connections between layers of the neural network; and

providing, by the computer system and based on the probability values, information that identifies one or more of the candidate words.

2. The computer-implemented method of claim 1, wherein the local context includes either i) at least one preceding word from a current position in the dialog that was uttered by the speaker or ii) a start of sentence token that indicates that the next word is a first word being uttered by the speaker.

3. The computer-implemented method of claim 2, wherein:

the first portion of the input layer comprises a plurality of nodes that represent a plurality of words, and

applying the local context as part of the input to the input layer comprises providing a predetermined value to a portion of the plurality of nodes that correspond to the at least one preceding word that comprises the local context, the predetermined value indicating that a word is part of the local context for the dialog.

4. The computer-implemented method of claim 3, wherein the plurality of words comprise the vocabulary of candidate words.

5. The computer-implemented method of claim 1, wherein:

the first portion of the input layer comprises a plurality of first nodes that represent a plurality of words,

a particular hidden layer from the one or more hidden layers comprises a plurality of hidden nodes, and

a first portion of the first connections fully connect the plurality first nodes with the plurality of hidden nodes such that each of the plurality of first nodes is connected to each of the plurality of hidden nodes.

6. The computer-implemented method of claim 5, wherein:

the second portion of the input layer comprises a plurality of second nodes that represent the plurality of dimensions of the at least one vector, the input applied to the second nodes comprising the values for the plurality of dimensions, the values for the plurality of dimensions comprising centroids of at least a portion of the plurality of words in the semantic context for the dialog, and

a second portion of the first connections fully connect the plurality second nodes with the plurality of hidden nodes such that each of the plurality of second nodes is connected to each of the plurality of hidden nodes.

7. The computer-implemented method of claim 5, wherein:

the second portion of the input layer comprises a plurality of second nodes that represent the plurality of words, the input applied to the second nodes comprising values that represent distances from the plurality of words to the semantic context for the dialog that is represented by the at least one vector, and

a second portion of the first connections fully connect the plurality second nodes with the plurality of hidden nodes such that each of the plurality of second nodes is connected to each of the plurality of hidden nodes.

8. The computer-implemented method of claim 1, wherein:

a particular hidden layer from the one or more hidden layers comprises a plurality of hidden nodes,

the output layer comprises a plurality of output nodes that represent the vocabulary of candidate words, and

the second connections fully connect the plurality hidden nodes with the plurality of output nodes such that each of the plurality of hidden nodes is connected to each of the plurality of output nodes, each of the second connections including a weighting value that quantifies an association between two nodes and that is used to propagate the input values through the neural network.

9. The computer-implemented method of claim 1, wherein the semantic model comprises a latent semantic analysis (LSA) model.

10. The computer-implemented method of claim 1, wherein the semantic model comprises a latent dirichlet allocation (LDA) model.

11. The computer-implemented method of claim 1, further comprising:

obtaining an identity of the speaker;

selecting, using the identity of the speaker, one or more portions of text that were written by the speaker or that were transcribed from previous verbal communication by the speaker; and

generating at least a portion of the semantic context based on the selected one or more portions of text.

12. The computer-implemented method of claim 11, further comprising:

obtaining an identity of at least one other user with whom the speaker is having the dialog; and

wherein the one or more portions of text are selected further using the identity of the at least one other user such that the one or more portions of text were written or spoken to the at least one other user.

13. The computer-implemented method of claim 1, further comprising:

obtaining a transcription of one or more previous utterances spoken by the speaker or to the speaker by another during the dialog;

generating at least a portion of the semantic context based on the obtained transcription.

14. The computer-implemented method of claim 1, further comprising:

selecting the one or more candidate words from the vocabulary of candidate words based on the one or more candidate words having the greatest probability values from among the generated probability values.

15. A computer system comprising:

one or more computers;

an interface to the one or more computers that is programmed to receive a request to predict a next word in a dialog being uttered by a speaker;

a neural network module that is programmed to access a neural network comprising i) an input layer that includes a first portion representing a local context for the dialog and a second portion representing a semantic context for the dialog, ii) one or more hidden layers that are at least partially interconnected with the input layer by first connections, and iii) an output layer that represents a vocabulary of candidate words and that is at least partially interconnected with at least one of the one or more hidden layers by second connections;

a local context module that is programmed to identify the local context for the dialog of the speaker;

a semantic context generator that is programmed to select, using a semantic model, at least one vector that represent the semantic context for the dialog, the at least one vector including values for a plurality of dimensions; and

a probability generator that is programmed to;

apply input to the input layer of the neural network, the input comprising i) the local context of the dialog and ii) the values for the plurality of dimensions of the at least one vector that represents the semantic context of the dialog,

generate probability values for at least a portion of the candidate words in the vocabulary of the output layer based on propagation of the input through the neural network using, at least, the first connections and the second connections between layers of the neural network, and

provide, based on the probability values, information that identifies one or more of the candidate words.

16. The computer system of claim 15, wherein the local context comprises at least one preceding word from a current position in the dialog that was uttered by the speaker.

17. The computer system of claim 16, wherein:

the first portion of the input layer comprises a plurality of nodes that represent a plurality of words, and

applying the local context as part of the input to the input layer comprises providing a predetermined value to a portion of the plurality of nodes that correspond to the at least one preceding word that comprises the local context, the predetermined value indicating that a word is part of the local context for the dialog.

18. The computer system of claim 15, wherein the semantic model comprises a latent semantic analysis (LSA) model.

19. The computer system of claim 15, wherein the semantic model comprises a latent dirichlet allocation (LDA) model.

20. A computer program product embodied in a computer readable storage device storing instructions that, when executed, cause one or more computing devices to perform operations comprising:

receiving a request to predict a next word in a dialog being uttered by a speaker;

accessing a neural network comprising i) an input layer that includes a first portion representing a local context for the dialog and a second portion representing a semantic context for the dialog, ii) one or more hidden layers that are at least partially interconnected with the input layer by first connections, and iii) an output layer that represents a vocabulary of candidate words and that is at least partially interconnected with at least one of the one or more hidden layers by second connections;

identifying the local context for the dialog of the speaker;

selecting, using a semantic model, at least one vector that represent the semantic context for the dialog, the at least one vector including values for a plurality of dimensions;

applying input to the input layer of the neural network, the input comprising i) the local context of the dialog and ii) the values for the plurality of dimensions of the at least one vector that represents the semantic context of the dialog;

generating probability values for at least a portion of the candidate words in the vocabulary of the output layer based on propagation of the input through the neural network using, at least, the first connections and the second connections between layers of the neural network; and

providing, based on the probability values, information that identifies one or more of the candidate words.