ASSIGNMENT OF SEMANTIC LABELS TO A SEQUENCE OF WORDS USING NEURAL NETWORK ARCHITECTURES

• US 20150066496A1
• Filed: 09/02/2013
• Published: 03/05/2015
• Est. Priority Date: 09/02/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Deoras, Anoop, Yao, Kaisheng, He, Xiaodong, Deng, Li, Zweig, Geoffrey Gerson, Sarikaya, Ruhi, Yu, Dong, Hwang, Mei-Yuh, Mesnil, Gregoire
• Granted Patent: US 10,867,597 B2
• Time in Patent Office: Days
• US Class Current: 704/232
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/047 Probabilistic or stochastic networks
• CPC Class Codes: G10L 15/16 using artificial neural networks
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

Technologies pertaining to slot filling are described herein. A deep neural network, a recurrent neural network, and/or a spatio-temporally deep neural network are configured to assign labels to words in a word sequence set forth in natural language. At least one label is a semantic label that is assigned to at least one word in the word sequence.

Claims

1. A method executed by a computing device, the method comprising:

receiving a sequence of tokens;

responsive to receiving the sequence of tokens, retrieving respective pluralities of feature values for tokens in the sequence of tokens;

providing a plurality of feature values corresponding to a token in the sequence of tokens as input to at least one of a computer-implemented deep neural network (DNN) or a computer-implemented recurrent neural network (RNN), the at least one of the computer-implemented DNN or the computer-implemented RNN trained to assign labels to tokens, the labels comprising at least one semantic label;

receiving output of the at least one of the DNN or the RNN, the output indicative of a semantic label that is to be assigned to the token; and

assigning the semantic label to the token based upon the output of the at least one of the DNN or the RNN.

2. The method of claim 1, wherein the sequence of tokens is received from an automatic speech recognition system that is configured to recognize words in a spoken utterance, the sequence of tokens comprising the words in the spoken utterance.

3. The method of claim 1, wherein the plurality of feature values are provided to the DNN, the DNN comprising between three hidden layers and one hundred hidden layers.

4. The method of claim 1, wherein the computing device is one of a video game console or a mobile telephone.

5. The method of claim 1, wherein the computing device is in network communication with at least one of a video game console or a mobile telephone.

6. The method of claim 1, wherein the sequence of tokens comprises a sequence of words, and wherein the token in the sequence of tokens is a word.

7. The method of claim 6, wherein a plurality of feature values comprise:

the word;

another word in the sequence of words;

a named entity corresponding to the word;

part of speech of the word; and

a semantic tag assigned to the another word in the sequence of words.

8. The method of claim 1, wherein the plurality of features are provided to the RNN, the RNN configured to encode semantic data pertaining to previously received tokens.

9. The method of claim 8, the RNN being an Elman-style RNN.

10. The method of claim 8, the RNN being a Jordan-style RNN.

11. A computing device, comprising:

a processor; and

a memory that comprises a spoken language understanding (SLU) system executed by the processor, the SLU system comprising;

a labeler component that receives a sequence of words and assigns a semantic label to at least one word in the sequence of words, the labeler component comprising;

at least one of a deep neural network (DNN) or a recurrent neural network (RNN) trained to assign labels to words, the at least one of the DNN or the RNN receiving feature values corresponding to the at least one word and outputting data that is indicative of the semantic label that is to be assigned to the at least one word based upon the feature values; and

an output component that assigns the semantic label to the at least one word based upon the data output by the at least one of the DNN or the RNN.

12. The computing device of claim 11, wherein the labeler component comprises the DNN, the DNN comprising at least three hidden layers.

13. The computing device of claim 11, wherein the labeler component comprises the RNN, the RNN configured to encode feature values of words in the sequence of words that precede the at least one word.

14. The computing device of claim 11, the SLU system further comprising a semantic feature identifier component that identifies semantic feature values of the at least one word, the semantic feature values provided to the at least one of the DNN or the RNN as input, the data output by the at least one of the DNN or the RNN based upon the input.

15. The computing device of claim 11 being one of a video game console or a mobile telephone.

16. The computing device of claim 11 being a server computing device that is in network communication with a video game console or a mobile telephone, the memory further comprising an automatic speech recognition (ASR) system that is executed by the processor, the ASR system receives data indicative of a spoken utterance from a user of the video game console or the mobile telephone and outputs the sequence of words.

17. The computing device of claim 11, further comprising a search component that executes a search over contents of a data repository based at least in part upon the semantic label assigned to the at least one word.

18. The computing device of claim 11, wherein the labeler component comprises the RNN, the RNN being an Elman-style RNN.

19. The computing device of claim 11, wherein the labeler component comprises the RNN, the RNN being a Jordan-style RNN.

20. A computer-readable storage medium comprising instructions that, when executed by a processor, cause the processor to perform acts comprising:

receiving, from an automatic speech recognition system, a sequence of words;

responsive to receiving the sequence of words, identifying a plurality of features for at least one word in the sequence of words;

providing a plurality of features as input to at least one of a deep neural network (DNN) or a recurrent neural network (RNN), the at least one of the DNN or the RNN trained to assign labels to respective words;

receiving output from the at least one of the DNN or the RNN, the output being a probability distribution over a plurality of labels assignable to the at least one word; and

assigning a semantic label from amongst the plurality of labels to the at least one word based upon the probability distribution.