DEEP LEARNING FOR SEMANTIC PARSING INCLUDING SEMANTIC UTTERANCE CLASSIFICATION

• US 20150310862A1
• Filed: 04/24/2014
• Published: 10/29/2015
• Est. Priority Date: 04/24/2014
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hakkani-Tur, Dilek Z., Tur, Gokhan, Heck, Larry Paul, Dauphin, Yann Nicolas
• Application Number:
  US14/260,419
• Publication Number:
  US 20150310862A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G10L 15/16 using artificial neural networks
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

One or more aspects of the subject disclosure are directed towards performing a semantic parsing task, such as classifying text corresponding to a spoken utterance into a class. Feature data representative of input data is provided to a semantic parsing mechanism that uses a deep model trained at least in part via unsupervised learning using unlabeled data. For example, if used in a classification task, a classifier may use an associated deep neural network that is trained to have an embeddings layer corresponding to at least one of words, phrases, or sentences. The layers are learned from unlabeled data, such as query click log data.

Claims

1. A method, comprising, performing a semantic parsing task, including providing feature data representative of input data to a semantic parsing mechanism, in which a model used by the semantic parsing mechanism comprises a deep model trained at least in part via unsupervised learning using unlabeled data, and receiving output from the semantic parsing mechanism in which the output corresponds to a result of performing the semantic parsing task.

2. The method of claim 1 wherein the input data corresponds to an utterance and wherein the semantic parsing mechanism comprises a classifier that uses the model, and further comprising classifying the input data into a class to generate the output.

3. The method of claim 1 wherein the input data corresponds to a class and at least one of a word, phrase or sentence, and wherein performing the semantic parsing task comprises determining relationship information between the class at least one of the word, phrase or sentence.

4. The method of claim 1 further comprising, training the model, including extracting features from a dataset.

5. The method of claim 4 wherein the model comprises a deep network, and further comprising using at least some of the features to generate embeddings of the deep network.

6. The method of claim 1 wherein the unlabeled data is obtained from one or more query click logs, and further comprising, training the model, including extracting features corresponding to a distribution of click rates among a set of base Uniform Resource Locators (URLs).

7. The method of claim 6 further comprising, selecting the set of base URLs for a specific domain.

8. The method of claim 1 further comprising, training the model, including computing features based upon zero-shot discriminative embedding.

9. The method of claim 8 wherein computing the features based upon zero-shot discriminative embedding comprising learning an embedding space and optimizing an entropy measure.

10. A system comprising, a classifier and associated deep network, the deep network trained to have an embeddings layer corresponding to at least one of words, phrases, or sentences, the embeddings layer learned at least in part from unlabeled data, the classifier coupled to a feature extraction mechanism to receive feature data representative of input text from the feature extraction mechanism, and the classifier configured to classify the input text as a result set comprising classification data.

11. The system of claim 10 further comprising a speech recognizer that converts an input utterance into the input text.

12. The system of claim 10 wherein the unlabeled data is obtained from at least one query click log.

13. The system of claim 12 wherein a classification layer in the deep network is based upon continuous value features extracted from the at least one query click log, including a click rate distribution.

14. The system of claim 12 wherein the embeddings layer is based upon data extracted from queries in the at least one query click log.

15. The system of claim 10 wherein the classifier comprises a support vector machine.

16. The system of claim 10 wherein the classifier is coupled to provide the result set to a personal assistant application.

17. One or more computer-readable storage devices or machine logic having executable instructions, which when executed perform steps, comprising, classifying textual input data into a class, including determining feature data representative of the textual input data, providing the feature data to a classifier, in which a model used by the classifier comprises a deep network trained at least in part on unlabeled data, and receiving a result set comprising a semantic class from the classifier.

18. The one or more storage devices or machine logic of claim 17 wherein the unlabeled data comprises query and URL click data for a set of base URLS, and further comprising, using a click rate distribution as feature data in training.

19. The one or more computer-readable storage devices or machine logic of claim 17 having further instructions comprises receiving the textual input data as converted from a spoken utterance.

20. The one or more computer-readable storage devices or machine logic of claim 17 further comprising, training the model, including computing features based upon zero-shot discriminative embedding.