Meta learning for question classification

• US 7,603,330 B2
• Filed: 04/24/2006
• Issued: 10/13/2009
• Est. Priority Date: 02/01/2006
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Swarup, Samarth, Gupta, Rakesh
• Primary Examiner(s): Vincent; David R
• Assistant Examiner(s): Brown, Jr.; Nathan H
• Application Number:
  US11/410,443
• Publication Number:
  US 20070203863A1
• Time in Patent Office: 1,268 Days
• Field of Search: 706/20, 706/25
• US Class Current: 706/20
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

A system and a method are disclosed for automatic question classification and answering. A multipart artificial neural network (ANN) comprising a main ANN and an auxiliary ANN classifies a received question according to one of a plurality of defined categories. Unlabeled data is received from a source, such as a plurality of human volunteers. The unlabeled data comprises additional questions that might be asked of an autonomous machine such as a humanoid robot, and is used to train the auxiliary ANN in an unsupervised mode. The unsupervised training can comprise multiple auxiliary tasks that generate labeled data from the unlabeled data, thereby learning an underlying structure. Once the auxiliary ANN has trained, the weights are frozen and transferred to the main ANN. The main ANN can then be trained using labeled questions. The original question to be answered is applied to the trained main ANN, which assigns one of the defined categories. The assigned category is used to map the original question to a database that most likely contains the appropriate answer. An object and/or a property within the original question can be identified and used to formulate a query, using, for example, system query language (SQL), to search for the answer within the chosen database. The invention makes efficient use of available information, and improves training time and error rate relative to use of single part ANNs.

Claims

1. A computer-implemented method for automatically classifying a first question, the method comprising:

receiving unlabeled audio or digital text data from an input module, said unlabeled audio or digital text data comprising data that is not previously associated with an expected answer;

automatically labeling said unlabeled audio or digital text data using a processor to produce first labeled audio or digital text data associating a first answer with the unlabeled audio or digital text data using a first artificial neural network, said first artificial neural network comprising a first set of weights, said first artificial neural network producing the first labeled audio or digital text data by performing one or more auxiliary tasks analyzing characteristics of said unlabeled audio or digital text data;

transferring said first set of weights to a second artificial neural network;

receiving second labeled audio or digital text data comprising a second question and a corresponding answer;

training said second artificial neural network with the processor using said second labeled audio or digital text data by modifying a second set of weights associated with the second artificial neural network responsive to the second labeled audio or digital text data and freezing the first set of weights;

receiving the first question from the input module; and

associating a question category with the first question using said second artificial neural network, said question category identifying a source for retrieving text data or audio data describing an answer corresponding to the first question.

2. The method of claim 1, further comprising the step of mapping the first question to a first database of a plurality of databases based on said question category.

3. The method of claim 1, wherein said automatically labeling comprises learning a predictive structure within said unlabeled audio or digital text data.

4. The method of claim 1, wherein said unlabeled audio or digital text data comprises a third question.

5. The method of claim 3, wherein said automatically labeling said unlabeled audio or digital text data comprises at least one of:

automatically generating a part-of-speech tag for said third question;

automatically predicting a wh-word from one or more remaining words in said third question; and

automatically predicting a syntactic relation-tuple between two words in said third question.

6. The method of claim 1, wherein said question category identifies a data source including text data or audio data describing one from the group consisting of:

facts-dates, facts-people, facts-other, response to object-property, response to person-action, implicit response to object-property, implicit response to person action, locations, observation-event-time, observation-object-status, observation-person-action, steps, web current events, and web weather.

7. The method of claim 1, further comprising the step of mapping said question category to a system query language (SQL) query.

8. The method of claim 1, wherein said first and second artificial neural networks each comprise an input layer, a hidden layer and an output layer.

9. The method of claim 1, wherein said training said second artificial neural network comprises forward activity and backward error propagation.

10. The method of claim 1, wherein said unlabeled audio or digital text data is contributed by human subjects via distributed data collection.

11. The method of claim 1, further comprising the step of freezing said first set of weights prior to transferring said first set of weights to said second artificial neural network.

12. The method of claim 2, further comprising the step of identifying at least one of an object and a property within said first question.

13. The method of claim 11, further comprising the step of querying said first database according to said object and said property.

14. The method of claim 1, wherein said second artificial neural network comprises a set of output nodes, and said question category is associated with the first question according to the strongest output node of said set of output nodes.

20. The method of claim 1, further comprising:

presenting the audio data or text data describing the answer corresponding to the first question identified by the question category associated with the first question using an output device.

21. The method of claim 1, wherein the source for retrieving text data comprises a database.

15. An apparatus for automatically classifying a first question, the apparatus comprising:

an input module configured to;

receive unlabeled audio or digital text data comprising data not previously associated with an expected answer,

receive first labeled audio or digital text data comprising a second question and a corresponding answer, and

receive the first question;

a processor module configured to;

automatically label said unlabeled audio or digital text data to produce second labeled audio or digital text data associating a first answer with the unlabeled audio or digital text data using a first artificial neural network, said first artificial neural network comprising a first set of weights and producing the second labeled audio or digital text data by performing one or more auxiliary tasks analyzing characteristics of said unlabeled audio or digital text data,

transfer said first set of weights to a second artificial neural network,

train said second artificial neural network using said first audio or digital text labeled data by modifying a second set of weights associated with the second artificial neural network responsive to the second audio or digital text labeled data and freezing the first set of weights, and

associate a question category with the first question using said second artificial neural network, said question category identifying a source for retrieving text data describing an answer corresponding to the first question.

16. The apparatus of claim 15, wherein said first and second artificial neural networks comprise a multipart artificial neural network.

22. The apparatus of claim 15, wherein the source for retrieving text data comprises a database.

17. An apparatus for automatically classifying a first question, the apparatus comprising:

means for receiving unlabeled audio or digital text data comprising data that is not previously associated with an expected answer;

means for automatically labeling said unlabeled audio or digital text data to produce first labeled audio or digital text data associating a first answer with the unlabeled audio or digital text data using a first artificial neural network, said first artificial neural network comprising a first set of weights, said first artificial neural network producing the first labeled audio or digital text data by performing one or more auxiliary tasks analyzing characteristics of said unlabeled audio or digital text data;

means for transferring said first set of weights to a second artificial neural network;

means for receiving second labeled data comprising a second question and a corresponding answer;

means for training said second artificial neural network using said second labeled audio or digital text data by modifying a second set of weights associated with the second artificial neural network responsive to the second labeled audio or digital text data and freezing the first set of weights;

means for receiving the first question; and

means for associating a question category with the first question using said second artificial neural network, said question category identifying a source for retrieving text data or audio data describing an answer corresponding to the first question.

18. The apparatus of claim 17, wherein said first and second artificial neural networks comprise a multipart artificial neural network.

23. The apparatus of claim 17, wherein the source for retrieving text data comprises a database.

19. A computer program product, comprising a computer-readable medium having computer program instructions embodied thereon to cause a computer processor to implement a method for automatically classifying a first question, the method comprising:

automatically labeling unlabeled audio or digital text data comprising data that is not previously associated with an expected answer to produce first labeled audio or digital text data associating a first answer with the unlabeled audio or digital text data using a first artificial neural network, said first artificial neural network comprising a first set of weights and producing the first labeled text data by performing one or more auxiliary tasks analyzing characteristics of said unlabeled audio or digital text data;

transferring said first set of weights to a second artificial neural network;

receiving second labeled audio or digital text data comprising a second question and a corresponding answer;

training said second artificial neural network using said second labeled audio or digital text data by modifying a second set of weights associated with the second artificial neural network responsive to the second labeled audio or digital text data and freezing the first set of weights;

receiving the first question; and

associating a question category with the first question using said second artificial neural network, said question category identifying a source for retrieving audio data or text data describing an answer corresponding to the first question.

24. The apparatus of claim 19, wherein the source for retrieving text data comprises a database.

25. A method of using a computer to perform the steps of:

receiving unlabeled audio or digital text data from an input module, said unlabeled audio or digital text data comprising data that is not previously associated with an expected answer;

automatically labeling said unlabeled audio or digital text data using a processor to produce first labeled audio or digital text data associating a first answer with the unlabeled audio or digital text data using a first artificial neural network, said first artificial neural network comprising a first set of weights, said first artificial neural network producing the first labeled audio or digital text data by performing one or more auxiliary tasks analyzing characteristics of said unlabeled audio or digital text data;

transferring said first set of weights to a second artificial neural network;

receiving second labeled audio or digital text data comprising a second question and a corresponding answer;

training said second artificial neural network with the processor using said second labeled audio or digital text data by modifying a second set of weights associated with the second artificial neural network responsive to the second labeled audio or digital text data and freezing the first set of weights; and

storing said second set of weights associated with the second artificial neural network and said first set of weights.