Method of text classification using discriminative topic transformation

• US 9,069,798 B2
• Filed: 05/24/2012
• Issued: 06/30/2015
• Est. Priority Date: 05/24/2012
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Hershey, John R., Le Roux, Jonathan
• Primary Examiner(s): Serrou, Abdelali
• Application Number:
  US13/479,656
• Publication Number:
  US 20130317804A1
• Time in Patent Office: 1,132 Days
• Field of Search: 704 1- 10, 704/245, 704/251, 704/257, 704/270
• US Class Current: 1/1
• CPC Class Codes: G06F 16/20 of structured data, e.g. relational data

Abstract

Text is classified by determining text features from the text, and transforming the text features to topic features. Scores are determined for each topic features using a discriminative topic model. The model includes a classifier that operates on the topic features, wherein the topic features are determined by the transformation from the text features, and the transformation is optimized to maximize the scores of a correct class relative to the scores of incorrect classes. Then, a class label with a highest score is selected for the text. In situations where the classes are organized in a hierarchical structure, the discriminative topic models apply to classes at each level conditioned on previous levels and scores are combined across levels to evaluate the highest scoring class labels.

Claims

1. A method for classifying text, comprising-steps of:

acquiring text as input data in a processor, wherein the text is derived from one or more hypotheses from an automatic speech recognition system operating on a speech signal;

determining text features from the text x, wherein the text features are ƒ

_j,k(x,y);

transforming the text features to topic features, wherein the transforming is according to g_l,k(x,y)=h_l(ƒ

_1,k(x,y), . . . ,ƒ

_J,k(x,y)),

where j is an index for a type of feature, k is an index of a class associated with the feature, y is a hypothesis of the class label, and h_l(•

) is a function that transforms the text features, and l is an index of the topic features;

determining scores from the topic features, wherein the determining steps use a model, wherein the model is a discriminative topic model comprising a classifier operating on the topic features, and the transforming is optimized to maximize the scores of a correct class relative to the scores of incorrect classes, wherein the discriminative topic model is

2. The method of claim 1, wherein the topic features are a linear transformation of the text features.

3. The method of claim 1, wherein parameters of the model are regularized using regularizers comprising L1-norm, L2-norm, and mixed-norm regularizers.

4. The method of claim 1, wherein the topic features relate to semantic aspects of the text.

5. The method of claim 1, wherein a linear transform

h_l(ƒ

_1,k(x,y), . . . ,ƒ

_J,k(x,y))=Σ

_jA_l,jƒ

_j,k(x,y)

is parameterized by a feature transformation matrix A to produces the topic features

6. The method of claim 1, wherein the weights are determined by cross-validation.

7. The method of claim 1, wherein the classifying is according to semantics of a natural language used by the text.

8. The method of claim 1, wherein the classes are organized in a hierarchical structure, wherein each class corresponds to a node in the hierarchy, wherein nodes are assigned to different levels of the hierarchy, wherein different classification parameters are used for one or more of the levels of the hierarchy, wherein classification is performed by traversing the hierarchy to evaluate partial scores of the classes at each level conditioned on hypotheses of the classes at previous levels, and combining the partial scores of the classes at one or more of the levels to determine a joint score.

9. The method of claim 8, wherein the hierarchy is represented as a tree.

10. The method of claim 8, wherein the hierarchy is represented as a directed acyclic graph.

11. The method of claim 8, wherein the hierarchy is traversed in a breadth-first manner.

12. The method of claim 8, wherein the scores at one or more levels are used to eliminate hypotheses from consideration at other levels.

13. The method of claim 12, wherein at a given level all but the highest scoring hypotheses are eliminated from further consideration.

14. The method of claim 12, wherein at a given level all but n highest scoring hypotheses are eliminated from further consideration, for some positive integer n.

15. The method of claim 8, wherein the joint score of a sequence of classes along a path from a top level to a class at another level is determined by summing the partial scores along the path.

16. The method of claim 15, wherein the score of the class at a particular level is determined by marginalizing the joint scores of all paths leading to the class.