LARGE SCALE UNSUPERVISED HIERARCHICAL DOCUMENT CATEGORIZATION USING ONTOLOGICAL GUIDANCE

• US 20120203752A1
• Filed: 02/08/2011
• Published: 08/09/2012
• Est. Priority Date: 02/08/2011
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Renders, Jean-Michel, Ha-Thuc, Viet
• Granted Patent: US 8,484,245 B2
• Time in Patent Office: Days
• US Class Current: 707/706
• CPC Class Codes: G06F 16/35 Clustering; Classification

Abstract

A classification method includes constructing queries from category descriptors representing categories of a taxonomy of hierarchically organized categories. The query constructed for a category c includes a query component based on descriptors of the category c and at least one query component based on descriptors of an ancestor or descendant category of the category c. A documents database is queried using the constructed queries to retrieve pseudo-relevant documents. Language models for the categories of the taxonomy are extracted from the pseudo-relevant documents by inferring a hierarchical topic model representing the taxonomy. An input document is classified by optimizing mixture weights of a weighted combination of categories of the hierarchical topic model respective to the input document.

Claims

1. A storage medium storing instructions executable by a processing device to perform a method comprising:

generating a hierarchical classifier for a taxonomy of hierarchically organized categories wherein each category is represented by one or more textual category descriptors, the hierarchical classifier being generated by a method including (i) constructing queries from the textual category descriptors representing the categories and querying a documents database using the constructed queries to retrieve pseudo-relevant documents and (ii) extracting language models comprising multinomial distributions over the words of a textual vocabulary for the categories of the taxonomy by inferring a hierarchical topic model representing the taxonomy from at least the pseudo-relevant documents; and

classifying an input document using the generated hierarchical classifier.

2. The storage medium as set forth in claim 1, wherein the classifying comprises:

inferring mixture weights for a mixture of the extracted language models to approximate the multinomial distribution of words of the input document over the words of the textual vocabulary.

3. The storage medium as set forth in claim 1, wherein the constructing of queries comprises:

constructing a query corresponding to each category of the taxonomy, the query for a category c of the taxonomy including;

a query component q(c) constructed from one or more textual category descriptors of the category c and

at least one of a query component q(c↑

) constructed from one or more textual category descriptors of an ancestor category c↑

of the category c and a query component q(c↓

) constructed from one or more textual category descriptors of a descendant category c↓

of the category c.

4. The storage medium as set forth in claim 1, wherein the constructing of queries comprises:

constructing a query corresponding to each category of the taxonomy, the query for a category of the taxonomy including a weighted combination of;

a query component constructed from one or more textual category descriptors of the category and

at least one contextual query component selected from a group consisting of a query component constructed from one or more textual category descriptors of an ancestor category of the category and a query component constructed from one or more textual category descriptors of a descendant category of the category.

5. The storage medium as set forth in claim 1, wherein the querying of a documents database comprises at least one of (i) inputting the constructed queries to an Internet search engine without any domain restriction, and (ii) inputting the constructed queries to an Internet search engine with a domain restriction.

6. The storage medium as set forth in claim 1, wherein the hierarchical topic model comprises a hierarchical extension of the latent Dirichlet allocation (LDA) generative model and the inferring of a hierarchical LDA generative model representing the taxonomy comprises performing Gibbs sampling.

7. The storage medium as set forth in claim 1, wherein the hierarchical topic model comprises a mixture model in which a document is represented as a mixture of the language models of a plurality of categories and their ancestor categories.

8. A method comprising:

constructing queries from category descriptors representing categories of a taxonomy of hierarchically organized categories;

querying a documents database using the constructed queries to retrieve pseudo-relevant documents;

extracting category profiles for the categories of the taxonomy from at least the pseudo-relevant documents by inferring a hierarchical topic model representing the taxonomy; and

classifying an input document by optimizing mixture weights of a weighted combination of categories of the hierarchical topic model respective to the input document;

wherein at least the constructing, extracting, and classifying operations are performed by a digital processing device.

9. The method as set forth in claim 8, wherein the category descriptors comprise textual descriptors, the querying constructs textual queries from the textual descriptors, the retrieved pseudo-relevant documents include textual content, and the language models comprise multinomial distributions over the words of a textual vocabulary.

10. The method as set forth in claim 9, wherein the classifying comprises:

classifying the input document based on comparison of the multinomial distribution of words of the input document over the words of the textual vocabulary with sub-trees of the hierarchical topic model.

11. The method as set forth in claim 8, wherein the constructing of queries comprises:

constructing for a category c of the taxonomy a query including a weighted combination of;

a query component q(c) constructed from one or more textual descriptors of the category c and

at least one of a query component q(c↑

) constructed from one or more textual descriptors of an ancestor category c↑

of the category c and a query component q(c↓

) constructed from one or more textual descriptors of a descendant category c↓

of the category c.

12. The method as set forth in claim 8, wherein the querying of a documents database comprises at least one of (i) inputting the constructed queries to an Internet search engine without any domain restriction, and (ii) inputting the constructed queries to an Internet search engine with a domain restriction.

13. The method as set forth in claim 8, wherein the hierarchical topic model comprises a hierarchical extension of the latent Dirichlet allocation (LDA) generative model.

14. The method as set forth in claim 13, wherein the inferring of a hierarchical LDA generative model representing the taxonomy comprises performing Gibbs sampling.

15. The method as set forth in claim 8, wherein the hierarchical topic model comprises a mixture model in which a document is represented as a mixture of the language models of a plurality of categories and their ancestor categories.

16. The method as set forth in claim 8, further comprising:

processing or archiving the input document based on an output of the classifying operation, wherein the processing or archiving is also performed by the digital processing device.

17. An apparatus comprising:

a digital processing device configured to generate a hierarchical classifier for a taxonomy of hierarchically organized categories wherein each category is represented by one or more category descriptors, the digital processing device generating the hierarchical classifier by a method including;

constructing queries for categories of the taxonomy of hierarchically organized categories, the query constructed for a category c of the taxonomy including a query component constructed from one or more textual descriptors of the category c and at least one of a query component constructed from one or more textual descriptors of an ancestor category of the category c and a query component constructed from textual descriptors of one or more descendant categories of the category c;

querying a documents database using the constructed queries to retrieve pseudo-relevant documents; and

extracting category profiles for the categories of the taxonomy from at least the pseudo-relevant documents by inferring a hierarchical topic model representing the taxonomy.

18. The apparatus as set forth in claim 17, wherein the digital processing device is further configured to classify an input document using the generated hierarchical classifier.

19. The apparatus as set forth in claim 17, wherein the digital processing device is further configured to classify an input document by inferring mixture weights for a mixture of the extracted category profiles that models the input document.

20. The apparatus as set forth in claim 17, wherein the category descriptors comprise textual category descriptors and the category profiles are language models comprising multinomial distributions over the words of a textual vocabulary.

21. The apparatus as set forth in claim 20, wherein digital processing device is further configured to classify an input document by inferring mixture weights for a weighted mixture of the language models that represents the multinomial distribution of words of the input document over the words of the textual vocabulary.

22. The apparatus as set forth in claim 17, wherein the querying of a documents database comprises inputting the constructed queries to an Internet search engine.

23. The apparatus as set forth in claim 17, wherein the hierarchical topic model comprises a hierarchical extension of the latent Dirichlet allocation (LDA) generative model and the inferring employs a Gibbs sampling algorithm.

24. The apparatus as set forth in claim 17, wherein the hierarchical topic model comprises a mixture model in which a document is represented as a mixture of the category profiles of a plurality of categories and their ancestor categories.

25. The apparatus as set forth in claim 17, wherein the extracting comprises:

extracting category profiles for the categories of the taxonomy from the pseudo-relevant documents augmented by a sparse set of manually labeled documents by inferring a hierarchical topic model representing the taxonomy.