ADDING PROTOTYPE INFORMATION INTO PROBABILISTIC MODELS

• US 20090076794A1
• Filed: 09/13/2007
• Published: 03/19/2009
• Est. Priority Date: 09/13/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Goldszmidt, Moises, Achan, Kannan, Ratinov, Lev
• Granted Patent: US 8,010,341 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 18/295 Markov models or related models, e.g. semi-Markov models; Markov random fields; Networks embedding Markov models
• CPC Class Codes: G06F 40/216 using statistical methods
• CPC Class Codes: G10L 15/142 Hidden Markov Models [HMMs]

Abstract

Mechanisms are disclosed for incorporating prototype information into probabilistic models for automated information processing, mining, and knowledge discovery. Examples of these models include Hidden Markov Models (HMMs), Latent Dirichlet Allocation (LDA) models, and the like. The prototype information injects prior knowledge to such models, thereby rendering them more accurate, effective, and efficient. For instance, in the context of automated word labeling, additional knowledge is encoded into the models by providing a small set of prototypical words for each possible label. The net result is that words in a given corpus are labeled and are therefore in condition to be summarized, identified, classified, clustered, and the like.

Claims

1. A system for processing information, comprising:

a module that is configured to apply a set of labels to a set of components using a probabilistic model;

a module that is configured to incorporate prototypical information in said probabilistic model by augmenting said probabilistic model with a conditional probability of the prototypical information; and

a module that is configured to determine whether said prototypical information is to be used in said probabilistic model based on a determination of at least one component in said set of components corresponding to a component in said prototypical information.

2. The system according to claim 1, wherein said set of components are words in a natural language.

3. The system according to claim 1, wherein said prototypical information includes a limited set of prototypical words for each possible label in said prototypical information.

4. The system according to claim 1, wherein said prototype information is preset to a specific distribution and a respective parameterization.

5. The system according to claim 1, wherein said prototype information has distribution and parameterization fitted automatically using standard statistical techniques.

6. The system according to claim 1, wherein said probabilistic model is a Hidden Markov Model.

7. The system according to claim 1, wherein said probabilistic model is a Latent Dirichlet Allocation Model.

8. A method for processing information, comprising:

configuring a probabilistic model to process a set of information;

adding prototype information to said probabilistic model, wherein said prototype information includes a set of labels and an associated set of components; and

assigning a first label to a first component of said set of information using said probabilistic model.

9. The method according to claim 8, wherein said set of information comprises a natural language.

10. The method according to claim 8, wherein said first component is at least one word.

11. The method according to claim 8, wherein said probabilistic model is a Hidden Markov Model.

12. The method according to claim 8, wherein said probabilistic model is a Latent Dirichlet Allocation Model.

13. The method according to claim 8, wherein at least one label from said set of labels is set to a specific value.

14. The method according to claim 8, wherein said first component is in said set of components.

15. A computer readable medium storing thereon computer executable instructions configured to process text, comprising:

an instruction configured to assign a label to a component according to a probabilistic model, wherein said probabilistic model includes incorporated prototype information;

an instruction configured to determine whether said component is included in said prototype information;

if said component is included in said prototype information, using said prototype information in said probabilistic model, otherwise not using said prototype information in said probabilistic model; and

an instruction configured to store said label assigned to said component.

16. The computer readable medium according to claim 15, wherein said probabilistic model includes at least one label configured to be assigned to said component, and wherein said at least one label is also configured to corresponds to a plurality of components.

17. The computer readable medium according to claim 15, further comprising an instruction configured to assign said label according to at least one additional probabilistic model.

18. The computer readable medium according to claim 15, wherein said component is similar but not exactly the same as a component in said prototype information.

19. The method according to claim 15, wherein said probabilistic model is a Hidden Markov Model.

20. The method according to claim 15, wherein said probabilistic model is a Latent Dirichlet Allocation Model.