Covariance estimation for pattern recognition

• US 20070005355A1
• Filed: 07/01/2005
• Published: 01/04/2007
• Est. Priority Date: 07/01/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Tian, Ye, Zhou, Jian-Lai, Soong, Frank Kao-Ping
• Granted Patent: US 7,805,301 B2
• Time in Patent Office: Days
• US Class Current: 704/237
• CPC Class Codes: G06F 18/295 Markov models or related models, e.g. semi-Markov models; Markov random fields; Networks embedding Markov models
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit

Abstract

A reliable full covariance matrix estimation algorithm for pattern unit'"'"'s state output distribution in pattern recognition system is discussed. An intermediate hierarchical tree structure is built to relate models for product units. Full covariance matrices of pattern unit'"'"'s state output distribution are estimated based on all the related nodes in the tree.

Claims

1. A method of developing a pattern recognition model, the method comprising:

training a plurality of models with diagonal covariance matrices;

estimating a full covariance matrix for each of the plurality of models based on related models; and

replacing the diagonal covariance matrices of the plurality of models with the estimated full covariance matrices.

2. The method of claim 1 wherein the plurality of models includes hidden markov models.

3. The method of claim 1 and further comprising building a hierarchical tree structure of states from the plurality of models.

4. The method of claim 3 wherein building the hierarchical tree structure includes using one of a top-down and bottom-up covariance matrix clustering process.

5. The method of claim 3 wherein building the hierarchical tree structure includes using a decision tree based building process.

6. The method of claim 3 wherein the hierarchical tree structure includes nodes for each state and estimating the full covariance matrix of each node in the hierarchical tree structure is performed using children nodes of some of the nodes in the structure.

7. The method of claim 3 and further comprising using the hierarchical tree structure to estimate full covariance matrix for each of the plurality of models.

8. The method of claim 7 wherein the hierarchical tree structure includes a root node and children nodes associated with the root node and wherein estimating the full covariance matrix for each node includes using the nodes along an upward path from said node to the root node.

9. The method of claim 7 wherein estimating the full covariance matrix includes using one of linear combination and non-linear combination to develop a set of combination weights.

10. The method of claim 1 wherein estimating the full covariance matrix includes only compensating off-diagonal parameters of the full covariance matrix of each model.

11. The method of claim 9 wherein the combination weights are estimated using maximum likelihood criterion.

12. The method of claim 9 wherein the combination weights are estimated using one of non-linear numeric optimization and an expectation maximization algorithm.

13. A method of developing a pattern recognition model, comprising:

building a tree of nodes including a root node, middle nodes and leaf nodes;

estimating a full covariance matrix for the root node and the middle nodes; and

estimating combination weights to estimate the full covariance matrix for each leaf node based on related nodes in the tree.

14. The method of claim 13 wherein estimating the full covariance matrix for the root node and middle nodes includes using leaf nodes.

15. The method of claim 14 and further comprising identifying an upward path for each leaf node including the root node and at least one middle node.

16. The method of claim 15 and further comprising estimating a full covariance matrix for each leaf node using the corresponding upward path and the combination weights.

17. The method of claim 13 wherein the combination weights are estimated using linear combination.