MINIMUM DIVERGENCE BASED DISCRIMINATIVE TRAINING FOR PATTERN RECOGNITION

• US 20080243503A1
• Filed: 03/30/2007
• Published: 10/02/2008
• Est. Priority Date: 03/30/2007
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zhang, Dongmei, Liu, Peng, Zhou, Jian-Iai, Soong, Frank Kao-Ping
• Application Number:
  US11/694,375
• Publication Number:
  US 20080243503A1
• Time in Patent Office: Days
• US Class Current: 704/244
• CPC Class Codes: G10L 15/063 Training

Abstract

A method of providing discriminative training of a speech recognition unit is discussed. The method includes receiving an acoustic indication of an utterance having a hypothesis space and comparing the hypothesis space against a reference. The method measures the Kullback-Leibler Divergence (KLD) between the reference and the hypothesis space to adjust the reference and stores the adjusted reference on a tangible storage medium.

Claims

1. A method of providing discriminative training of a speech recognition unit, comprising:

receiving an acoustic indication of an utterance having a hypothesis space;

comparing the hypothesis space against a reference;

measuring the Kullback-Leibler Divergence (KLD) between the reference and the hypothesis space to adjust the reference; and

storing the adjusted reference on a tangible storage medium.

2. The method of claim 1, and further comprising:

smoothing the minimum divergence based discriminative training by interpolating between the minimum divergence and a maximum likelihood calculation.

3. The method of claim 2, wherein interpolating between the divergence and a maximum likelihood includes applying a smoothing constant.

4. The method of claim 1, wherein measuring the KLD includes employing a forward-backward algorithm.

5. The method of claim 1, wherein comparing the hypothesis space against a reference comprises:

calculating a posterior probability.

6. The method of claim 1, wherein comparing the hypothesis space against a reference comprises:

calculating a gain function indicative of an accuracy measure of the hypothesis space given the reference.

7. The method of claim 6 wherein calculating the gain function includes calculating an indication of the acoustic similarity of the hypothesis space given the reference.

9. The method of claim 1, wherein receiving the acoustic indication includes receiving a plurality of Hidden Markov Models.

8. The method of claim it wherein adjusting the reference includes adopting an Extended Baum-Welch algorithm to update a parameter.

10. A method of automatically recognizing a pattern, comprising:

receiving pattern training data configured to train a pattern recognition model;

aligning the acoustic training data with a portion of the pattern recognition model;

calculating a gain indicative of a similarity between the pattern training data and the pattern recognition model;

adjusting the pattern recognition model to account for the pattern training data; and

providing the adjusted pattern recognition model to a pattern recognition application stored on a tangible computer medium

11. The method of claim 10, wherein receiving pattern data includes receiving speech pattern data configured to train an acoustic speech recognition model.

12. The method of claim 10, wherein calculating a gain includes calculating a Kullback-Leibler Divergence (KLD) between a portion of pattern training data and the recognition model.

13. The method of claim 10, wherein calculating a gain includes employing a forward-backward algorithm over a portion of the pattern training data.

14. The method of claim 10 and further comprising:

employing a smoothing algorithm by applying a constant indicative of a maximum likelihood statistic to adjust the calculated gain.

15. The method of claim 14, wherein employing the smoothing algorithm includes interpolating between the maximum likelihood statistic and the gain.

16. A pattern recognition system configured to train a model having a plurality of parameters, comprising:

a data store located on a tangible computer medium and configured to accept pattern training data;

a discriminative training engine configured to receive an observation and compare the observation with a portion of the pattern training data; and

wherein the discriminative training engine is configured to employ a minimum divergence based discriminative training algorithm to modify the pattern training data.

17. The system of claim 16, wherein the discriminative training engine is configured to calculated a KLD between a portion of the pattern training data and the observation.

18. The system of claim 16 and further comprising:

an application module configured to access the pattern training data.

19. The system of claim 16, wherein the pattern training data includes a plurality of Hidden Markov Models.

20. The system of claim 16, wherein the discriminative training engine is configured to apply a smoothing algorithm to the pattern training data.