Calculating cost measures between HMM acoustic models

• US 8,234,116 B2
• Filed: 08/22/2006
• Issued: 07/31/2012
• Est. Priority Date: 08/22/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Liu, Peng, Zhou, Jian-Lai, Kao-Ping K. Soong, Frank
• Primary Examiner(s): Hudspeth, David R
• Assistant Examiner(s): KOVACEK, DAVID M
• Application Number:
  US11/507,859
• Publication Number:
  US 20080059184A1
• Time in Patent Office: 2,170 Days
• Field of Search: 704200-201, 704203-204, 704/231, 704236-240, 704/243, 704246-257, 704/258, 704/266, 704/269, 704E17006-E17012, 704E15018-E15035
• US Class Current: 704/256.3
• CPC Class Codes: G10L 15/142 Hidden Markov Models [HMMs]

Abstract

Measurement of Kullback-Leibler Divergence (KLD) between hidden Markov models (HMM) of acoustic units utilizes an unscented transform to approximate KLD between Gaussian mixtures. Dynamic programming equalizes the number of states between HMMs having a different number of states, while the total KLD of the HMMs is obtained by summing individual KLDs calculated by state pair by state pair comparisons.

Claims

1. A computer-implemented method for measuring the total Kullback-Leibler Divergence (KLD) of two hidden Markov models (HMM), the two HMMs including a first HMM having a first number of states and a second HMM having a second, different number of states, the method comprising:

modifying both the first number of states in the first HMM and the second number of states in the second HMM to equalize the first number and the second number of states, wherein modifying includes performing a series of operations relative to a plurality of states of the first HMM and the second HMM comprising;

for each of the plurality of states, identifying penalty values associated with one or more possible modification operations relative to the state, the one or more possible modification operations being taken from a set of modification operations including deleting a state of at least one of the HMMs and substituting a state of at least one of the HMMs;

selecting one or more of the possible modification operations that equalizes the number of states of the two HMMs and minimizes a total of the penalty values; and

performing the one or more selected modification operations to equalize the number of states of the two HMMs;

after modifying the first and the second HMMs to equalize the number of states, calculating an individual KLD for each pair of states, state by state, for the two HMMs, using a processor of a computer;

summing the individual KLDs together to obtain a total KLD for the two HMMs; and

outputting the total KLD.

2. The method of claim 1 wherein the step of modifying includes assigning an associated penalty value to the modification.

3. The method of claim 2 wherein the step of modifying includes modifying at least one of the HMMs in a manner to minimize the penalty value.

4. The method of claim 1 wherein modifying includes inserting a state in one of the HMM models.

5. The method of claim 1 wherein modifying includes deleting a state in one of the HMM models.

6. The method of claim 1 wherein modifying includes substituting a state for one of the states in one of the HMM models.

7. The method of claim 1 wherein modifying includes inserting a non-skippable short pause (sp) state in one of the HMM models.

8. A computer readable medium having instructions which when executed by a computer perform a method for measuring the total Kullback-Leibler Divergence (KLD) of two hidden Markov models (HMM), the two HMMs including a first HMM having a first number of states and a second HMM having a second, different number of states, the method comprising:

modifying at least one of the HMMs to equalize the first number of states and the second number of states, wherein modifying includes performing a series of operations to at least one of the HMMs, the operations including duplicating a state of one of the first and the second HMMs such that the HMM of the one of the first and the second HMMs includes the state and a copy of the state;

calculating an individual KLD using a processor of the computer for each pair of states, state by state, for the two HMMs; and

summing the individual KLDs together to obtain a total KLD for the two HMMs.

9. The computer readable medium of claim 8 wherein the step of modifying includes assigning an associated penalty value to the modification.

10. The computer readable medium of claim 9 wherein the step of modifying includes modifying at least one of the HMMs in a manner to minimize the penalty value.

11. The computer readable medium of claim 10 wherein the operations further include inserting a state in one of the HMM models.

12. The computer readable medium of claim 10 wherein the operations further include deleting a state in one of the HMM models.

13. The computer readable medium of claim 10 wherein the operations further include substituting a state for one of the states in one of the HMM models.

14. The computer readable medium of claim 10 wherein the operations further include inserting a non-skippable short pause (sp) state in one of the HMM models.

15. A system for comparing a pair of acoustic hidden Markov models (HMMs), the system comprising:

a first acoustic hidden Markov model (HMM) having a first number of states;

a second acoustic HMM having a second number of states, the second number being different than the first number; and

a KLD (Kullback-Leibler Divergence) calculating module configured to;

receive the first and second acoustic HMMs;

modify at least one of the first and second acoustic HMMs to equalize the first and second number of states, wherein the KLD calculating module is configured to, for each of the first and second number of states, calculate penalty values for possible state modification operations, wherein the KLD calculating module is configured to select and perform one or more of the possible state modification operations that both equalizes the number of states of the first and second acoustic HMMs and minimizes a total of the penalty values, wherein one of the one or more possible state modification operations includes duplicating a state of one of the first and the second acoustic HMMs such that the HMM of the one of the first and the second acoustic HMMs includes the state and a copy of the state;

calculate the KLD between the first and second acoustic HMMs using a processor of a computer by calculating the KLD from individual KLDs for each pair of states, state by state, for the first and second acoustic HMMs, the KLD calculating module further configured to provide as an output at least one of the calculated KLD and information related to sorting of the first and second acoustic HMMs.

16. The system of claim 15 and further comprising an application processing module configured to receive the output and perform a task related to processing speech units.