CLASSIFIER-BASED SYSTEM COMBINATION FOR SPOKEN TERM DETECTION

• US 20140278390A1
• Filed: 03/12/2013
• Published: 09/18/2014
• Est. Priority Date: 03/12/2013
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Kingsbury, Brian E. D., Kuo, Hong-Kwang Jeff, Mangu, Lidia Luminita, Soltau, Hagen
• Granted Patent: US 9,477,753 B2
• Time in Patent Office: Days
• US Class Current: 704/232
• CPC Class Codes: G06F 16/60 of audio data
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/083 Recognition networks G10L15/142, G10L15/16 take precedence
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 25/54 for retrieval

Abstract

Systems and methods for processing a query include determining a plurality of sets of match candidates for a query using a processor, each of the plurality of sets of match candidates being independently determined from a plurality of diverse word lattice generation components of different type. The plurality of sets of match candidates is merged by generating a first score for each match candidate to provide a merged set of match candidates. A second score is computed for each match candidate of the merged set based upon features of that match candidate. The first score and the second score are combined to provide a final set of match candidates as matches to the query.

Claims

1. A method for processing a query, comprising:

determining a plurality of sets of match candidates for a query using a processor, each of the plurality of sets of match candidates being independently determined from a plurality of diverse word lattice generation components of different type;

merging the plurality of sets of match candidates by generating a first score for each match candidate to provide a merged set of match candidates;

computing a second score for each match candidate of the merged set based upon features of that match candidate; and

combining the first score and the second score to provide a final set of match candidates as matches to the query.

2. The method as recited in claim 1, wherein the plurality of diverse word lattice generation components of different type include at least one of a different type of acoustic model, decoding technique, audio segmentation, and training data type.

3. The method as recited in claim 2, wherein the acoustic model includes at least one of a Gaussian mixture model and a deep neural network.

4. The method as recited in claim 2, wherein the decoding technique includes at least one of static decoding and dynamic decoding.

5. The method as recited in claim 2, wherein the audio segmentation includes at least one of a first audio segmentation based on a speech state, a silence state and a non-transmission state, a second audio segmentation based on the speech state and a non-speech state, and a third audio segmentation based on the speech state and the non-speech state tuned for a desired result.

6. The method as recited in claim 2, wherein the data training type includes at least one of noisy data and a combination of noisy and clean data.

7. The method as recited in claim 1, wherein computing the second score includes training a classifier based upon at least one of keyword features and system features.

8. The method as recited in claim 7, wherein the keyword features include at least one of a number of phones and a number of words.

9. The method as recited in claim 7, wherein the system features include at least one of a score, a rank of the score, and a rank of a duration of the match candidate.

10. The method as recited in claim 1, wherein combining the first score and the second score includes multiplying the first score and the second score to provide a final score and selecting final scores to provide the final set of match candidates according to a predefined threshold.

11. A method for processing a query, comprising:

determining a plurality of sets of match candidates for a query using a processor, each of the plurality of sets of match candidates being independently determined from a plurality of diverse word lattice generation components of different type including at least one of acoustic model, decoding technique, audio segmentation, and training data type;

merging the plurality of sets of match candidates by generating a first score for each match candidate to provide a merged set of match candidates;

computing a second score for each match candidate of the merged set by training a classifier based upon at least one of keyword features and system features of that match candidate; and

combining the first score and the second score to provide a final set of match candidates as matches to the query.

12. The method as recited in claim 11, wherein keyword features include at least one of a number of phones and a number of words.

13. The method as recited in claim 11, wherein system features include at least one of a score, a rank of the score, and a rank of a duration of the match candidate.

14. The method as recited in claim 11, wherein combining the first score and the second score includes multiplying the first score and the second score to provide a final score and selecting final scores to provide the final set of match candidates according to a predefined threshold.

15. A computer readable storage medium comprising a computer readable program for processing a query, wherein the computer readable program when executed on a computer causes the computer to perform the steps of:

determining a plurality of sets of match candidates for a query using a processor, each of the plurality of sets of match candidates being independently determined from a plurality of diverse word lattice generation components of different type;

merging the plurality of sets of match candidates by generating a first score for each match candidate to provide a merged set of match candidates;

computing a second score for each match candidate of the merged set based upon features of that match candidate; and

combining the first score and the second score to provide a final set of match candidates as matches to the query.

16. A system for processing an audio query, comprising:

a keyword search engine module configured to determine a plurality of sets of match candidates for a query using a processor, each of the plurality of sets of match candidates being independently determined from a plurality of diverse word lattice generation components of different type stored on a computer readable storage medium;

a combination module configured to merge the plurality of sets of match candidates by generating a first score for each match candidate to provide a merged set of match candidates; and

a classifier module configured to compute a second score for each match candidate of the merged set based upon features of that match candidate,

the combination module further configured to combine the first score and the second score to provide a final set of match candidates as matches to the query.

17. The system as recited in claim 16, the plurality of diverse word lattice generation components of different type include at least one of a different type of acoustic model, decoding technique, audio segmentation, and training data type.

18. The system as recited in claim 17, wherein the acoustic model includes at least one of a Gaussian mixture model and a deep neural network.

19. The system as recited in claim 17, wherein the decoding technique includes at least one of static decoding and dynamic decoding.

20. The system as recited in claim 17, wherein the audio segmentation includes at least one of a first audio segmentation based on a speech state, a silence state and a non-transmission state, a second audio segmentation based on the speech state and a non-speech state, and a third audio segmentation based on the speech state and the non-speech state tuned for a desired result

21. The system as recited in claim 17, wherein the data training type includes at least one of noisy data and a combination of noisy and clean data

22. The system as recited in claim 16, wherein the classifier module is further configured to train a classifier based upon at least one of keyword features and system features.

23. The system as recited in claim 22, wherein the keyword features include at least one of a number of phones and a number of words.

24. The system as recited in claim 22, wherein the system features include at least one of a score, a rank of the score, and a rank of a duration of the match candidate.

25. The system as recited in claim 16, wherein the combination module is further configured to multiply the first score and the second score to provide a final score and select final scores to provide the final set of match candidates according to a predefined threshold.