System and method for identifying semantic intent from acoustic information

• US 20060129397A1
• Filed: 12/10/2004
• Published: 06/15/2006
• Est. Priority Date: 12/10/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Yu, Dong, Acero, Alejandro, Li, Xiao, Mahajan, Milind, Gunawardana, Asela J.
• Granted Patent: US 7,634,406 B2
• Time in Patent Office: Days
• US Class Current: 704/245
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules

Abstract

In accordance with one embodiment of the present invention, unanticipated semantic intents are discovered in audio data in an unsupervised manner. For instance, the audio acoustics are clustered based on semantic intent and representative acoustics are chosen for each cluster. The human then need only listen to a small number of representative acoustics for each cluster (and possibly only one per cluster) in order to identify the unforeseen semantic intents.

Claims

1. A method of processing acoustic information, comprising:

extracting a plurality of sets of acoustic information of interest from a data store;

performing speech recognition on the acoustic information to obtain speech recognition results;

clustering the sets of acoustic information into clusters based on a semantic analysis of the speech recognition results; and

identifying, for each cluster, a set of acoustic information as being representative of a corresponding cluster.

2. The method of claim 1 wherein clustering comprises:

clustering the sets of acoustic information together if the corresponding speech recognition results have a similar semantic analysis.

3. The method of claim 2 wherein the semantic analysis is indicative of semantic intent expressed by the speech recognition results and wherein clustering comprises:

clustering sets of acoustic information that have corresponding speech recognition results with similar semantic intent.

4. The method of claim 3 wherein clustering comprises:

initializing clusters based on lexical items in the speech recognition results;

assigning the speech recognition results to the initialized clusters;

merging similar clusters; and

re-assigning the speech recognition results to the clusters.

5. The method of claim 4 wherein initializing clusters comprises:

generating a cluster corresponding to each lexical item in the speech recognition results; and

computing a prior probability for each cluster based on a number of speech recognition results that include the lexical item corresponding to that cluster.

6. The method of claim 4 wherein assigning includes:

generating a language model corresponding to each cluster based on the speech recognition results assigned to that cluster.

7. The method of claim 6 wherein assigning comprises:

refining the assignment of speech recognition results to the clusters based on a probability that the language models corresponding to the clusters will produce the speech recognition results; and

re-generating the language models corresponding to the clusters.

8. The method of claim 4 wherein initializing comprises:

removing clusters corresponding to lexical items that appear an insufficient number of times in the speech recognition results.

9. The method of claim 4 wherein merging comprises:

computing a distance between a pair of clusters; and

merging the pair of clusters into one cluster if the computed distance meets a threshold distance; and

performing the steps of computing and merging for a plurality of pairs of clusters to obtain a refined set of clusters.

10. The method of claim 4 wherein merging comprises:

merging clusters until a defined auxiliary function changes by a predetermined amount.

11. The method of claim 4 wherein merging comprises:

iteratively merging clusters and re-assigning speech recognition results to clusters.

12. The method of claim 9 wherein re-assigning comprises:

re-assigning the speech recognition results to the refined set of clusters to obtain a set of refined clusters.

13. The method of claim 12 wherein clustering further comprises:

filtering the set of refined clusters based on a similarity of the speech recognition results to one another in each of the refined clusters, to obtain filtered clusters.

14. The method of claim 13 wherein filtering comprises:

removing clusters that have a compactness that exceeds a threshold compactness value.

15. The method of claim 13 and further comprising:

ranking the filtered clusters based on the speech recognition results assigned to the filtered clusters.

16. The method of claim 15 wherein identifying a set of acoustic information as being representative, comprises:

computing a distance in similarity between each speech recognition result assigned to a given cluster and other speech recognition results assigned to the given cluster; and

identifying a speech recognition result having a smallest distance in similarity to the other speech recognition results in the given cluster; and

identifying, as the representative set of acoustic information for the given cluster, the set of acoustic information corresponding to the identified speech recognition result.

17. The method of claim 1 wherein identifying acoustic information as being representative, comprises:

identifying the acoustic information based on a likelihood that speech represented by the acoustic information will be generated, given the corresponding cluster.

18. The method of claim 4 wherein extracting sets of acoustic information of interest comprises:

identifying acoustic information of interest stored by a human-machine interface application configured to process input speech indicative of semantic intent covered by the application.

19. The method of claim 18 and further comprising:

identifying the representative set of acoustic information for each cluster as representing either a semantic intent covered by the application or as an unrepresented semantic intent.

20. The method of claim 19 wherein the application processes the input speech using an application grammar and wherein performing speech recognition on the acoustic information comprises:

performing speech recognition using the application grammar and a second grammar;

identifying whether the speech recognition results are generated with the application grammar or the second grammar; and

wherein identifying the representative set of acoustic information for a given cluster as representing either a semantic intent covered by the application or as an unrepresented semantic intent, is performed based on a determination of whether the speech recognition results assigned to the given cluster were generated using the application grammar or the second grammar.

21. The method of claim 18 wherein the application comprises an automatic voice response application and wherein extracting comprises:

extracting stored call log information received by the application.

22. The method of claim 21 wherein the stored call log information indicates whether the call log information was generated from a failed call in which a caller prematurely hung up.

23. The method of claim 22 wherein identifying the representative set of acoustic information for each given cluster as representing either a semantic intent covered by the application or as an unrepresented semantic intent, based on whether the speech recognition results assigned to the given cluster correspond to acoustic information from failed calls.

24. The method of claim 22 wherein extracting comprises:

extracting only acoustic information generated from failed calls.

25. The method of claim 19 and further comprising:

generating a revision to a grammar used by the application to accommodate for unrepresented semantic intents

26. The method of claim 25 wherein generating a revision comprises:

generating a language model for each cluster; and

outputting, as the revision, the language model corresponding to clusters for which the representative set of acoustic information corresponds to an unrepresented semantic intent.

27. A system for processing acoustic information, comprising:

a clustering component configured to cluster sets of acoustic information, from an application, into clusters based on a semantic analysis of speech recognition results of speech recognition performed on the sets of acoustic information, and to identify, for each cluster, a set of acoustic information as being representative of a corresponding cluster.

28. The system of claim 27 wherein the semantic analysis is indicative of semantic intent expressed by the speech recognition results and wherein the clustering component is configured to cluster those sets of acoustic information that have corresponding speech recognition results with similar semantic intent.

29. The system of claim 28 wherein the clustering component comprises:

a language model-based clustering component configured to initialize clusters based on lexical items in the speech recognition results, assign the speech recognition results to the initialized clusters, generate a language model for each cluster based on the speech recognition results assigned to each cluster, and refine assignment of the speech recognition results to the clusters based on probabilities generated by the language models.

30. The system of claim 29 wherein the clustering component is configured to merge similar clusters with one another to obtain merged clusters, and to re-assigning the speech recognition results to the merged clusters.

31. The system of claim 29 wherein the clustering component is configured to initialize the clusters by generating a cluster corresponding to each lexical item in the speech recognition results, and computing a prior probability for each cluster based on a number of speech recognition results that include the lexical item corresponding to that cluster.

32. The system of claim 29 wherein the clustering component is configured to merge similar clusters by computing a distance between a pair of clusters, merging the pair of clusters into one cluster if the computed distance meets a threshold distance, and performing the steps of computing and merging for a plurality of pairs of clusters to obtain a refined set of clusters.

33. The system of claim 32 wherein the clustering component is configured to re-assign the speech recognition results to the refined set of clusters to obtain a set of refined clusters.

34. The system of claim 33 wherein the clustering component further comprises:

a filtering component configured to filter the set of refined clusters based on a similarity of the speech recognition results to one another in each of the refined clusters, to obtain filtered clusters.

35. The system of claim 34 wherein the clustering component is configured to identify a set of acoustic information as being representative by computing a distance in similarity between each speech recognition result assigned to a given cluster and other speech recognition results assigned to the given cluster, identify a speech recognition result having a smallest distance in similarity to the other speech recognition results in the given cluster, and identify, as the representative set of acoustic information for the given cluster, the set of acoustic information corresponding to the identified speech recognition result.

36. The system of claim 35 wherein the application processes input acoustic speech information based on a grammar and wherein the clustering component is further configured to identify the representative set of acoustic information for each cluster as representing either a semantic intent covered by the grammar or as an unrepresented semantic intent.

37. The system of claim 36 wherein the clustering component further comprises:

a grammar updating component configured to generate a revision to the grammar used by the application to accommodate for unrepresented semantic intents.

38. A computer readable medium storing instructions which, when executed by a computer, cause the computer to process acoustic information by performing steps of:

extracting a plurality of sets of acoustic information of interest from a data store;

performing speech recognition on the acoustic information to obtain speech recognition results;

clustering the sets of acoustic information into clusters based on a semantic analysis of the speech recognition results; and

identifying, for each cluster, a set of acoustic information as being representative of a corresponding cluster.

39. The computer readable medium of claim 38 wherein clustering comprises:

clustering the sets of acoustic information together if the corresponding speech recognition results have a similar semantic analysis.

40. The computer readable medium of claim 39 wherein the semantic analysis is indicative of semantic intent expressed by the speech recognition results and wherein clustering comprises:

clustering sets of acoustic information that have corresponding speech recognition results with similar semantic intent.